Techniques for detecting moving devices

Abstract

A mobile device may determine a set of range values ​​between a first mobile device and a second mobile device based on the difference between a first time and a second time for measuring the radio signal. The mobile device may determine first odometry information using a first sensor on the first mobile device, which indicates a first movement of the first mobile device over a certain period of time. The mobile device may receive second odometry information via a data channel, which is determined from second measurements captured using a second sensor on the second mobile device. The mobile device may use the set of range values ​​and the first and second odometry information to solve the angle between a first reference frame for the first device and a second reference frame for the second device in order to display a directional arrow pointing to the current position of the second mobile device.

Description

[Background technology]

[0001] People can use the navigation functions of their mobile devices to locate others (e.g., friends, family, colleagues). Determining a device's location is a fundamental problem in mobile computing. The importance and future potential of location-aware applications have led to the design and implementation of systems to provide location information, particularly in indoor and urban environments where Global Navigation Satellite Systems (GNSS), such as the Global Positioning System (GPS), do not function well. For example, GNSS systems may be inadequate in crowded locations (e.g., concert venues), indoors, underground, urban areas, or areas with dense foliage. Radio ranging systems have been developed that may be able to overcome some of the challenges in GNSS systems.

[0002] (Cross-reference to other applications) This application claims priority to U.S. Provisional Application No. 63 / 470,695, filed on 2 June 2023, relating to "TECHNIQUES FOR FINDING A DEVICE IN MOTION," and U.S. Patent Application No. 18 / 679,998, filed on 31 May 2024, relating to "TECHNIQUES FOR FINDING A DEVICE IN MOTION," both of which are incorporated herein by reference in their entirety for all purposes.

[0003] Location systems provide more accurate location information when a mobile device is stationary than when it is moving. Tracking a moving device is difficult because, if the device's position itself does not change during the averaging process, the inevitable errors that occur in the distance samples used to locate the device are more easily filtered out.

[0004] Therefore, improvements are desired in determining the position of one mobile device by a second mobile device when one or both devices are moving. [Overview of the project]

[0005] Various techniques are provided to enable a user of one device to detect a user of another device. For example, inertial odometry techniques (e.g., vision, inertial measurement unit (IMU), or both) may be used to locate one mobile device using a second mobile device. Various fusion techniques may also be used to combine information from GNSS-based navigation, ranging information, visual inertial odometry, and pedestrian dead reckoning, in order to improve techniques for locating devices, especially in challenging scenarios. Fusion techniques can calculate uncertainty values ​​and select information with the least uncertainty to generate pointers in location-based applications. Techniques for the efficient transfer of trajectory information between a first mobile device and a second mobile device are also disclosed.

[0006] In one general embodiment, for each of several ranging sessions occurring during a certain period, the technique may include transmitting a radio ranging signal at a first time. The technique may include receiving a radio response signal from a second mobile device at a second time. The technique may include determining a range value between the first mobile device and the second mobile device based on the difference between the first time and the second time, thereby determining a set of range values. The technique may include determining first odometry information from first measurements captured during that period using a first sensor on the first mobile device, the first odometry information indicating first movement of the first mobile device during that period. The technique may include receiving second odometry information via a data channel between the first mobile device and the second mobile device, determined from second measurements captured during the period using a second sensor on the second mobile device, the second odometry information indicating second movement of the second mobile device during that period. The technique may include solving for an angle between a first reference frame of a first device and a second reference frame of a second device using a set of range values, first odometry information, and second odometry information. The technique may also include displaying a directional arrow on the display of the first mobile device, pointing from the first current position of the first mobile device to the second current position of the second mobile device. Other embodiments of this aspect include a corresponding computer system, apparatus, and a computer program stored in one or more computer storage devices, each configured to perform the operations of the method.

[0007] In various embodiments, the second sensor may be an optical sensor, and the second sensor information may be odometry information. In various embodiments, the first and second devices are in motion. In various embodiments, the data channel may include a narrowband channel controlled by an ultra-wideband processing chip. In various embodiments, the second sensor may be an accelerometer, and the second sensor information may be acceleration information of the second mobile device. In various embodiments, solving the angle between the first reference frame of the first device and the second reference frame of the second device may involve using a least-squares equation based on at least range values, the first odometry information, and the second sensor information. In various embodiments, solving the angle between the first reference frame of the first device and the second reference frame of the second device may involve calculating the vertical displacement, horizontal displacement, and direction of travel offset between the first and second mobile devices. In various embodiments, the first odometry information may include visual inertial odometry information. In various embodiments, the data channel may include a narrowband channel controlled by an ultra-wideband processing chip. In various embodiments, the technique may include determining whether the relative position between devices is changing. The technique may further include suppressing the indication of direction from a first mobile device to a second mobile device based on an angle. Implementations of the described technique may include hardware, methods or processes, or computer media.

[0008] In one general embodiment, the technique may include transmitting a radio ranging signal at a first time. The technique may include receiving a radio response signal from a second mobile device at a second time. The technique may include determining a first range value between the first mobile device and the second mobile device based on the difference between the first time and the second time. The technique may include determining a first uncertainty in the first range value. The technique may include determining a first location of the first mobile device based on a first GNSS signal acquired by the first mobile device. The technique may include receiving a second location of the second mobile device based on a second GNSS signal via a data channel between the first mobile device and the second mobile device. The technique may include determining a second range value between the first location and the second location. The technique may include determining a second uncertainty in the second range value. The technique may include determining a position vector between a first mobile device and a second mobile device using a first range value, a first uncertainty, a second range value, and a second uncertainty. The technique may also include displaying a pointer based on the position vector. Other embodiments of this aspect include a corresponding computer system, apparatus, and a computer program stored in one or more computer storage devices, each configured to perform the operations of the method.

[0009] In one general embodiment, the technique may include storing a grid of reference points in a global reference frame. The technique may include determining a first location of the first mobile device within the global reference frame using measurements made by the first mobile device. The technique may include detecting a radio signal transmitted from a second mobile device. The technique may include determining the relative position between the first mobile device and the second mobile device based on the radio signal. The technique may include establishing a radio communication channel with the second mobile device. The technique may include receiving an offset value from the second mobile device via the radio communication channel, corresponding to the distance between the second mobile device and the first reference point among the reference points, where the offset value is measured by the second mobile device. The technique may include identifying a stored reference point in the grid of reference points corresponding to the first reference point, based on the first location of the first mobile device, the relative position between the first mobile device and the second mobile device, and the offset value. The technique may include determining a second location of a second mobile device based on a stored reference point and offset value. Other embodiments of this aspect include a corresponding computer system, apparatus, and a computer program stored in one or more computer storage devices, each configured to perform the operation of the method.

[0010] In various embodiments, the reference points are separated by at least a first threshold distance. In various embodiments, the first threshold distance is less than an offset value. In various embodiments, the grid coordinates are less than 5 bytes. The technique may include determining the second grid coordinates of the second device location. The technique may include determining the direction from the first mobile device to the second mobile device based at least on the location of the first mobile device and the location of the second mobile device in the global coordinate system, and displaying a graphical user interface indicating the determined direction. The technique may include determining whether the range between the first mobile device and the second mobile device is less than a predetermined distance. The technique may include determining the offset between the local coordinates of the second mobile device in the local coordinate system and the local coordinates of the second mobile device in the global coordinate system based on a plurality of defined reference points.

[0011] In one general embodiment, the technique may include determining first inertial odometry information from first inertial measurements captured over a first period using an inertial sensor on a first mobile device. The technique may also include identifying a first reference frame corresponding to the first inertial measurements. The technique may further include determining first visual odometry information from first visual measurements captured over a first period using a visual sensor on the first mobile device. The technique may also include identifying a second reference frame corresponding to the first visual measurements. The technique may further include determining a first transformation between the second reference frame and the first reference frame. The technique may also include determining the displacement of the first mobile device in the first reference frame over a first period using the first visual odometry information and the transformation. Other embodiments of these techniques include corresponding methods, computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the operations of the technique.

[0012] The implementation forms of the described techniques may include hardware, methods or processes, or computer tangible media. Other embodiments are directed to systems, portable consumer devices, and computer-readable media associated with the methods described herein.

[0013] A better understanding of the nature and advantages of the embodiments of the present disclosure can be obtained by referring to the following detailed description and the accompanying drawings.

Brief Description of the Drawings

[0014] [Figure 1] Shows a map of Balboa Park in San Diego, California.

[0015] [Figure 2A] Shows a first scenario where the device of the person to be explored is stationary or restricted or bounded within an area where movement is defined.

[0016] [Figure 2B] Shows a second scenario where the device of the person to be explored is not bounded by a boundary or is moving.

[0017] [Figure 3] Shows an exemplary geometry between the movement of the explorer device and the potential movement of the device of the person to be explored.

[0018] [Figure 4] Shows an exemplary geometry between the explorer device and the device of the person to be explored.

[0019] [Figure 5] Shows an exemplary source of uncertainty for calculating the location of the device of the person to be explored.

[0020] [Figure 6] Shows the relationship between the trajectory of the explorer device and the device of the person to be explored.

[0021] [Figure 7] This chart illustrates an exemplary particle filtering technique for locating mobile devices.

[0022] [Figure 8] This is a flowchart of a process for locating a mobile device, as an example of what is disclosed here.

[0023] [Figure 9A] Simplified diagrams of image frames in a congested environment are shown according to various embodiments. [Figure 9B] Simplified diagrams of image frames in a congested environment are shown according to various embodiments.

[0024] [Figure 10] This is a simplified diagram illustrating the conversion between reference frames according to various embodiments.

[0025] [Figure 11] This is a simplified diagram of an architecture for converting between reference frames, based on various embodiments.

[0026] [Figure 12] This is a simplified state diagram showing events that trigger the determination of conversions between reference frames, according to various embodiments.

[0027] [Figure 13] This is a flowchart of process 1300 for conversion between reference frames, as an example of this disclosure.

[0028] [Figure 14] This illustrates an exemplary scenario in which fusion techniques may be used to locate mobile devices.

[0029] [Figure 15] This is a flowchart of the process for integrating location techniques, as an example of this disclosure.

[0030] [Figure 16] This indicates the priority order for estimating pointers or arrows to locate the searched device using the searcher device.

[0031] [Figure 17] This is a flowchart of a process for detecting a mobile device using a fusion technique, as an example of the disclosure.

[0032] [Figure 18] An exemplary layout for geodetic displacement techniques is shown.

[0033] [Figure 19] This diagram shows a cross-functional flowchart for efficient position transfer between a first mobile device and a second mobile device.

[0034] [Figure 20] This is a flowchart of a process for efficiently transferring position information, as an example of what is disclosed here.

[0035] [Figure 21] A sequence diagram for performing distance measurement between two mobile devices according to an embodiment of the present disclosure is shown.

[0036] [Figure 22] The following is a sequence diagram of a ranging operation including a mobile device having three antennas according to an embodiment of the present disclosure.

[0037] [Figure 23] This is a block diagram of components of a mobile device operable to perform ranging according to an embodiment of the present disclosure.

[0038] [Figure 24] This is a block diagram of an exemplary device that could be a mobile device. [Modes for carrying out the invention]

[0039] Mobile device tracking is the process of identifying the location of a mobile device, whether it is stationary or moving. Techniques for locating a second mobile device (the one being tracked) using one mobile device (the tracker) generally assume that the second mobile device is stationary. Mobile device tracking becomes more difficult when both devices are moving.

[0040] There are several types of mobile devices that can be used for ranging techniques to estimate the distance or proximity to other devices or objects. These mobile devices may include smartphones, tablets, wearable devices, laptops and notebook computers, and Internet of Things (IoT) devices.

[0041] Various techniques are provided to enable a user of one device to detect a user of another device. For example, inertial odometry techniques (e.g., vision, inertial measurement unit (IMU), or both) and ranging may be used to locate one mobile device using a second mobile device. Various fusion techniques may also be used to combine information from GNSS-based navigation, ranging information, visual inertial odometry, and other motion sensors to improve techniques for locating devices, especially in challenging scenarios, such as during interference and when the device being searched is also moving. Fusion techniques can calculate uncertainty values ​​and select information with the least uncertainty to generate pointers in location-based applications. Techniques for the efficient transfer of position or trajectory information between a first mobile device and a second mobile device are also disclosed. I. Tasks using human detection techniques

[0042] Figure 1 shows a map of Balboa Park in San Diego, California. The park may have several explorer devices and a single explored device. The explorer devices may correspond to children's devices, and the explored device may be a parent's device. There may be a scenario where a child freely explores an area of ​​Balboa Park but wants to be able to locate their parent's mobile device. Another example might be schoolchildren on a field trip with their teacher. The schoolchildren may want to know where their teacher is in the park.

[0043] Various techniques can help the explorer's device determine the location of the explored mobile device by using a combination of location measurements (e.g., motion data, distance measurements such as time of flight, and pedestrian dead reckoning using GPS) on the explorer's mobile device and / or the explored's mobile device. Some implementations can generate arrows on the explorer's device's display that point to the explored. Exemplary techniques using motion data include visual inertial odometry (VIO) techniques and the use of internal measurement units (IMUs) such as accelerometers to determine the relative location of the explored device to the explorer's device. A. Use of stationary target techniques

[0044] Figure 2A shows a first scenario in which the device being searched is stationary or its movement is restricted or bounded within a defined area. The searcher device is not bounded. The searcher device can determine the distance between itself and the device being searched using wireless ranging techniques. The searcher device can measure its own movement using motion sensors, thus better understanding the range changes resulting from the searcher device's movement. Therefore, in the first scenario, the searcher device can locate the device being searched and generate an arrow using ranging techniques even if the searcher device is moving.

[0045] Figure 2B illustrates a second scenario in which both the explorer and the explored device are unbounded. In this second scenario, resolving the movement of both the explorer and the explored device can be more difficult. While the explorer device's movement can be measured using motion sensors, it is more difficult for the explorer device to understand the movement of the explored device. In this scenario, using the previous techniques, the explorer mobile device would be unable to generate a solution for locating the explored mobile device, especially if the explored device moves excessively.

[0046] When two mobile devices are moving, several factors can make it difficult to locate one device using the other. These factors can include signal interference, signal strength and range, dynamic environment, time synchronization, and tracking algorithms.

[0047] Signal Interference: Moving objects can cause signal interference between two devices. This movement can create obstacles such as walls, buildings, or other objects that obstruct the signal path. This interference can weaken or disrupt the wireless signal, making it difficult for devices to communicate effectively.

[0048] Signal Strength and Range: The signal strength and range of wireless communication technologies such as Wi-Fi, Bluetooth, or GPS can vary depending on the distance between the devices and their surroundings. If the devices are too far apart or the signal strength is weak, it may be difficult for one device to detect or locate the other.

[0049] Dynamic Environment: Moving devices often encounter changing environments. For example, if both devices are in a vehicle, they may pass through areas where signal availability changes, such as tunnels, areas with poor network coverage, or areas with high electromagnetic interference. These changes can affect the devices' ability to establish a reliable connection and locate each other.

[0050] Time Synchronization: For devices to accurately locate each other, they typically rely on precise time synchronization. Moving devices may experience different clock drift or synchronization problems due to fluctuations in their speed of movement or their internal clocks, which can lead to inaccuracies during the location tracking process.

[0051] Tracking Algorithms: Locating one device using another device typically involves complex tracking algorithms that consider factors such as signal strength, time delay, and triangulation. When both devices are moving, these algorithms must consider constant changes in relative position, velocity, and signal characteristics, which introduces additional complexity and can reduce the accuracy of the tracking process.

[0052] Overall, the combination of signal interference, fluctuating signal strength and range, dynamic environments, time synchronization issues, and complex tracking algorithms can make it difficult to locate one mobile device together with another when both devices are moving. B. Difficult Geometry

[0053] If the displacement of the device being investigated is known, the movement of the device being investigated can be taken into account. A potential problem exists if the geometry between the investigator device and the device being investigated does not change. This can occur if the investigator device follows a certain range behind the device being investigated.

[0054] Figure 3 shows an exemplary geometry between the movement of a seeker device and the movement of a potential device being sought. In Figure 3, the seeker device is moving along a first trajectory 308. The seeker device can calculate range information between itself and the device being sought. If the device being sought and the seeker device have the same trajectory, the device being sought can be in one of three locations: a first location 310 with a first trajectory 312, a second location 314 with a second trajectory 316, and a third location 318 with a third trajectory 320. If the range information is constant, the first range d1 302 can be equal to the second range d2 and the third range d3 306. In this scenario, due to the symmetry of the problem, it may be difficult to use the range information to resolve ambiguity regarding the location of the device being sought.

[0055] Potentially, other sensors, such as visual odometry sensors, may be able to clarify which of the device being searched's potential trajectories is correct.

[0056] The explorer device may determine this scenario exists if multiple potential solutions are found for the location of the device being explored. If the explorer device and the device being explored are moving in parallel, it may not be possible to calculate the bearing from the explorer device to the device being explored. II.Person detection techniques

[0057] There are several different techniques for using a first mobile device to locate a second mobile device. If the mobile devices have GNSS capabilities (e.g., GPS), one device can use GNSS positioning to determine its own location and share that information with the other device. The second device can then use this location information to navigate towards the first device or display it on a map.

[0058] Mobile devices can estimate the distance and direction between them using wireless signals such as Wi-Fi or Bluetooth. By measuring signal strength, or by using techniques such as Received Signal Strength Indication (RSSI), one device can estimate its proximity to another device.

[0059] In various embodiments, communication between two devices may be established through messaging or calls. When the devices are in contact, one device may request the other device to share its location or provide updates on its position.

[0060] Mobile devices with Bluetooth capabilities can detect nearby devices using Bluetooth Low Energy (BLE) technology. By scanning for nearby devices or using functions like beacons, one device can determine if another device is nearby.

[0061] Mobile devices can leverage network-based location services provided by cellular networks. By utilizing network infrastructure and triangulation techniques, the approximate location of a mobile device can be determined.

[0062] The availability and accuracy of these techniques may vary depending on factors such as device capabilities, network coverage, signal strength, and user permission. A. Determine the arrow pointing from the investigator to the person being investigated.

[0063] Mobile devices can determine the direction from the searcher's mobile device to the searched mobile device using various positioning techniques. Various technologies and techniques can be used to display location indications (e.g., arrows) to help users find lost or misplaced items and mobile devices. Mobile devices can determine their current coordinates using their location services module. This can be achieved using GPS, Wi-Fi positioning, cellular network data, or a combination of these methods.

[0064] A mobile device can calculate the location information of a user's target device (e.g., the mobile device being tracked). This information can typically be associated with the user's identifier and accessed through a location service application. By comparing the current location of the user's device with the location of the target device, the location application can calculate the relative position or orientation between the two devices. This calculation can use information such as distance, bearing, and orientation between the devices.

[0065] The calculated relative position can be visually represented as a pointer or arrow within the user interface of the location service application. Directional information (e.g., the pointer or arrow) can indicate the direction in which the user needs to move to approach the target device. The length and size of the directional information (e.g., the pointer or arrow) may vary based on the estimated distance or proximity to the target device.

[0066] Directional information (e.g., pointers or arrows) can typically be displayed on a map or graphical user interface to provide the user with a visual reference. Location service applications may also provide additional information, such as estimated distance, last known location, or other relevant details, to help the user locate a target device.

[0067] Directional information (e.g., pointers or arrows) and specific implementations of the underlying technologies may vary depending on the device platform, operating system, and the capabilities of the location application. Location service applications may incorporate other features, such as voice alerts, device pinging, or augmented reality overlays, to further assist in device discovery.

[0068] Figure 4 shows an exemplary geometry between the searcher device and the device being searched. The searcher device 402 can measure the range 404 (or distance) between the searcher device 402 and the device being searched 406 using ranging techniques (e.g., time-of-flight measurement). The range 404 information alone is not sufficient to determine the position of the device being searched 406. The searcher device 402 can calculate the bearing 408 between the searcher device 402 and the device being searched 406. In various embodiments, the searcher device 402 can use multiple antennas to receive radio signals (e.g., ranging signals) and calculate the time difference of the return signals to determine the bearing 408.

[0069] In various embodiments, the explorer device 402 can determine its direction of travel 410 using a mobile compass built into the explorer device 402. The explorer device 402 can determine its relative position to the device being explored 406, indicated by an arrow 412, by subtracting its direction of travel 410 from the bearing 408 relative to the device being explored 406. The arrow 412 can be displayed on the explorer device 402's display. B. Determining Uncertainty

[0070] Figure 5 illustrates exemplary sources of uncertainty for calculating the location of the device being searched. The uncertainty in the direction of arrow 412 may be a combination of other uncertainties. For example, there may be an azimuthal uncertainty 502. The azimuthal uncertainty 502 may depend on the range value. There may also be a direction of travel uncertainty 504. The direction of travel uncertainty 504 may result from an inaccurate measurement of the direction of the searcher device 402 using a mobile compass. Other uncertainty values ​​(e.g., range uncertainty) may also exist.

[0071] Location service applications can use uncertainty values ​​to determine which positioning technique should draw directional information (e.g., a pointer or arrow). For example, if the uncertainty value of the ranging technique is less than the uncertainty value of the GNSS position, the ranging technique can be used to generate directional information (e.g., a pointer or arrow). In various embodiments, weights may be applied to one or more of the positioning techniques. The weighted positions may be averaged to determine the position for determining the location of the directional information (e.g., a pointer or arrow).

[0072] Since various measurements (e.g., direction of travel and bearing) are used to calculate arrow 412, the arrow may also have an uncertainty value. For example, the arrow uncertainty can be calculated using the following formula.

number

[0073] Mobile devices possess various capabilities for locating other mobile devices. For example, GNSS-based techniques, ranging techniques, visual inertia techniques, RSSI techniques, and dead reckoning techniques can all be used to determine the locations of a first and second mobile device. The ranging technique can be combined with visual odometry techniques to determine the location of the second mobile device by solving the angle between the first and second reference frames by defining a reference plane with range values ​​determined by ranging. A. Visual / Inertial Odometry Techniques

[0074] Visual inertial odometry (VIO) is a sensor fusion technique used to estimate the pose (position and orientation) of a moving camera or object in a 3D environment by combining visual information from a camera with inertial measurements from an inertial measurement unit (IMU).

[0075] Visual inertial odometry techniques can leverage the complementary strengths of visual information and inertial measurements from a mobile device (e.g., a mobile device) to individually overcome the limitations of each sensor. Visual data can provide rich and detailed information about the environment, enabling accurate feature tracking and mapping. Meanwhile, IMUs can provide high-frequency motion measurements that can be robust to lighting conditions. IMU measurements can be used to estimate the camera's acceleration, angular velocity, and orientation.

[0076] The VIO system can extract distinctive visual features from camera images, such as corners, edges, or other prominent points. These features serve as reference points for tracking and mapping.

[0077] The VIO technique can track features extracted across consecutive frames by matching them based on their appearance and motion. This process may involve estimating correspondences between features within different frames. This process can utilize optical flow or feature descriptor techniques for tracking. Features are generally considered static and part of the environment. After matching them, the displacement of those features is calculated along with the camera displacement.

[0078] Simultaneously, the IMU can provide continuous measurements of the camera's linear acceleration and angular velocity. These measurements can be integrated over time to estimate the camera's velocity and position using numerical integration techniques. These techniques may include, but are not limited to, the trapezoidal rule or higher-order integral methods. The inertial sensor can also function as a constraint on camera movement between frames.

[0079] Tracked visual features and motion estimated from the IMU can be combined through sensor fusion techniques such as extended Kalman filters (EKF) or nonlinear optimization approaches. The fusion algorithm can align visual measurements with inertial measurements by minimizing the error between predicted feature positions and their actual locations.

[0080] The fused information can be used to estimate the camera's pose (position and orientation) relative to an initial reference frame or a known map. Pose estimation can be achieved by updating the state of the VIO system using a sensor fusion algorithm.

[0081] As the mobile device and camera move, the VIO system can simultaneously build a map of the environment and position itself within that map. Tracked features and estimated camera poses can be used to create a visual map. The visual map can be further refined using techniques such as bundle adjustment or loop closure detection.

[0082] By combining visual information from a camera with inertial measurements from an IMU, VIO can provide robust and accurate estimates of the camera's, and therefore the mobile device's, motion, even in challenging conditions where a single sensor may not be sufficient. The integration of visual and inertial data improves the system's ability to track objects, estimate their trajectories, and navigate complex environments.

[0083] In various embodiments, only the seeker device (and not the sought device) possesses visual inertial odometry capabilities. In various embodiments, the sought device may possess only inertial capabilities.

[0084] Since many RF systems are already deployed and are part of the communication infrastructure, radio frequency (RF) measurements can be widely adopted for indoor positioning. RF radio technologies used for positioning include WLAN or Wi-Fi, RFID, UWB, Bluetooth, ZigBee, and LTE. UWB is a high-bandwidth communication technology with multipath robustness and good material penetration, and can achieve centimeter-level accuracy for 3D indoor positioning. However, its performance can degrade under strong scattering conditions. UWB systems typically utilize time-based measurements such as time of arrival (ToA) or time of arrival of arrival (TDoA) for position estimation. B. Both the person being investigated and the investigator possess inertia odometry.

[0085] In various embodiments, both the seeker device and the sought device can employ ranging techniques and inertial displacement (visual or by other means) to determine the location of the sought device relative to the seeker device. The ranging technique (e.g., time-of-flight ranging) can provide possible trajectories, but two or more relative trajectories may be possible. Odometry information allows for solving for angles to place both distance and trajectory into the same reference frame, thus enabling accurate identification of the sought device's position. The position of the sought device can be calculated using the least-squares equation, as described below.

[0086] Figure 6 shows the relationship between the trajectory of the explorer device and the explored device. Each of the explorer device and the explored device can operate in different coordinate systems. The explorer device can move along a first trajectory 602. The first trajectory 602 can be determined on the explorer device using the VIO technique. The VIO technique can result in a second trajectory 604 for the explored device. At least initially, the second trajectory 604 can be within an arbitrary coordinate frame. The explorer device can determine a range 606 (shown as a series of range measurements 606a to 606e). However, other potential curves may be possible, such as a third trajectory 608.

[0087] At various points in time, the explorer device can determine the range value between two devices. Based on the range value and VIO information, the explorer device can link up these two coordinate systems (for example, for the explorer device and the explored device). The explorer device can determine the initial offset of the explored device and then note that these two frames are actually already gravity-aligned. Since gravity is strong and both coordinate systems understand gravity fairly well, the z-axis between the two coordinate systems should be approximately the same. Gravity values ​​may be at different altitudes, but the explorer device may be able to detect what that difference is simply by using some gravity sensor. Alternatively, the explorer device can assume that the explorer device and the explored device are at approximately the same altitude.

[0088] The explorer device can, in fact, determine the difference between two coordinate systems with respect to all three directions (x, y, z). The first mobile device can determine not only where the two coordinate systems are relative to each other in the horizontal frame, but also how far apart the two coordinate systems are with respect to the vertical. This can be described as a joint estimation problem that the mobile device is solving for x, y, z, which means displacement, then angle theta (θ). The angle theta (θ) can be any orientation of the two coordinate frames relative to each other. The angle theta (θ) can be calculated with respect to the pitch, roll, and yaw of the mobile device, or rotation around gravity. The VIO simply establishes the coordinate system arbitrarily based on the direction the original device was facing when it started. The mobile device may be facing a different direction when the visual inertial odometry technique is initiated, and therefore may have any angle that the explorer device can determine.

[0089] Therefore, the explorer device only needs to determine an additional single horizontal angle between the coordinate systems in order to understand where the explorer device is located relative to the explored device.

[0090] A data channel can be established between a seeker mobile device and a searched mobile device. The data channel can be used to transmit information (e.g., visual odometry information) from the searched device to the seeker mobile device. In various embodiments, a wireless chip (e.g., a UWB chip) can provide such a data channel.

[0091] In various embodiments, the device being searched can transmit an applicable timestamp, and then transmit the estimated 3D position of a second mobile device in its own established coordinate system. The searcher device does not necessarily need to know what the other coordinate system is, but can determine what the change in distance over time from the other device is. Thus, the searcher device can determine the displacement between the devices. The searcher device can establish an estimated position by the change in projected position. In particular, the searcher device can determine the vector from the device being searched to the searcher device, and thus the searcher device can calculate where the second mobile device should be.

[0092] As shown in Figure 6, the range value 606 between the first and second mobile devices does not fit the second trajectory 604 of the device being searched. However, if the second trajectory 604 is rotated by an angle θ608, the range value 606 appears to fit the third trajectory 610. To determine exactly where the device being searched is within the searcher's coordinate frame, the first mobile device can be calculated for x, y, and z. The first mobile device can also calculate the angle θ608, which is a rotation around its horizontal angle. Essentially, this is a problem when there are searcher device trajectories and searched device trajectories that we want to place relative to the searcher device. The range value can establish constraints on where its searched trajectory can exist. This can be a joint problem where the x, y, and z origins and the angle θ608 can be solved together. A methodology that can be used to solve this problem is a nonlinear least squares method for making that solution. This can be optimized in essence to obtain the minimum error on the range while changing the angle θ608 and x, y, and z.

[0093] In various embodiments, the VIO technique may use six to eight measurements such that the values ​​are accurate enough to perform least squares and then align those points. After that point, as the explorer device determines new values, the technique can fine-tune and make the position more precise. The initial estimation of the trajectory is the most difficult.

[0094] In various embodiments, the explorer device can simulate different trajectories that the explored device may take in time (e.g., while walking). Each of these different trajectories may be weighted based on the likelihood of the explored device's actual position. The explorer device can generate directional information (e.g., a pointer or arrow) based on the average area of ​​the projected position.

[0095] The position of the second device can be determined using the following formula.

number

number

number

number

number

[0096] Using these techniques, several initial guesses can be made based on the minimum range observed between the explored and the explorer in a given time. This technique can attempt each of the three orthogonal directions for offset 0, and a 60-degree increment for theta. Each guess is iterated using the Gauss-Newton method. A stopping criterion can be determined and reached (e.g., the step size is very small, or the maximum number of iterations is reached). The final result is derived from each guess.

number

number

number

[0097] C. Only the explorer possesses inertial odometry. In various embodiments, only the explorer device possesses inertial odometry. In this case, other techniques can be used to determine the location of the explored device. In some cases, a particle filter can be used to estimate the location.

[0098] Particle filtering is a probabilistic estimation technique that can be used in conjunction with visual inertia odometry (VIO) to track objects or estimate their poses. Particle filtering techniques can be particularly useful in situations where the state of a system cannot be accurately represented by a single Gaussian distribution and where nonlinear or non-Gaussian uncertainty exists. In the context of VIO, particle filtering techniques can function as follows:

[0099] A particle filter can be initiated by initializing a set of particles, each representing a possible state hypothesis for the pose (position and orientation) of an object (e.g., a device being searched). These particles can typically be sampled from a prior distribution based on an initial estimate or prior knowledge of the system.

[0100] The pose of each particle can be propagated forward in time using a motion model that incorporates inertial measurements from the IMU. The motion model predicts the new pose of each particle based on the previous pose and the estimated motion derived from the IMU data. This step takes into account the expected motion of the object.

[0101] Visual information from the camera of the explorer device can be used to update particle weights. The VIO system can match observed visual features with predicted features based on the pose hypothesis for each particle. Matching can be performed using techniques such as feature tracking or feature descriptors. Particle weights can be calculated based on the similarity between the observed and predicted features. Particles that align better with the visual observations may be assigned higher weights.

[0102] In the resampling step, particles can be selected from the current set of particles with a probability proportional to their weights. Particles with higher weights are more likely to be selected multiple times, while particles with lower weights may not be selected at all. This process results in a new set of particles that more accurately approximates the posterior distribution.

[0103] The estimated object pose can be calculated based on resampled particles. This can be done by calculating a weighted average or the most likely pose among the resampled particles. The resulting pose estimate represents the best system estimate of the object's current state.

[0104] These steps can be repeated as new measurements become available. The particle filter continuously updates and refines the object's pose estimation based on the fusion of visual and inertial data.

[0105] The advantage of particle filters lies in their ability to represent and track a wide range of uncertainties and to handle nonlinear and non-Gaussian distributions. The particles within the filter allow for diverse representations of possible object poses, enabling the system to explore different hypotheses. Through iterative updates, the filter converges toward a more accurate estimate of the object's pose based on observations from both visual and inertial sensors.

[0106] By integrating particle filters with visual inertial odometry, it is possible to achieve robust object tracking and pose estimation even in challenging scenarios where uncertainty and nonlinearity exist.

[0107] Particle filters can be used to take advantage of the fact that ranging (e.g., TOF) data generally have only a positive bias. Particle filters are an application of Monte Carlo methods to Bayesian estimation. Measured distances are rarely shortened, but can be lengthened due to multipath or interference from bodies. IMUs provide the correct shape of the explored trajectory, but not the correct scale. Particles are randomly generated from a motion (or dynamics) model. Sampling from dynamics can be a very diffuse distribution. Therefore, the proportion of particles sampling trajectories close to the measured values ​​can be very small. Thus, a large number of particles may be needed to represent high-probability regions in state space. Resampling is a strategy to improve the number of particles that traverse trajectories with high likelihood. At each time step, copies of more likely particles replace less likely particles through a random sampling process. Some resampling strategies include multinomial resampling, stratified resampling, systematic resampling, and residual resampling.

[0108] The particle filter can simulate different trajectories that the searched object may be moving along, based on motion data (e.g., IMU data) and ranging data (e.g., TOF data). Each trajectory can be weighted. Arrows can be generated based on the average estimate of where the trajectory ends. Different paths can be at different scales, e.g., 70-150% of what the motion from (e.g., IMU) provides. There can be thousands of particles (possible trajectories) simulated with possible scales. Each particle can have random variation, and the likelihood of a given position is determined based on TOF and the searcher trajectory. The five variables for a particle are x, y, z, θ, and scale. If a particle has a low likelihood, it can be removed, and other random variations of particles with higher likelihood can be used. Data channels are used to obtain arbitrary odometry information from the searched object, if available.

[0109] Figure 7 shows an exemplary chart 700 of a particle filtering technique for locating object 710. The exemplary chart is plotted as z-location (in meters) versus x-location (in meters), but other coordinates and measurements can also be used. The current explorer location 702 is shown along with the explorer trajectory 702. Since object 710 can be detected using a ranging technique, a range circle 706 is shown indicating the range to object 710. Object 710 should be located on the range circle 702. Since object 710 is also moving, the object trajectory 712 is also shown. Figure 7 shows particles 714 representing possible state hypotheses for the pose (position and orientation) of an object (e.g., the explorer device). The explorer device can use the range circle 706 and particles 714 to calculate possible positions of object 710. The explorer device can generate an arrow 716 pointing in the direction of object 710. D. Flowchart

[0110] Figure 8 is a flowchart of a process 800 for locating a mobile device, according to an example of the present disclosure. For example, one or more process blocks in Figure 8 may be executed by the mobile device.

[0111] In block 805, process 800 may include transmitting a radio ranging signal at a first time. The radio signal may be one of many radio protocols, but is not limited to, ultra-wideband (UWB), Bluetooth (BT), Bluetooth Low Energy (BLE), Wi-Fi, Zigbee, etc. For example, a mobile device may transmit a radio ranging signal at a first time, as described above.

[0112] In block 810, process 800 may include receiving a radio response signal from a second mobile device in a second time period. The radio response signal may be received by one or more antennas on the mobile device. The radio response signal may be the same radio protocol and radio ranging signal. For example, the device may receive a radio response signal from a second mobile device in a second time period as described above.

[0113] In block 815, process 800 may include determining a range value between a first mobile device and a second mobile device based on the difference between a first time and a second time, thereby determining a set of range values. Radio signals can travel at the speed of light (c). If the first mobile device (e.g., a searcher device) knows the transmission time of the radio ranging signal, the reception time of the radio response, and the processing time of the second mobile device (e.g., the device being searched), the first mobile device can calculate the range value by multiplying the time delay (e.g., reception time - transmission time - processing delay) by the speed of light (c). For example, the first mobile device may determine a range value between a first mobile device and a second mobile device based on the difference between a first time and a second time, as described above, thereby determining a set of range values.

[0114] In various embodiments, blocks 805, 810, and 810 may be executed for each of a plurality of ranging sessions occurring during a given period.

[0115] In block 820, process 800 may include determining first odometry information from first measurements captured during a period using a first sensor on the first mobile device, the first odometry information indicating first movement of the first mobile device during the period. In various embodiments, the first odometry information may include visual inertial odometry information. In various embodiments, the first odometry information may be received from one or more of a camera and a motion sensor. The motion sensor may be an IMU. For example, the first mobile device may determine first odometry information from first measurements captured during a period using a first sensor on the first mobile device, as described above, the first odometry information indicating first movement of the first mobile device during the period.

[0116] In block 825, process 800 may include receiving second odometry information via a data channel between the first mobile device and the second mobile device, which is determined from second measurements captured during a period using a second sensor on the second mobile device, and the second odometry information indicates the second movement of the second mobile device during that period. For example, the second mobile device may determine the second odometry information from second measurements captured during that period using a second sensor on the second mobile device, as described above, and the second odometry information indicates the second movement of the second mobile device during that period.

[0117] In various embodiments, the second sensor is an optical sensor, and the second sensor information is odometry information.

[0118] In various embodiments, the data channel may be generated via a wireless signal chip. In various embodiments, the data channel may include a narrowband channel controlled by an ultra-wideband processing chip.

[0119] For example, the device may receive, as described above, second odometry information determined from second measurements captured during the period using a second sensor on the second mobile device via a data channel between the first mobile device and the second mobile device, and the second odometry information indicates the second movement of the second mobile device during the period.

[0120] In block 830, process 800 may include solving for the angle between the first reference frame of the first device and the second reference frame of the second device using a set of range values, first odometry information, and second odometry information. In various embodiments, the angle between the first reference frame of the first device and the second reference frame of the second device can be solved using the least squares formula as described above. For example, the devices can solve for the angle between the first reference frame of the first device and the second reference frame of the second device using a set of range values, first odometry information, and second odometry information as described above.

[0121] In various embodiments, solving for the angle between a first reference frame of a first device and a second reference frame of a second device is done using a least-squares equation based on at least a range value, first odometry information, and second sensor information.

[0122] In various embodiments, solving for the angle between the first reference frame of the first device and the second reference frame of the second device may include calculating the vertical displacement, horizontal displacement, and directional offset between the first mobile device and the second mobile device.

[0123] In various embodiments, the first device and the second device are in motion.

[0124] In various embodiments, the second sensor is an accelerometer, and the second sensor information is acceleration information from a second mobile device.

[0125] In block 835, process 800 may include displaying directional information on the display of the first mobile device indicating the direction from the first current position of the first mobile device to the second current position of the second mobile device. For example, as described above, the device may display a directional arrow on the display of the first mobile device indicating the direction from the first current position of the first mobile device to the second current position of the second mobile device. In various embodiments, the directional information may be an arrow.

[0126] Process 800 may include additional implementations, such as any single implementation or any combination of implementations, relating to one or more other processes described below and / or elsewhere in this specification.

[0127] In one embodiment, the mobile device may include one or more processors and memory coupled to one or more processors, the memory storing instructions that cause one or more processors to perform one or more of the operations described above.

[0128] In one embodiment, a non-temporary-readable medium can store a number of instructions that, when executed by one or more processors of the mobile device, cause the mobile device to perform any of the operations described above.

[0129] In various embodiments, process 800 further includes determining whether the relative position between the devices changes and suppressing the display of direction from the first mobile device to the second mobile device based on the angle.

[0130] Figure 8 shows an exemplary block of process 800, but note that in some implementations, process 800 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those shown in Figure 8. Additionally or alternatively, two or more blocks of process 800 may be executed in parallel. IV. Shared Reference Frames for Switching Between Odometry Techniques

[0131] Visual odometry can fail in some situations. For example, when a device is moving with a crowd of people, the device may not be able to detect the movement. Visual odometry (e.g., visual inertial odometry) detects movement by measuring changes to identify features in a sequence of images. However, visual odometry can fail when a device is trying to identify the features of an individual moving with the device. The device may be able to detect movement within an image, but may not be able to detect displacement across the image because the tracked features are in step with the device. Alternatively, the device may not be able to detect displacement because large amounts of movement do not mean there are enough static features to compare across image frames. The device may be able to detect large amounts of movement across image frames, but may not have enough information to determine the correspondence between consecutive frames.

[0132] Inertial odometry can be used to determine the movement of a device when visual odometry (e.g., visual inertial odometry) is unreliable (e.g., fails). While inertial odometry techniques can be accurate over short distances, such techniques may be less accurate than visual odometry in some situations. To compensate for inertial odometry errors, a mobile device may compare the inertial odometry result with the visual odometry result to determine the error value of the inertial odometry value. When a mobile device determines that the visual odometry result is unreliable, the device may use the inertial odometry value and the current error value to determine the movement of the mobile device until a reliable visual odometry value becomes available. A. Visual odometry error in congested environments

[0133] Congested environments can cause visual inertial odometry (VIO) techniques to produce unreliable results (e.g., failures). VIO techniques measure device movement by comparing the apparent movement (e.g., disparity) of features across a sequence of images captured by the device's camera. Therefore, VIO techniques may only be able to determine device movement if at least one feature is present in multiple images and at least one feature changes position between images. However, such features can be difficult to identify in congested environments.

[0134] Figures 9A and 9B show simplified diagrams 900 and 901 of image frames in a congested environment according to various embodiments. Figure 9A shows a first scenario in which a congested environment may cause the VIO technique to produce unreliable results. Image frame 902 shows a first image frame captured by a device (e.g., a computing device, a mobile device, etc.) performing the VIO technique during the first scenario. In the first scenario, the device performs the VIO technique while moving through a congested environment in which other objects / individuals move with the device. For example, the device may be held or worn by an individual in a crowd leaving a stadium after a sports match. The device identifies a feature, shown as a black box, on an individual 904 walking in front of the device. The feature may be a pattern of pixels that can be individually identified (e.g., an instance of a feature may be recognized as that particular instance across the image), and the feature may include edges, corners, blobs, ridges, etc.

[0135] Image frame 906 shows a second image frame captured by a device performing the VIO technique during the first scenario. The device may have moved significantly between image frame 902 and image frame 906, but the device is moving with the crowd, and the movement may not be apparent. The VIO technique performed by the device recognizes features on individual 904, including the crowd. However, since the crowd is moving with the device, the features appear static. For example, feature 908 is in approximately the same position between image frame 902 and image frame 906. Therefore, the VIO technique does not detect that the device has moved, as the features do not show any apparent movement across the image frames. The VIO technique may fail to detect movement if none of the features appear to be moving, or if a threshold number of features appear to be moving. Features may appear to move if the difference in the position of features between frames exceeds a threshold.

[0136] Referring now to Figure 9B, a second scenario is shown in which a congested environment can cause the VIO technique to produce unreliable results. In the second scenario, there is sufficient movement in the congested environment, and therefore the device cannot distinguish features between image frame 910 and image frame 912. Features are shown as filled-in black boxes, and the second scenario could be a crowded subway station where a large crowd of individuals is moving in different directions. However, the second scenario could be any environment with a large amount of movement (e.g., a crowded roundabout with many vehicles).

[0137] As shown in Figure 9B, the VIO technique may fail to identify features present in both image frame 910 and image frame 912. The inability to identify a threshold number of features in both image frames may be due to crowd movement. For example, a physical object may be identified as a feature in frame 910, and crowd movement between frames may obscure the object in frame 912. In another example, the feature could correspond to a pattern on a first individual's shirt, and crowd movement could mean that a second individual has blocked the pattern from the camera's field of view.

[0138] As an addition or alternative, crowd movement may mean that features are identified on individuals or objects that move out of the camera's field of view between image frames. For example, a feature identified on a car at a crowded intersection in the first frame may move away from the intersection before the second frame is captured. Therefore, the VIO technique may not be able to detect the movement of a device between image frame 910 and image frame 912 because there are not enough features to compare across the image frames to determine the movement of the device relative to those features. The number of features may be sufficient if the number of features present in both image frames exceeds a threshold. The threshold for either the first or second scenario may be one, two, three, four, five, ten, fifteen, twenty, twenty-five, fifty, or one hundred features. B. Standard Frame

[0139] Device displacement in a physical environment can be determined using either inertial odometry (IO) or visual inertial odometry (VIO) techniques. Displacement can be represented as a series of poses (position and orientation) over a period of time or as a continuous displacement over a period of time. Devices may switch between odometry techniques in scenarios where the displacement estimated by one odometry technique becomes unreliable. However, each odometry technique may represent poses or displacements within separate reference frames, and switching between odometry techniques may mean switching between reference frames.

[0140] A reference frame can be a mapping of a coordinate system to the physical environment. For example, coordinates assigned to a pose in a VIO reference frame may be determined relative to axes determined based on the camera's pose during initialization, while the coordinate system of an IO reference frame may include coordinates determined relative to axes corresponding to gravity and magnetic north. For these different reference frames, the same physical position may be represented by different coordinates depending on whether the pose was determined by the VIO technique or the IO technique. Therefore, switching between odometry techniques may require converting some or all of the coordinates representing different poses to a common reference frame so that displacements between poses can be determined.

[0141] Figure 10 is a simplified diagram 1000 illustrating transformations between reference frames using various techniques. The body frame 1002 can be a reference frame for the physical characteristics of the mobile device being tracked. For example, the body frame 1002 may include orthogonal axes passing through each of the outer surfaces of the mobile device. Humans can understand the device's pose in the body frame 1002 (e.g., screen side up), but odometry techniques may not directly represent the device's displacement in the body frame 1002.

[0142] Each odometry technique can assign coordinates to the mobile device's pose within a separate reference frame. These separate reference frames may have fixed relationships with respect to the body frame 1002 and to each other, and transformations between the body frame and the VIO frame 1004 and IO frame 1006 may be known. However, transformations between the VIO frame 1004 and IO frame 1006 may not be known. The mobile device's displacement can be determined by comparing different poses of the device, and no transformations between reference frames are necessary as long as all poses are within the same reference frame. For example, the mobile device's position may first be tracked in the VIO frame 1004, and the device's displacement can be determined by comparing the coordinates representing the pose within this reference frame. In the case of the VIO frame 1004, the change in the mobile device's displacement over time (t) can be expressed using the following notation: VIO(t) VIO

[0143] Continuing this example, a mobile device may shift from tracking displacement using the VIO technique to tracking displacement using the IO technique. The mobile device may make this change because the VIO poses have become unreliable. To begin tracking the mobile device's position using the IO technique, the mobile device may need to determine a correspondence between a VIO pose (e.g., coordinates representing the physical position and orientation in VIO frame 1004) and a subsequent IO pose (e.g., coordinates representing the physical position and orientation in IO frame 1006). Without this correspondence, the mobile device may be unable to determine displacement between a VIO pose and a subsequent IO pose because the orientation, origin, and scale may differ in each reference frame. This correspondence can be used to determine transformations that may be used to switch poses between reference frames.

[0144] A transformation can be a function that uses the correspondence between reference frames to convert coordinates from a first reference frame to coordinates in a second coordinate system. By a simple analogy, a transformation can be used to switch between imperial and metric units. For example, an American visitor to Edinburgh, Scotland, wants to check the local weather, but Scottish news reports temperatures in Celsius (metric), while the American is only familiar with Fahrenheit (imperial). In this case, the units of each system have different scales (e.g., degrees in Fahrenheit are 5 / 9 of degrees in Celsius), different origins (e.g., 0 degrees Celsius is the freezing point of water, and 0 degrees Fahrenheit is below freezing), but the same orientation (e.g., both systems increase as the temperature rises). Using this correspondence, the following transformations between the two systems can be determined:

number

[0145] Returning to FIG. 10, a transformation may be used to change a pose in an IO frame to a VIO frame. The transformation between the VIO frame and the IO frame may be determined by identifying the poses in each frame that correspond to the same physical position. However, the VIO technique and the IO technique may not start simultaneously, and it may be necessary to normalize the time scale for each frame. For example, the IO technique may initialize and determine poses more quickly than the VIO technique, and thus, the first pose for a physical position in one frame may have a different timestamp than the second pose for a physical position in a second time frame. In some embodiments, the VIO frame may initialize and determine poses more quickly than the IO frame. This delay between the poses in the two frames may be represented by t d and may be represented by the following notation, where the pose in IO frame 1006 that is equivalent to the pose in VIO frame 1004 at time t is represented by: IO(t - t d ) IO

[0146] The transformation can be determined by comparing equivalent poses within each reference frame. For example, if the VIO pose at time t (e.g., the pose in VIO frame 1004) is the first set of x, y, z coordinates at (5, 2, 7), and the equivalent IO pose at time t - t d (e.g., the pose in IO frame 1006) is the second set of coordinates at (7, 2, 5), the transformation may be the angle of rotation about the y-axis. In this example, the transformation is a rotation, but in some embodiments, the transformation may include any combination of rotations about any number of axes, changes in scale along any number of axes, and changes in the origin on any number of axes. C. Compensation for IO Bias (Direction and Velocity)

[0147] Inertial odometry (IO) techniques can be used to determine device displacement when visual inertial odometry (VIO) techniques become unreliable. However, the displacement and pose projected by IO techniques are less accurate than those determined through VIO techniques. VIO techniques determine the pose of a mobile device relative to identified features in a series of images. Displacement is then determined by comparing these poses. IO techniques can determine displacement by providing the outputs of various motion / magnetometer sensors to a pedestrian dead reckoning (PDR) algorithm. This PDR algorithm converts the sensor outputs into estimated device displacements based on a model that estimates human movement. Pose is then determined using this displacement. The predicted displacement of the PDR algorithm may be less accurate than the pose and displacement determined by VIO techniques, and with IO techniques, each subsequent pose is determined based on the displacement from the previous pose. Therefore, pose errors propagate to subsequent poses, and IO techniques may become less accurate as these errors accumulate.

[0148] IO techniques can be made more accurate by calculating the bias of the displacements projected by these techniques. The bias can be calculated when both IO and VIO techniques are available. A pose determined by the VIO technique can be compared to the corresponding pose determined by the IO technique. Any discrepancy between the pose calculated by the VIO technique and the pose calculated by the IO technique can be used to determine the bias for the IO technique. For example, if the distance projected by the IO technique is 90% of the distance projected by the VIO technique, the bias for the IO distance can be set to 1 / 90%. The bias can be calculated based on a single discrepancy, or on the average discrepancy over any number of samples or a period of any length. The bias can include separate biases calculated for distance, velocity, or angle along any number of axes.

[0149] When the VIO technique is unreliable or unavailable, a bias can be applied to the displacements and poses calculated by the IO technique. If the VIO technique is unavailable or unreliable (e.g., VIO fails), the bias may be used to correct the displacement or pose determined using the IO technique. In some embodiments, the bias may be used to indicate uncertainty in the displacement or pose calculated using either the IO or VIO technique. In some embodiments, a discrepancy between the VIO projection and the IO projection may be used to determine a bias if the discrepancy is below a threshold, but may be used to determine that the VIO result is unreliable if the discrepancy is above a threshold. For example, if the magnitude of the discrepancy is less than 50% (e.g., the displacement calculated by IO is at least 50% of the VIO displacement), the discrepancy may be designated as a bias. If the magnitude of the discrepancy is greater than 50%, one or more of the IO or VIO techniques may be designated as unreliable. For example, if the IO displacement (e.g., the displacement determined by the IO technique) is 40% of the VIO displacement, the IO technique may be designated as unreliable. Any number of thresholds can be used; for example, a first threshold can be used to identify unreliable displacements, and a second threshold can be used to identify biases. In some embodiments, the thresholds may differ for VIO or IO displacements. D. Splice reference frame

[0150] A mobile device may perform multiple switches between odometry techniques. For example, a mobile device may use a first instance performing the visual inertial odometry (VIO) technique to determine its displacement, switch to the IO technique when the first instance performing the VIO technique becomes unreliable, and then switch to a second instance performing the VIO technique when it becomes reliable again. Each switch between odometry techniques may involve transformations between reference frames, and separate instances of the same odometry technique may have different reference frames. Continuing the above example, the reference frame may differ between the first instance performing the VIO technique and the second instance. Errors in each instance in which an odometry technique is performed can accumulate, and these errors, along with errors in the transformation, can change the reference frame over time. Therefore, the reference frame may be a spliced ​​reference frame representing the fusion of different reference frames used during tracking.

[0151] Figure 11 is a simplified diagram 1100 of an architecture for transformation between reference frames in various embodiments. As shown in Figure 1100, the transformation is an operation to rotate a pose in a first reference frame to a pose in a second reference frame. However, other transformations are also possible, and the transformation may include an operation to determine the coordinates in the second reference frame using the coordinates in the first reference frame when the scale or origin of one or more axes differs between the two reference frames.

[0152] Referring more closely to Figure 11, the odometry system 1102 can determine the device's pose over time using the output from the mobile device's sensors. The pose over time may be a series of poses (e.g., position and orientation) at discrete points in time, a series of displacements, or a series of net displacements where each displacement corresponds to a period. The displacement may be a change in position and orientation from a first pose to a second pose. The odometry system 1102 can compare a series of images (e.g., visual information) with the output from the inertial sensor (e.g., inertial information) to determine the VIO system 1104 within a VIO reference frame. The VIO system 1104 can determine the pose as a function of time (t), and this function may be expressed in the following notation. VIO(t) VIO Here, the subscript VIO indicates that the pose is determined within the VIO reference frame.

[0153] The odometry system 1102 can determine IO pauses as a function of time using inertial sensor outputs. IO pauses within an IO frame can be represented using the following notation: IO(tt d ) IO Here, the subscript IO indicates that the pause is determined within the IO reference frame. VIO systems 1104 and 1105 may output pauses at different times. Compared to IO system 1106, VIO system 1104 may take longer to initialize the pause and begin outputting it. Additionally or alternatively, VIO system 1105 may take longer to provide an output for a given input.

[0154] The outputs from the VIO system 1104 and the IO system 1106 at a given time (t) may correspond to different input times. This mismatch in output time can be corrected by subtracting a time delay from either the VIO system 1104 or the IO system 1106. For example, the IO system may start outputting pause at t0, but the VIO system 1104 may not be initialized and (t0+t d It may not be ready to output a pause until (t0-t). Once the VIO system 1104 is initialized, the system may begin determining the output for queued inputs. Therefore, a pause for an input measured over a given period may be output by the two systems at different times. This discrepancy in output time is due to (t0-t d The VIO input in ) can be handled by an interpolator 1112 that can interpolate it. After this interpolation, both the VIO input and the IO input can correspond to the same point in time.

[0155] The output from the odometry system 1102 may be provided to the transformation system 1108. The transformation system 1108 can determine transformations between reference frames. For example, the transformation shown in Figure 1100 involves a three-dimensional rotation (e.g., a quaternion). The output from the odometry system 1102 may need to be processed before the transformation can be determined. For example, the output of the VIO system 1104 may be discrete, while the output of the IO system 1106 may be continuous. To compare the outputs of these two systems, the transformation system 1108 may need to project (e.g., interpolate) the continuous data for the output of the VIO system 1104.

[0156] The output of the VIO system 1104 can be stored in the VIO buffer 1110 as a series of discrete poses. These discrete poses can be calculated at regular intervals (e.g., at the time of each image frame capture), and the difference between two consecutive poses can represent the net change in position during the interval between the two poses. In contrast, the output of the IO system 1106 may be a series of positions that estimate the displacement of the mobile device over time. The interpolator 1112 can use the discrete poses in the VIO buffer 1110 and the displacement output by the IO system 1106 to estimate the displacement of the mobile device performing the operation disclosed with reference to Figure 1100. The interpolator can be used to find the interpolated VIO output corresponding to the timestamp of each IO measurement. The interpolator can also take into account the difference in time frames used by the VIO system 1104 and the IO system 1106 (e.g., t IO and t VIO The displacement can be changed (so that it is equivalent to the other).

[0157] After the VIO output is interpolated, the IO output and the interpolated VIO output can be provided to the splice frame system 1114. These outputs can be provided as one or more quaternions in a common time frame. A quaternion representing a rotation of θ radians around the unit axis (x, y, z) can be expressed as follows:

number

number

number

[0158] First, the splice frame can be a VIO frame, and the first rotation calculated by the splice frame system can be a rotation from the initial IO frame to the initial VIO frame designated as the initial splice frame. For example, the VIO output may become unreliable, and the rotation may facilitate switching from VIO navigation to IO navigation. Continuing the example, the VIO output may become reliable again, but the second VIO frame may be different from the initial VIO frame. Accordingly, the splice frame system 1114 may calculate a rotation from the second VIO frame to the splice frame so that the device can return to VIO navigation. The IO system 1106 may output altitude and a second velocity change (generated without pedestrian dead reckoning), which are events that may exist when other outputs from the IO system are unavailable. The quaternions of altitude and velocity change may be applied separately from the quaternions of the other outputs of the IO system 1106. These quaternions may be combined in the splicing block, represented by the black circle with a white "X" in Figure 11.

[0159] The outputs of the splice frame system 1114 and the odometry system 11102 are provided to the position system 1116. The position system 1116 receives the output of the VIO system 1104 and the quaternion unit vector output by the splice frame system 1114.

number

number

[0160] Transformations such as quaternions can be computed during transitions between odometry techniques. For example, a transformation can be computed when unreliable VIO information becomes reliable again. However, not all changes in odometry techniques require the transformation system to compute a transformation.

[0161] Figure 12 is a simplified state diagram showing events that trigger the determination of conversions between reference frames in various embodiments. Block 1202 can represent the state of a device that is not performing either an IO technique or a VIO technique. For example, block 1202 may represent the state of a device before an odometry technique is requested by the user. As mentioned above, the IO technique can be initialized more quickly than the VIO technique, and the state of the device can proceed to block 1204 first in response to the request for an odometry technique. In such a situation, the IO reference frame may be designated as the splice reference frame, and the body frame to the splice frame

number

number

number

[0162] As described above, the IO technique may be initialized before the VIO technique becomes available. A device performing the IO technique may transition from the state represented by block 1204 to the state represented by block 1206 if the VIO technique becomes available in addition to the IO technique. The technique may be available if the device can perform the technique and receive a reliable output of that technique.

[0163] The transformation can be calculated when the device transitions from a state represented by block 1204 to a state represented by block 1206. For example, the transition between these blocks means that the device has access to a reliable VIO output, in which case the device's position can be estimated using the VIO technique. These VIO outputs can be provided as position or displacement in a VIO reference frame. However, past positions are recorded in a splice reference frame (e.g., a splice frame). From the VIO reference frame to the splice frame

number

number

number

number

[0164] The device may transition from a state represented by block 1206 to a state represented by block 1204. The splice reference frame may have changed depending on the use of the VIO technique in block 1206, and the conversion from the IO reference frame to the splice reference frame is from the body frame to the splice frame.

number

number

[0165] A device may transition from a state represented by block 1206 to a block represented by block 1208. In this state, the device may only have access to the output of the VIO technique. However, since the VIO technique is used to determine the device's position in both states, it may not be necessary to compute a conversion in response to this transition. However, if the device transitions from block 1208 to block 1206, a conversion from the current IO reference frame to a splice frame may be computed. The conversion between the VIO frame and the splice frame may remain the same (for example,

number

number

number

[0166] The device may transition from the state represented by block 1208 to the state represented by block 1202. This transition can occur when VIO becomes unreliable and IO techniques are unavailable. In such a situation, odometry techniques are not performed, and the device may be reset because the device does not track its position in any reference frame. When the device transitions from block 1202 to block 1208, the device begins to perform VIO techniques and determine its position in the current VIO reference frame. Thus, the VIO reference frame may be designated as a splice reference frame, from the body frame to the splice frame.

number

number

number

[0167] Figure 13 is a flowchart of process 1300 for conversion between reference frames, according to an example of the present disclosure. For example, one or more process blocks in Figure 13 may be executed by a mobile device.

[0168] In block 1305, first inertial odometry information can be determined from first inertial measurements. The first inertial measurements can be captured over a first period of time using an inertial sensor on a mobile device. The inertial sensor may include one or more accelerometers; for example, the inertial sensor may consist of three accelerometers arranged on orthogonal axes. Additionally or alternatively, the inertial sensor may include any number of magnetometers or gyroscopes, which may be arranged along any number of axes. The inertial information may be discrete or continuous; for example, the inertial information may be a continuous region output of some or all of the inertial sensor over a first period of time.

[0169] In block 1310, a first reference frame corresponding to a first inertia measurement can be identified. The first reference frame may be an IO reference frame, and the first reference frame may be a three-dimensional coordinate system. In various embodiments, the first reference frame may be a splice reference frame.

[0170] In block 1315, the first visual odometry information may be determined from first visual measurements captured over a first period of time. The first visual measurements may be captured by a visual sensor of a first mobile device. The first visual measurements may be image frames captured by the visual sensor, or the first visual measurements may be one or more identified features from the image frames. The first visual odometry information may be one or more positions of the first mobile device, each of which may correspond to the position of the first mobile device during the capture of the image frames.

[0171] The first visual sensor may be one or more cameras, and the cameras may be part of the first mobile device. In some embodiments, the first visual sensor may be one or more cameras on an electronic device that is communicatively connected (e.g., wired or wireless) to the first mobile device. For example, the first visual sensor may be one or more cameras on a wearable device such as smart glasses or headphones. In some embodiments, the first mobile device may be a virtual reality device or an augmented reality device.

[0172] In block 1320, a second reference frame corresponding to the first visual measurement can be captured. The second reference frame may be a VIO reference frame.

[0173] In block 1325, a first transformation between a second reference frame and a first reference frame can be determined. The first transformation may be a rotation around one or more axes. For example, the first transformation may be a quaternion rotation or a rotation matrix. In some embodiments, the transformation may include one or more of the following: a change in scale around one or more axes, or a change in the origin along one or more axes.

[0174] In block 1330, the displacement of the first mobile device in a first reference frame can be determined. The displacement can be determined for a first period, and the displacement can be determined using first visual odometry information from block 1315 and transformations from block 1325. The displacement may be one or more poses of the first mobile device during the first period, or the displacement may be a continuous path of the first mobile device during the first period.

[0175] Determining the displacement of the first mobile device in the first reference frame may include mapping the first visual odometry information onto the second reference frame based on a transformation. Mapping may mean applying the transformation from block 1325 to the first visual odometry information. The first visual odometry information may be any combination of displacement or one or more poses in the second reference frame. Once mapped, the first inertial information and the second inertial information can be compared to determine the error in the first inertial odometry information during the first period. The error may be the percentage difference between any combination of pose or displacement corresponding to the first inertial information and pose or displacement corresponding to the first visual odometry information.

[0176] The first inertial odometry information can be transformed by the error of the first inertial odometry information in order to obtain the transformed first inertial odometry information. For example, if the error indicates that the first inertial odometry information produces a displacement having a magnitude that is 80% of the magnitude determined by the first visual odometry information (e.g., the error is 0.80), then the displacement represented by the first inertial odometry information can be divided by the error in order to determine the transformed first inertial odometry information. The displacement of the first mobile device in the first reference frame can be determined based on any combination of the transformed first inertial odometry information or the first visual odometry information. The error may be an error in the output of an inertial sensor. For example, the error may be a percentage difference between acceleration, velocity, or magnetometer readings output by an inertial sensor. Separate errors may exist for acceleration, velocity, or magnetometer readings on any number of axes.

[0177] This technique may include determining second inertial odometry information from second inertial measurements taken over a second period following a first period. The second inertial odometry information may be determined using the error in the first inertial odometry information (for example, the output or displacement of an inertial sensor may be corrected by the error). The second visual odometry information may be determined over the second period.

[0178] Second inertial odometry information and second visual odometry information can be compared to determine whether the second visual odometry information is unreliable. For example, pauses from second inertial odometry information and second visual odometry information can be compared to determine whether the second visual odometry information is unreliable. For instance, if visual odometry information indicates that the mobile device is stationary, but inertial odometry information indicates that the device has moved, then visual odometry information may be unreliable.

[0179] In some cases, the second visual odometry information can be identified as unreliable by comparing it to a threshold without comparing it to the second inertial odometry information. For example, the second visual odometry information may be unreliable if the displacement between consecutive poses is sufficiently large. In some embodiments, the visual odometry information may be identified as unreliable if the visual measurements used to generate the visual odometry information do not meet one or more visual measurement thresholds. For example, an image frame in a visual measurement may not meet a visual measurement threshold if the brightness of the image frame is below a threshold, above a threshold, the signal-to-noise ratio of the image frame is below a threshold, or the number of features identified within the image frame is below a threshold.

[0180] The mobile device may stop using the second visual odometry information during the second period if it determines that the second visual odometry information is unreliable over the second period. The mobile device may switch to using the second inertial odometry information to determine the displacement of the first mobile device in the first reference frame if it determines that the second visual odometry information is unreliable. The mobile device may use the second inertial odometry information to determine the displacement during the second period. The mobile device may use the first transformation, the inverse of the first transformation, or a different transformation to determine the displacement of the first mobile device in the first reference frame.

[0181] A mobile device may perform a third visual measurement during a second period or a third period following the second period. The third visual measurement may be used to determine third visual odometry information, and if the third visual odometry measurement is determined to be reliable, the mobile device may use the third visual odometry information to determine the displacement of the mobile device during the third period. Determining the displacement in the first reference frame may include determining a second transformation between the third reference frame and the first reference frame corresponding to the third visual measurement.

[0182] In various embodiments, a mobile device may include one or more processors and memory coupled to one or more processors. The memory may store instructions that cause one or more processors to perform any one or more of the operations described above.

[0183] In various embodiments, a non-temporary computer-readable medium can store instructions that, when executed on one or more processors, perform any one or more of the operations described above.

[0184] Figure 13 shows an exemplary block of process 1700, but note that in some implementations, process 1300 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those shown in Figure 13. Additionally or alternatively, two or more blocks of process 1300 may be executed in parallel. V. Integration of human detection techniques

[0185] Mobile devices possess various different capabilities for detecting other mobile devices. In certain situations, human detection techniques may not be effective, and mobile devices may use alternative techniques or combine positions obtained from various techniques. For example, GNSS-based systems may experience interference in urban areas, indoors, or areas with dense foliage. In these cases, other human detection techniques can be used alone or in combination with GNSS positions. Various different combinations are described below. A. Interference problems and causes of uncertainty

[0186] GNSS-based navigation systems are often used to determine the location of mobile devices. GNSS-based systems such as GPS (Global Positioning System), GLONASS, Galileo, and BeiDou can rely on precise and accurate signals from satellites to determine their position on Earth. However, these signals can be subject to various noise issues that can affect the performance and accuracy of GNSS navigation. Some common noise issues in GNSS navigation signals may include multipath interference, atmospheric interference, signal jamming, receiver noise, interference, and clock drift.

[0187] Multipath interference can occur when GNSS signals are reflected from surfaces such as buildings, vehicles, or terrain before reaching the receiver antenna. Reflected signals can interfere with the direct signal, potentially causing errors in position estimation. Multipath interference is more common in urban environments with tall buildings and can lead to inaccurate positioning.

[0188] Atmospheric interference can introduce noise as GNSS signals pass through the Earth's atmosphere, and various atmospheric conditions can introduce noise. For example, ionospheric delay can cause signal refraction, leading to errors in estimating signal travel time. Similarly, tropospheric conditions such as temperature and humidity fluctuations can cause signal attenuation and delay. These atmospheric influences can affect the accuracy of GNSS positioning.

[0189] Signal obstruction can present a problem when natural features such as buildings, dense foliage, and mountains or canyons can block the line of sight between the satellite and the receiver antenna. When the line of sight is blocked, the received signal may be weakened or completely lost, resulting in degraded or intermittent GNSS signal reception.

[0190] GNSS receivers themselves can introduce noise into the received signal, which can degrade the accuracy of position estimation. Receiver noise can originate from various sources, including internal electronics, thermal effects, and electromagnetic interference. High-quality GNSS receivers employ techniques to minimize receiver noise and improve the signal-to-noise ratio.

[0191] GNSS signals operate in the radio frequency spectrum and can be susceptible to interference from other mobile devices or radio transmissions. Radio frequency interference (RFI) from nearby devices operating in the same frequency range can disrupt GNSS signals and lead to positioning errors.

[0192] Precise timing is crucial in GNSS navigation, and clock drift can present problems. Satellite signals carry precise timing information, and clock mismatches between satellites and receivers can introduce errors. Clock drift in either the receiver or satellite clock can lead to inaccuracies in calculated positions.

[0193] To mitigate these noise problems, various techniques can be employed, including antenna design optimization, signal processing algorithms, and the use of multiple GNSS constellations for improved positioning accuracy. Furthermore, differential GNSS techniques, where a stationary reference receiver provides corrections to a mobile receiver, can mitigate some noise problems and help improve positioning accuracy in real-time applications. When GNSS becomes unreliable, mobile devices can determine the uncertainty between GNSS techniques and other positioning techniques (e.g., ranging, RSSI determination).

[0194] Figure 14 shows a first user with a first mobile device 1402 attempting to locate a second user with a second mobile device 1404. Multiple satellites 1406 can transmit timing signals 1414, which can typically be received by the mobile device's antenna. The mobile device can use the timing information to determine its own position.

[0195] Four satellites 1406 are shown in Figure 14, but additional satellites 1406 may be within the mobile device's field of view. For example, the number of GPS satellites within the field of view at any given location and time varies depending on several factors, including location on Earth, time of day, and surrounding environment. However, the GPS satellite constellation currently consists of a total of 24 operational satellites.

[0196] To provide accurate positioning, a GPS receiver typically requires signals from at least four satellites. Using signals from these four satellites, the receiver can perform trilateration to determine its position in three-dimensional space (latitude, longitude, and altitude), as well as the precise time.

[0197] In reality, the number of GPS satellites within the field of view can vary between four and more, depending on the receiver's location. In open areas with unobstructed views of the sky, it's generally possible to have more satellites in view simultaneously. For example, under ideal conditions, it's possible to have 6 to 12 or even more satellites in the field of view.

[0198] Figure 14 shows an area of ​​dense foliage 1412 that may block or introduce noise to the timing signal 1414 from satellite 1406. As shown in Figure 14, dense foliage 1412 may reduce the timing signal 1414 received by the first mobile device, making the GPS position inaccurate, unreliable, or potentially unavailable. However, the second mobile device 1404 may be able to receive the timing signal 1414 from various satellites 1406. The second mobile device 1404 may be able to determine an accurate position.

[0199] The first mobile device 1402 and the second mobile device 1404 may be able to calculate their relative positions using other positioning techniques (e.g., ranging techniques) via wireless communication 1416 between the devices.

[0200] Other wireless transmitters 1410 may also be within the range of the first wireless device 1402 and the second wireless device 1404. Various positioning techniques (e.g., Wi-Fi positioning, LTE positioning, or 5G positioning) may be used by the mobile device. The wireless transmitter 1410 shown in Figure 14 may include a Wi-Fi access point (AP), an LTE cellular tower, or a 5G cellular tower.

[0201] Positioning techniques that use Wi-Fi signals, commonly known as Wi-Fi positioning or Wi-Fi-based location, utilize signals from Wi-Fi access points (APs) to estimate the location of a mobile device. Signal strength-based location involves measuring the received signal strength (RSS) from nearby Wi-Fi APs. Databases, often called wireless maps or fingerprint databases, are created by collecting RSS values ​​at known locations. The databases contain information about the locations of Wi-Fi APs and their corresponding RSS fingerprints.

[0202] In offline mode, mobile devices can collect RSS measurements from nearby access points (APs) and match them with a stored fingerprint database to estimate their location. The best match between the measured RSS value and the stored fingerprint determines the device's position.

[0203] In online mode, a mobile device can send the measured RSS value to a server or cloud-based infrastructure that performs the matching process. The server compares the measured RSS value to a stored fingerprint and returns the estimated location to the device.

[0204] The position of a mobile device can be estimated based on the distance or angle between the device and multiple Wi-Fi access points (APs) using triangulation or trilateration techniques. The distance between the mobile device and multiple APs can be estimated using RSS values ​​or signal propagation models. Using distance information, the device's position can be calculated using techniques such as trilateration or polylateration. By measuring the angle of arrival (AOA) or time difference of arrival (TDOA) of Wi-Fi signals from different APs, the device's position can be estimated using techniques such as triangulation or angle of arrival.

[0205] Combining Wi-Fi positioning with other positioning technologies such as GPS, cellular networks, or sensor fusion can improve accuracy, reliability, and coverage. By integrating multiple sources of positioning data, devices can achieve improved performance in a variety of scenarios.

[0206] Combining Wi-Fi signals with data from other sensors such as accelerometers, gyroscopes, magnetometers, or barometers enables more robust positioning and compensation for environmental factors.

[0207] Leveraging additional information such as GPS-assisted data or cellular network data can support the Wi-Fi positioning process and improve accuracy, especially in challenging environments where Wi-Fi signals may be limited. Wi-Fi-based positioning has limitations, including signal interference, multipath effects, environmental changes, and the need for up-to-date databases. Furthermore, the accuracy of Wi-Fi-based positioning can vary depending on the density and distribution of Wi-Fi access points, signal quality, and environmental factors.

[0208] LTE (Long-Term Evolution) signals, commonly used in mobile communications, can also be utilized to estimate the location of a mobile device. This process involves measuring signal characteristics and leveraging cell tower information. A mobile device can measure the received signal strength (RSS) from a nearby LTE base station (cell tower). The RSS value can indicate the relative proximity of the device to different towers. A database or network infrastructure contains information about the location and characteristics of LTE cell towers. This information includes tower coordinates (latitude and longitude) and unique identifiers (cell ID, eNodeB ID).

[0209] A mobile device can identify the serving cell tower and adjacent towers based on the measured RSS value. This step helps determine possible tower candidates that contribute to the device's signal reception. In the case of LTE, trilateration is performed using the identified cell towers and their known coordinates to estimate the device's location by measuring the distance from a known reference point.

[0210] The distance between a mobile device and each candidate cell tower can be estimated based on factors such as received signal strength, signal propagation model, and path loss calculation. Once the distance to at least three towers is determined, a trilateration algorithm can be applied to calculate the device's location. The device's position can be estimated by intersecting circles (or 3D spheres) with radii equal to the estimated distance from the towers. Additional techniques such as time difference of arrival (TDOA) and signal fingerprinting can be used to improve accuracy.

[0211] Using the Time of Arrival (TDOA) technique, mobile devices can measure the time it takes for a signal to travel from the device to a different tower, and the TDOA technique can further improve location estimation.

[0212] Signal fingerprinting can involve collecting signal propagation data from known locations, allowing for the creation of a signal fingerprint database. Current signal characteristics can be compared to this database to determine the location of a device based on the closest match.

[0213] The accuracy of location estimation using LTE signals can vary depending on several factors, including signal strength, signal quality, cell tower density, and environmental conditions. Furthermore, network infrastructure and access to necessary databases are crucial for performing accurate location estimation using LTE signals.

[0214] Calculating the exact position of a mobile device using 5G signals typically involves similar principles to LTE, but with potential improvements in accuracy and capabilities. The mobile device can measure various parameters of the 5G signals it receives, such as signal strength, time of arrival (TOA), signal delay, and signal phase. These measurements are used to gather information about nearby 5G base stations (gNodeBs) and their signals.

[0215] Similar to LTE, there is a database or network infrastructure that contains information about the location and characteristics of 5G gNodeBs. This database includes gNodeB coordinates (latitude and longitude) and unique identifiers.

[0216] Based on the measured signal parameters, the mobile device can identify the serving gNodeB and adjacent gNodeBs. This step helps determine which gNodeBs may be contributing to the device's signal reception. Using trilateration or multilateration techniques, the device's location can be estimated based on the identified gNodeBs and their known coordinates. The distance or time difference of arrival (TDOA) between the mobile device and each candidate gNodeB is estimated using signal parameters and propagation models. Advanced techniques such as angle of arrival (AOA) or phase-based measurements can be used to improve accuracy. Once the distance or TDOA to at least three gNodeBs is determined, a trilateration or multilateration algorithm can be applied to calculate the device's position. The device's position can be estimated by intersecting circles or spheres (or hyperbolas in the case of TDOA) with radii or time differences equal to the estimated distances.

[0217] Similar to LTE, various improvement techniques can be used to enhance accuracy. Combining 5G signals with other positioning technologies such as GPS, Wi-Fi, or sensor fusion can improve accuracy and reliability. Building a database of signal fingerprints from known locations can help match current signal characteristics to accurately estimate device location. Leveraging advanced signal processing techniques and algorithms such as beamforming and large-scale MIMO can improve the accuracy and robustness of 5G-based positioning.

[0218] The accuracy of position estimation using 5G signals can be affected by factors such as signal quality, multipath interference, obstacles, and the availability of infrastructure and databases. Furthermore, the deployment of advanced features in 5G networks, such as higher frequency bands, beamforming, and advanced antenna arrays, can potentially improve the accuracy and reliability of 5G-based positioning. B. Techniques for using uncertainty to determine position fusion

[0219] A mobile device can calculate uncertainty for each of the various techniques that the first mobile device may use to detect a second mobile device. The uncertainty value can vary based on the environment, location, or even the mobile device itself. The uncertainty value can be used by the first mobile device to select a particular technique for detecting the second device. In various embodiments, the uncertainty value can be used to weight a determined combination of locations (e.g., the mean) for the second mobile device. In various embodiments, the uncertainty value may be large enough for the mobile device to ignore a given technique for detecting the second mobile device.

[0220] Figure 15 shows a cross-functional flowchart 1500 for the integration of human detection techniques between a first mobile device 1502 and a second mobile device 1504. Both mobile devices participating in the ranging session are equipped with transceivers (e.g., UWB transceivers) capable of transmitting and receiving radio signals (e.g., UWB signals). These transceivers can use ultrashort pulses with a broad spectrum, enabling accurate distance measurement.

[0221] At a first time, the first mobile device 1502 can transmit a first radio signal. The first radio signal may include a distance measurement request 1508. The distance measurement request 1508 may include an identifier for the first mobile device. In various embodiments, the distance measurement request 1508 may include the time for transmitting the distance measurement request 1508. The radio distance measurement signal may be a variety of radio protocols (e.g., UWB, Bluetooth, BLE, Zigbee, Wi-Fi, etc.).

[0222] The second mobile device 1504 can receive the distance measurement request 1508 using a wireless transceiver. The second mobile device 1504 can store the time of the received distance measurement request and the identifier of the first mobile device. In various embodiments, the second mobile device 1504 can send a response message 1510. The response message 1510 may include the identifier of the first mobile device 1502. The response message 1510 may include the time of reception of the distance measurement request 1508 in the second wireless device 1504. The response message 1510 may include the identifier of the second wireless device 1504. The response message 1510 may include a processing time or delay time. The processing time or delay time can be estimated. The processing time or delay time may be the time it takes for the second mobile device 1504 to receive, process, and send the response message 1510 after receiving the distance measurement request 1508. Processing time or delay time may take into account signal processing, pulse detection and timing, data transmission time, and system latency.

[0223] The first mobile device 1502 can receive the response message 1510 at the second time.

[0224] In 1514, the first mobile device 1502 can determine a range value between the first mobile device 1502 and the second mobile device 1504 by using the first transmission time and the second time to determine the time difference. Processing time or delay time may be subtracted from the time difference to determine the time of flight (TOF) measurement. The TOF measurement time may be used to determine the round-trip distance between the first mobile device 1502 and the second mobile device 1504. The round-trip distance may be halved to determine a range value between the first mobile device and the second mobile device.

[0225] Figure 15 shows a ranging round 1512 consisting of three exchanges, each having a ranging request 1508 and a ranging response 1510. Although three exchanges are shown, ranging round 1512 may have more or fewer exchanges between mobile devices. Although one ranging round 1512 is shown, more ranging rounds may be shown.

[0226] In various embodiments, the second mobile device 1504 can determine a range value between the first mobile device 1502 and the second mobile device 1504.

[0227] In 1518, the first mobile device can determine the uncertainty value in range measurement. The first mobile device 1502 can evaluate the quality and reliability of the received radio signal. Factors such as signal strength, signal-to-noise ratio (SNR), multipath effects, and interference levels can affect the accuracy of range measurement. By analyzing these signal quality parameters, the first mobile device 1502 can estimate the uncertainty associated with range measurement.

[0228] Statistical methods can be used to evaluate variability and uncertainty in range measurements. The first mobile device 1502 can collect multiple range measurements over time and analyze them using statistical techniques such as standard deviation, variance, or confidence interval. These statistical measures provide estimates of uncertainty or error range associated with distance measurements.

[0229] Uncertainty in range measurement can propagate from various sources, including signal noise, timing errors, and hardware limitations. By considering the individual sources of uncertainty and their individual error contributions, the first mobile device 1502 can estimate the overall uncertainty in range measurement through error propagation calculations.

[0230] Wireless systems often require calibration procedures to account for system-specific biases or inaccuracies. During calibration, the device can determine calibration uncertainty, which represents the uncertainty in the calibration process itself. This calibration uncertainty can be used to quantify the overall uncertainty in range measurements.

[0231] Mobile devices can take into account environmental factors that may affect wireless range measurement. For example, variations in temperature, humidity, and electromagnetic interference can introduce uncertainty. By monitoring and considering these environmental factors, the first mobile device 1502 can estimate the relevant uncertainties in range measurement.

[0232] If the first mobile device 1502 utilizes multiple positioning technologies such as GPS, Wi-Fi, or sensor fusion, the first mobile device 1502 can leverage the strengths of each technology to estimate uncertainty. By combining wireless range measurements with other positioning data, the device can perform data fusion techniques such as Kalman filtering or Bayesian estimation to estimate uncertainty in the final position estimate.

[0233] The first mobile device 1502 can receive the GNSS signal 1520 using the GNSS system 1506. The first mobile device 1502 can use the received GNSS signal 1520 to determine its position. The position may not be accurate or may have a high degree of uncertainty due to the noise problem described above. The second mobile device 1504 can receive the GNSS signal 1520 using the GNSS system 1506 and determine the second position of the second mobile device 1504.

[0234] The first mobile device 1502 can receive position information 1522 of the second mobile device 1504 via a data channel between the first mobile device 1502 and the second mobile device 1504. In various embodiments, UWB devices can establish direct links with each other to exchange data, or they can operate in broadcast mode, where data is transmitted simultaneously to multiple devices within range.

[0235] The first mobile device 1502 can determine the second range value 1524 using the position of the first mobile device 1502 and the position information 1522 of the second mobile device.

[0236] The first mobile device 1502 can use GNSS-based position information to determine the uncertainty value 1526 for the range between the first mobile device 1502 and the second mobile device 1504.

[0237] The uncertainty of GNSS position information refers to a measure of potential errors or lack of accuracy when determining an exact location using GNSS signals. The quality of the received GNSS signals affects the accuracy of position determination. Insufficient signal conditions, such as low signal strength, multipath interference, or signal blockage by obstacles, can introduce errors and increase uncertainty. The geometric arrangement of GNSS satellites within the receiver's field of view plays an important role in position accuracy. Ideally, the receiver should have a diverse set of satellites spread out in the sky to ensure better triangulation and improve accuracy. Insufficient satellite geometry with satellites clustered in a particular region or a small number of visible satellites can increase uncertainty.

[0238] The Earth's atmosphere can introduce errors into GNSS signals, mainly due to atmospheric delay and signal propagation effects. Factors such as ionospheric and tropospheric delays, which vary with weather and environmental conditions, can affect the accuracy of position determination and contribute to uncertainty.

[0239] The GNSS receiver itself can introduce errors, including clock errors, receiver noise, multipath effects, and limitations in signal processing algorithms. These errors can affect the accuracy of position estimation and contribute to uncertainty.

[0240] GNSS receivers rely on accurate information about the position, orbit, and clock corrections of satellites (known as ephemeris and almanac data). Errors or outdated data in these parameters can lead to inaccuracies in position determination and increase uncertainty.

[0241] Differential GNSS techniques, such as using a reference station or satellite-based augmentation system (SBAS), can improve accuracy by providing correction data. However, errors or discrepancies in differential corrections can still introduce uncertainty.

[0242] Errors in individual measurements, such as range measurements or satellite clock errors, can propagate and accumulate through positioning algorithms, leading to increased uncertainty in the final position estimate.

[0243] To mitigate and quantify uncertainty in GNSS positioning information, techniques such as error estimation, statistical analysis, and data fusion with other positioning technologies (such as sensor fusion or Wi-Fi positioning) may be employed. Furthermore, by utilizing advanced GNSS receivers with improved signal processing algorithms, multi-constellation support, and real-time correction, accuracy in GNSS positioning can be enhanced and uncertainty reduced.

[0244] The first mobile device 1502 can determine which positioning technique provides the lowest uncertainty value and select that technique to determine the location of the second mobile device 1504. For example, if the GNSS position has significantly more uncertainty than the ranging positioning technique, the position of the second mobile device using the ranging technique may be used. If the ranging position has significantly more uncertainty than the GNSS derived position for the second mobile device, the GNSS derived position may be used.

[0245] In various embodiments, each of the various positioning techniques can be used. The positions determined for each technique can be averaged to determine the approximate position of the second mobile device 1504. In various embodiments, each of the various positioning techniques can be weighted before determining the average. In various embodiments, the weighting can take into account various uncertainty values ​​for each positioning technique, in addition to the precision of each positioning technique.

[0246] In 1528, the first mobile device 1502 can determine the position vector from the first mobile device 1502 to the second mobile device 1504 using various positioning techniques and sensor data available on the mobile device. Both mobile devices can use GNSS receivers to determine their individual positions. Each device calculates its latitude, longitude, and altitude.

[0247] Once the individual positions are determined, the mobile devices can calculate the relative distance between them. This can be achieved by using the latitude, longitude, and altitude information of both devices to calculate the Euclidean distance in a three-dimensional coordinate system.

[0248] Mobile devices often include sensors such as accelerometers, gyroscopes, and magnetometers to measure orientation. By analyzing sensor data, each device can estimate its orientation or direction of travel relative to the Earth's magnetic field or other reference frame.

[0249] Each device can use this orientation information to calculate its relative bearing to other devices. Bearing represents the direction from one device to another and is typically measured as an angle relative to true north or another reference direction.

[0250] Using relative distance and orientation information, mobile devices can calculate a position vector from one device to another. The position vector consists of magnitude (distance) and direction (orientation), representing the displacement from one device to the other in a coordinate system.

[0251] If necessary, the position vector can be converted to a different coordinate system, such as a Cartesian coordinate system (x, y, z) or a local coordinate system, based on the specific requirements of the application.

[0252] The first mobile device 1502 can determine a position arrow to the second mobile device. To determine the arrow based on the position vector, the first mobile device 1502 can use the magnitude and direction of the vector to represent the length and orientation of the arrow.

[0253] The magnitude of the position vector represents the distance or displacement between two points (for example, the position of the first mobile device 1502 and the position of the second mobile device 1504). The first mobile device 1502 can use this magnitude to determine the length of an arrow. For example, a larger magnitude allows the first mobile device 1502 to draw a longer arrow, while a smaller magnitude corresponds to a shorter arrow.

[0254] The direction of a position vector represents the orientation or direction from one point to another. The first mobile device 1502 can use this direction to determine the direction of an arrow. For example, if the direction is north, the first mobile device may draw an upward-pointing arrow. Similarly, if the direction is southeast, the arrow may be drawn diagonally towards the southeast.

[0255] The first mobile device 1502 can consider a coordinate system or reference frame in which the position vector is defined. The first mobile device 1502 can ensure that an arrow is drawn within this coordinate system to accurately represent direction.

[0256] In 1530, the first mobile device 1502 may use graphical tools or software to draw directional information (e.g., a pointer or arrow) based on the determined length and orientation. This may include using vector drawing tools or incorporating arrow symbols into the desired representation. The representation of directional information (e.g., a pointer or arrow) may vary depending on the specific context and visualization requirements. The user may select different styles, colors, or annotations to improve the clarity and meaning of the representation of directional information (e.g., a pointer or arrow).

[0257] The accuracy of the representation of directional information (e.g., pointers or arrows) depends on the accuracy of the position vector calculation. C. Modules for Sequencing Fusion Techniques

[0258] Figure 16 shows the priority order or hierarchy 1600 for estimating arrows to locate the device being searched using the searcher device. At 1602, the technique can determine whether the visual inertial odometry (VIO) technique is available for the device being searched. The first priority is to use a combination of satellite and VIO techniques from both the searcher and the device being searched.

[0259] In 1606, the device being searched can determine whether it is stationary or not moving. If the device being searched is not moving, satellite techniques (e.g., GNSS location techniques) may be used.

[0260] In 1610, if the device under investigation is moving, a fusion process may be used to determine the location of the device under investigation. The location of the device under investigation can be determined using the People Searcher (PF) process 1612 or the Core Locator (CL) process 1614.

[0261] The PF process 1612 may be a combination of visual inertial odometry from the explorer device, along with ranging (e.g., UWB ranging) and pedestrian dead reckoning or delta velocity techniques. The CL process 1614 can utilize the location of the explorer device, the direction of travel of the explorer device, and the location of the explored. Both the PF process 1612 and the CL process 1614 may result in angular errors and uncertainty errors. In 1616, the PF process 1612 and the CL process 1614 can select the solution that yields the lowest uncertainty value.

[0262] The fused logic can be a loosely coupled solution that selects a solution from either a location-based estimator or a particle filter, e.g., a UWB-based particle filter. If a solution is available from both estimators, the arrow from the estimator with the lowest uncertainty is selected. A range value (e.g., UWB distance) may be used to detect anomalies in the location solution and determine the minimum distance below which a location-based arrow will not be generated. Hysteresis checks are also implemented to minimize variation between location-based arrows and range-based arrows (e.g., UWB arrows) when both solutions are available, if necessary to bounce. D. Exemplary combinations of techniques for fusion

[0263] The following paragraphs will explain the integration of various location techniques. 1. GNSS navigation combined with visual inertial odometry signals

[0264] Combining GNSS signals with visual inertial odometry (VIO) measurements can improve positioning and navigation accuracy and robustness in several scenarios, particularly when GNSS signals are degraded or unavailable. This combination is often referred to as sensor fusion or sensor integration. GNSS signals and VIO measurements can be combined as follows:

[0265] Visual inertial odometry may involve using a combination of visual sensors (such as cameras) and inertial sensors (gyroscopes and accelerometers) to estimate the position and orientation of a device relative to its initial starting point. VIO techniques can typically provide high-speed measurements at tens or hundreds of Hz, enabling real-time tracking of motion.

[0266] Before combining the measurements, calibration between the GNSS receiver and the VIO system may be performed to align their coordinate systems and correct for any sensor bias or time offset. The GNSS receiver and VIO system may be synchronized with respect to time. This synchronization can be achieved using techniques such as timestamps or time interpolation.

[0267] Once GNSS and VIO measurements are properly calibrated and synchronized, fusion algorithms can be employed to combine the data. Common fusion techniques include Kalman filters and particle filters.

[0268] Kalman filters can be used in sensor fusion. Kalman filters can employ mathematical models to estimate the state of a device (position, velocity, orientation) based on measured values ​​and their associated uncertainties. By fusing GNSS and VIO measurements, Kalman filters can provide accurate and consistent estimates of the device's position and orientation.

[0269] Particle filtering, also known as Monte Carlo localization, can be used in sensor fusion. Particle filtering works by maintaining a set of particles that represent possible states of the system. These particles are updated and resampled based on the likelihood of measurements from both the GNSS and VIO systems.

[0270] The fusion algorithm can assign weights or confidence levels to GNSS and VIO measurements based on their reliability and accuracy. These weights can be dynamically adjusted based on signal quality, the presence of signal obstructions, or the accuracy of the VIO system.

[0271] The fused output provides a more accurate and reliable estimate of the device's position, velocity, and orientation. This combined information can be used for navigation, mapping, augmented reality, or any application requiring precise localization.

[0272] The fusion of GNSS signals with VIO measurements can be a complex task, and various algorithms and techniques exist for different scenarios and requirements. Advanced techniques such as tightly coupled integration can further improve fusion accuracy by considering the interdependence between the GNSS system and the VIO system. 2. Fusion of local measurements (distance measurement, odometry, and GPS)

[0273] Combining wireless ranging signals, visual inertial odometry (VIO) measurements, and GNSS measurements can provide more robust and accurate positioning and navigation solutions. This fusion of multiple sensors is generally referred to as sensor fusion or sensor integration. These three types of measurements can be combined as follows:

[0274] Wireless ranging signals, such as those used in Wi-Fi, Bluetooth, or ultra-wideband (UWB) technologies, can provide distance or range measurements between a device and known reference points in the environment. These reference points may be fixed beacons or access points with known positions.

[0275] As mentioned above, GNSS systems provide positioning information based on satellite signals. GNSS measurements include the device's latitude, longitude, altitude, and sometimes speed.

[0276] VIO uses visual sensors (e.g., cameras) and inertial sensors (e.g., gyroscopes, accelerometers) to estimate the device's position, orientation, and motion relative to its starting point.

[0277] Before fusing measurements, calibration and synchronization between the wireless ranging system, GNSS system, and VIO system are crucial. To ensure accurate fusing, coordinate system alignment, sensor bias correction, and time synchronization are necessary.

[0278] Radio range measurements, VIO estimates, and GNSS measurements must be correlated with each other. This correlation may involve determining which radio range measurement corresponds to the current VIO or GNSS measurement based on timing, proximity, or other relevant criteria.

[0279] Measurements can be combined using fusion algorithms such as extended Kalman filters (EKF) or particle filters. These algorithms can generate optimized estimates of the device state (position, velocity, orientation) by considering the uncertainty and intensity of each measurement source.

[0280] The fusion algorithm can assign weights or confidence levels to measurements based on their reliability, precision, and current operating conditions. The weights can be dynamically adjusted to account for the varying quality of each measurement source.

[0281] The fusion algorithm combines radio ranging, VIO, and GNSS measurements to generate a more accurate and robust estimate of the device's position, velocity, and orientation. The fused output can be used for navigation, localization, mapping, augmented reality applications, or any use case requiring precise positioning information.

[0282] By combining wireless ranging signals, VIO measurements, and GNSS measurements, the strength of each sensor type can be leveraged to compensate for their individual limitations. This integration enables more reliable positioning and navigation solutions, particularly in challenging GNSS conditions or environments with limited visual features for VIO.

[0283] Fusion techniques can cross-check between GNSS-based range values ​​and distance values ​​(e.g., UWB distance measurement) when both are available. This cross-checking can make it possible to identify potentially inaccurate GNSS-based arrows.

[0284] For example, if the distance measurement (e.g., UWB range value) is significantly smaller than the GNSS-based distance, the system should add pessimism to the GNSS-based arrow and potentially not yield to it, because it means that one of those two GNSS fixes could be an error, or potentially both of them could be errors.

[0285] The system can generate a GNSS quality metric from a composite arrow based on GNSS-based location. The GNSS quality metric can be compared to a ranging statistical metric (e.g., one provided by UWB ranging). A quality metric with a higher confidence level can statistically correct other range values.

[0286] In various embodiments, GNSS-based arrows can use points of interest corresponding to locations where a person was nearby (i.e., when odometry and ranging are unavailable). For example, two people, the person being searched and the searcher, agree to meet at a defined location (e.g., a Starbucks location). The Starbucks location can be stored in a map in memory. When odometry and ranging are unavailable, the searcher device can rely on the Starbucks location from the map in memory. As the searcher device approaches the Starbucks location, since each point in the map has corresponding global coordinates, a better composite arrow can be constructed using that location. 3. PDR+GNSS+VIO

[0287] Combining pedestrian dead reckoning (PDR) techniques with GNSS and visual inertial odometry (VIO) measurements can provide enhanced positioning and navigation for pedestrians, particularly in scenarios where GNSS signals or visual features are limited. These three types of measurements can be integrated as follows:

[0288] Pedestrian dead reckoning (PDR) relies on measuring and integrating the number of steps and changes in orientation to estimate a pedestrian's movement. These motion-related measurements can be captured using specialized sensors such as accelerometers, gyroscopes, magnetometers, or inertial measurement units (IMUs).

[0289] GNSS systems such as GPS can provide absolute positioning information in outdoor environments. However, in urban areas or locations with tall buildings or signal obstructions, GNSS signals may be degraded or unavailable.

[0290] VIO combines visual sensors (e.g., cameras) and inertial sensors (e.g., accelerometers, gyroscopes) to estimate the movement of a device and its position relative to its starting point. VIO is effective in environments with sufficient visual features, but it can suffer from drift over time.

[0291] Before combining the measured values, calibration and synchronization between the PDR system, GNSS system, and VIO system are essential. Coordinate system alignment, sensor bias correction, and time synchronization ensure accurate fusion.

[0292] PDR steps and orientation changes can be associated with corresponding timestamps of GNSS and VIO measurements. This association allows for the alignment of PDR estimates with absolute position information provided by GNSS and relative motion information obtained from VIO.

[0293] Measurements can be combined using fusion algorithms such as Extended Kalman Filter (EKF), Particle Filter, or Unscented Kalman Filter (UKF). The algorithm integrates PDR, GNSS, and VIO measurements, taking into account their uncertainties and intensities, to estimate the pedestrian's position, velocity, and orientation.

[0294] The fusion algorithm assigns weights or confidence levels to measurements based on their reliability, accuracy, and current operating conditions. These weights can be dynamically adjusted to adapt to the quality of each measurement source.

[0295] The fusion algorithm combines PDR, GNSS, and VIO measurements to generate optimized estimates of a pedestrian's position, speed, and orientation. The fused output compensates for the limitations of individual sensors, providing improved positioning and navigation information.

[0296] The integration of PDR, GNSS, and VIO measurements enables continuous positioning and navigation even in areas where GNSS availability or visual features are limited. By fusing complementary information from multiple sensors, the system can reduce drift, improve accuracy, and maintain reliable positioning for pedestrian users.

[0297] Other positioning combinations can be used. E. Flow

[0298] Figure 17 is a flowchart of process 1700 for detecting a mobile device using a fusion technique, according to an example of the present disclosure. For example, one or more process blocks in Figure 17 may be executed by a mobile device.

[0299] In block 1705, process 1700 may include transmitting a radio ranging signal at a first time. The radio ranging signal may be a variety of radio protocols (e.g., UWB, Bluetooth, BLE, Zigbee, Wi-Fi, etc.). A mobile device may have a radio transmitter. For example, a mobile device may transmit a radio ranging signal at a first time as described above.

[0300] In block 1710, process 1700 may include receiving a radio response signal from a second mobile device in a second time period. The radio response signal may be received by one or more antennas on the mobile device. The radio response signal may use the same radio protocol as the radio ranging signal. For example, the device may receive a radio response signal from a second mobile device in a second time period, as described above.

[0301] In block 1715, process 1700 may include determining a first range value between a first mobile device and a second mobile device based on the difference between a first time and a second time. A radio signal can travel at the speed of light (c). If the first mobile device (e.g., a seeker device) knows the transmission time of the radio ranging signal, the reception time of the radio response, and the processing time of the second mobile device (e.g., the device being sought), the first mobile device can calculate the range value by multiplying the time delay (e.g., reception time - transmission time - processing delay) by the speed of light (c). For example, the device may determine a first range value between a first mobile device and a second mobile device based on the difference between a first time and a second time, as described above.

[0302] In block 1720, process 1700 may include determining a first uncertainty in a first range of values. For example, the device may determine a first uncertainty in a first range of values ​​as described above.

[0303] Uncertainty in radio ranging signals can be determined through various methods and considerations. Some approaches that may be used to assess uncertainty in radio ranging signals include measurement error analysis, environmental analysis, signal-to-noise ratio analysis, calibration and reference standard analysis, and Monte Carlo simulation.

[0304] One way to determine uncertainty is to analyze the measurement error associated with the ranging signal. This involves characterizing the accuracy and precision of the measuring system used for ranging. Factors such as instrument calibration, noise, interference, and signal processing algorithms can contribute to the measurement error. Statistical techniques such as standard deviation or root mean square error (RMSE) can be used to quantify uncertainty based on the measurement data.

[0305] Wireless ranging signals can be affected by environmental conditions such as multipath propagation, interference, and atmospheric effects. These factors can introduce uncertainty into the measurement. To determine this uncertainty, mobile devices may need to consider the specific environment in which the ranging is performed and evaluate the impact of these factors on signal quality.

[0306] The signal-to-noise ratio (SNR) of a ranging signal provides a measure of signal strength relative to background noise. Higher SNR values ​​generally indicate better signal quality and lower uncertainty. By evaluating the SNR, mobile devices can estimate the uncertainty in the ranging signal.

[0307] Calibration of ranging systems and the use of reference standards can help quantify uncertainty. Calibration involves comparing measurements from the ranging system to known standards or criteria. By understanding the calibration process and associated uncertainties, mobile devices can determine the overall uncertainty in ranging signals.

[0308] Monte Carlo simulation is a computational technique that involves running multiple simulations with randomly varied input parameters to estimate uncertainty. In the context of radio ranging, Monte Carlo simulation can be used to model uncertainties associated with factors such as noise, interference, and environmental conditions. By running numerous simulations, mobile devices can analyze the distribution of measurement results and determine the range of uncertainty.

[0309] The specific methods and levels of uncertainty assessment may depend on the ranging technology, measurement system, and application context.

[0310] In block 1725, process 1700 may include determining a first location of the first mobile device based on a first GNSS signal acquired by the first mobile device. For example, the device may determine a first location of the first mobile device based on a first GNSS signal acquired by the first mobile device, as described above.

[0311] In block 1730, process 1700 may include receiving the second location of the second mobile device based on the second GNSS signal via a data channel between the first mobile device and the second mobile device. For example, the device may receive the second location of the second mobile device based on the second GNSS signal via a data channel between the first mobile device and the second mobile device, as described above.

[0312] In various embodiments, the data channel includes a narrowband channel controlled by an ultra-wideband processing chip.

[0313] In block 1735, process 1700 may include determining a second range value between a first location and a second location. For example, the device may determine a second range value between a first location and a second location as described above.

[0314] In block 1740, process 1700 may include determining a second uncertainty in the second range value. For example, the device may determine a second uncertainty in the second range value as described above.

[0315] Determining uncertainty in Global Navigation Satellite System (GNSS) positioning involves considering various factors that contribute to positioning errors. GNSS uncertainty can take into account satellite geometry, signal quality, receiver error sources, differential positioning, error propagation and statistical analysis, as well as positioning solution quality indicators.

[0316] The geometry of satellites within the field of view affects the accuracy of GNSS positioning. Poor satellite geometry, such as satellites clustered in small areas of the sky or at low elevation angles, can increase positioning uncertainty. Evaluating degradation rate (DOP) parameters, such as positional degradation rate (PDOP) or horizontal degradation rate (HDOP), can provide insights into satellite geometry and its impact on positioning uncertainty.

[0317] The quality of the GNSS signal received by the receiver is important. Factors such as signal strength, multipath interference, and ionospheric / tropospheric effects can introduce errors. Evaluating parameters such as the signal-to-noise ratio (SNR), carrier-to-noise ratio (C / N0), and receiver-reported quality indicators (e.g., signal quality indicator, signal-space precision) can help assess the reliability of the signal and estimate its uncertainty.

[0318] GNSS receivers have inherent error sources that contribute to positioning uncertainty. These errors can include clock errors, receiver noise, multipath effects, and receiver bias. Understanding the receiver's specifications and performance characteristics can help quantify the uncertainty associated with these error sources.

[0319] Differential GNSS techniques involve using a reference receiver with known coordinates to improve positioning accuracy. Differential correction can mitigate common errors such as atmospheric delay and satellite clock errors. By utilizing differential positioning, uncertainty in position estimates can be reduced.

[0320] Errors in individual measurements can propagate throughout the positioning calculation. Applying statistical analysis methods such as error propagation techniques (e.g., root of sum of squares) can help estimate the overall uncertainty in the final position solution. Understanding the statistical properties of the error sources, such as their standard deviation or covariance matrix, is essential for accurate error propagation.

[0321] GNSS receivers often provide indicators of the quality of their positioning solutions, such as horizontal accuracy estimates (HAE), vertical accuracy estimates (VAE), or estimated position error (EPE). These indicators provide a measure of uncertainty in position estimates and can be used to assess the reliability of GNSS solutions.

[0322] In block 1745, process 1700 may include determining the position vector between the first mobile device and the second mobile device using a first range value, a first uncertainty, a second range value, and a second uncertainty. For example, the device may determine the position vector between the first mobile device and the second mobile device using the first range value, a first uncertainty, a second range value, and a second uncertainty, as described above.

[0323] In various embodiments, the first device and the second device are in motion.

[0324] In block 1750, process 1700 may include displaying a pointer based on a position vector. For example, a device may display a pointer based on a position vector, as described above.

[0325] In various embodiments, a mobile device may include one or more processors and memory coupled to one or more processors. The memory may store instructions that cause one or more processors to perform any one or more of the operations described above.

[0326] In various embodiments, a non-temporary computer-readable medium can store instructions that, when executed on one or more processors, perform any one or more of the operations described above.

[0327] Figure 17 shows an exemplary block of process 1700, but note that in some implementations, process 1700 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those shown in Figure 17. Additionally or alternatively, two or more blocks of process 1700 may be executed in parallel. VI. Efficient transfer between devices

[0328] Mobile devices can transmit information between devices using wireless signals. The larger the information, the greater the bandwidth required to transfer it. The following technique describes a geodetic displacement technique used by one mobile device to detect a second mobile device. Geodetic displacement techniques can result in efficient information transfer between devices. A. Grid with reference point and offset

[0329] Figure 18 shows an exemplary layout for the geodetic displacement technique. Geodetic displacement is a positioning technique that involves calculating the coordinates of a point based on its displacement from a known reference point in a geodetic coordinate system.

[0330] This technique can be initiated by establishing one or more control points with known coordinates using a reliable positioning method such as GNSS (Global Navigation Satellite System) or surveying techniques. These control points can serve as origins for geodetic displacement calculations.

[0331] This technique may include determining the displacement vector between a reference point and a target point for the location of a second mobile device. The displacement vector may consist of the difference in latitude, longitude, and possibly altitude (if applicable) between the reference point and the target point.

[0332] This technique can use geodetic coordinate systems such as the World Geodetic System (WGS84) or specific local geodetic data. These coordinate systems can take into account the curvature of the Earth and provide an accurate representation of points on Earth.

[0333] This technique uses geodetic displacement vectors to determine the coordinates of a target point by adding the displacement value to the coordinates of a reference point or subtracting the displacement value from the coordinates of a reference point. The latitude and longitude coordinates are updated based on the displacement value, and the altitude may also be adjusted where applicable.

[0334] The accuracy of positioning techniques using geodetic displacement depends on the accuracy of the coordinates of the reference points and the accuracy of the displacement vector measurements. The reliability of the geodetic coordinate system used is also important for accurate positioning.

[0335] This technique can account for any errors or uncertainties related to displacement vector measurements or the coordinates of the reference point. Appropriate error estimation and propagation techniques can be applied to assess the overall uncertainty at the determined position.

[0336] This positioning technique, which uses geodetic displacement, can be commonly used in applications where the displacement between two points is known or measured, such as surveying, geodesy, and the establishment of geodetic control networks. It provides a method for determining the position of a target point relative to a known reference point in a geodetic coordinate system.

[0337] Figure 18 shows the location 1802 of the first mobile device. Figure 18 shows the maximum operating range 1804 of the location technique. Figure 18 shows various default reference points 1806 (identified as 1806a, 1806b, 1806c, 1806d, 1806e, 1806f, 1806g, 1806h, 1806i, 1806j, 1806k).

[0338] As described above, the candidate peer locations 1808 for the second mobile device can be set distances from the default reference point 1806. Several candidate peer locations 1808 are shown as 1808a, 1808b, 1808c, 1808d, 1808e, 1808f, 1808g, 1808h, 1808i, 1808j, and 1808k. As shown in Figure 18, each of the candidate peer locations 1808 is a set distance from the default reference point.

[0339] This technique can use the maximum operating range 1804 when determining which of the candidate peer locations 1808 is the exact position of the second mobile device. Only candidate peer location 1808b is within the maximum operating range 1804. Therefore, the mobile device can select candidate peer location 1808b as the location of the second mobile device. The position of the second mobile device can be represented as a simple offset 1810 from a default reference point 1806b. This simple offset 1810 can be sent to the first mobile device to determine the location of the second mobile device. B. Sequence diagram for communicating position information

[0340] Figure 19 shows a cross-functional flowchart 1900 for efficient position transfer between the first mobile device 1902 and the second mobile device 1904. The GNSS system 1906 can provide positioning information as described above.

[0341] In 1908, the first mobile device 1902 can store multiple reference points in the memory of the first mobile device 1902. The first mobile device 1902 can store a grid of reference points for a global reference frame. In various embodiments, the second mobile device 1902 can store multiple reference points in the memory of the second mobile device 1904.

[0342] At 1910, the first mobile device 1902 can receive GNSS information. The first mobile device 1902 can use the measurements made by the first mobile device 1902 to determine the first location of the first mobile device within a global reference frame.

[0343] In 1912, the first mobile device 1902 can determine the radio signal transmitted from the second mobile device 1904. Based on the radio signal 1912, the first mobile device can determine the relative position between the first mobile device 1902 and the second mobile device 1904. For example, the radio signal may be a UWB signal, and the first radio device can determine the range to the second radio device via UWB ranging. In various embodiments, the radio signal may include odometry information from the second radio device.

[0344] In 1914, the first mobile device can determine the relative position of the second wireless device. In various embodiments, the relative position of the second wireless device can be determined by analyzing the direction or angle at which the wireless signal arrives at a plurality of antennas or sensor arrays on the device, and the relative position can be estimated. The angle of arrival technique may typically require an array of antennas or sensors to accurately determine the angle of the arriving signal.

[0345] The first mobile device 1902 can establish a wireless communication channel between the first mobile device and the second mobile device. To establish a communication channel, wireless transceivers (e.g., Wi-Fi, Bluetooth, UWB) can synchronize their timing and carrier frequencies. In various embodiments, the communication channel may be one of Bluetooth, Zigbee, or peer-to-peer Wi-Fi. Synchronization ensures that both the transmitter and receiver are operating on the same time scale and frequency reference. Common synchronization techniques include time-division multiple access (TDMA) or the use of preamble sequences for timing and frequency synchronization.

[0346] Wireless transceivers can perform an initialization process to set up communication channel parameters. This process may include exchanging control information between communication devices, such as channel settings, power levels, modulation schemes, and synchronization parameters. Initialization ensures that both devices are configured to communicate with each other.

[0347] In 1916, the first mobile device 1902 can send a communication message to the second mobile device 1904. The second mobile device 1904 can receive the communication message. The second mobile device can send a response message. The response message may include control information for the second mobile device 1904.

[0348] In 1917, the first mobile device was capable of receiving response messages.

[0349] Wireless transceivers estimate the characteristics of the communication channel to account for signal propagation effects and multipath interference. Channel estimation involves analyzing the received signal and estimating channel parameters such as path delay, amplitude, and phase shift. This information is crucial for efficient data transmission and reception.

[0350] Once a communication channel is established, the radio transceiver can transmit data using the selected modulation scheme. The data is encoded onto radio pulses, and the modulated pulses are transmitted wirelessly.

[0351] The receiving end's radio transceiver demodulates and decodes the received radio pulses to reconstruct the transmitted data. The receiver performs the reverse process of the transmitter, using synchronization, demodulation, and decoding techniques to extract information from the received signal.

[0352] Wireless transceivers can use error correction coding techniques to improve the reliability of data transmission. Error correction coding adds redundancy to the transmitted data and allows the receiver to detect and correct errors introduced during transmission.

[0353] In 1918, the second mobile device 1904 can determine the offset value between a reference point and its position in a global reference frame. The offset value can be stored in the memory of the second mobile device 1904. The offset value can be transmitted to the first mobile device using a radio channel. The offset value may be only a few bytes, as an offset from the latitude and longitude of the selected reference point. The offset value can be measured by the second mobile device 1904.

[0354] In 1919, the first mobile device 1902 can receive an offset value via a wireless channel.

[0355] In 1920, the first mobile device 1902 can identify a stored reference point in a grid of reference points corresponding to a first reference point, based on the first location of the first mobile device 1902, the relative position between the first mobile device 1902 and the second mobile device 1904, and an offset value.

[0356] In 1922, the first mobile device 1902 can determine the second location of the second mobile device 1904 based on a stored reference point, the reference point being separated by at least a first threshold distance. The first threshold distance may be less than an offset value.

[0357] The first mobile device can store the second location in the memory of the second mobile device (1902).

[0358] In some embodiments, a mobile device can formulate a method of transmitting data that takes into account the fact that the device being searched for may only be in a specific location, based on the fact that these two devices are communicating with each other. This technique can be used when a radio signal is detected between the two devices. This technique can reduce the size (e.g., number of bytes) required for the information file. In various embodiments, only about 16 bytes may be used to obtain location information. In various embodiments, this location information may be reduced to about 3 bytes due to bandwidth constraints.

[0359] Using this technique, a global reference frame can be stored or generated on all devices, including the two devices in question. The set of grids may be defined worldwide with, for example, a precision of 1,000 meters, but could be with a precision of 50 meters. Offsets (e.g., latitude, longitude) can be defined relative to the grid. The devices do not know the original grid points, but after several measurements it becomes clear that only one original grid point is possible. In this way, the offsets can be defined with as few bytes as possible. The offsets for the second mobile device can be generated and transmitted to the first wireless device. The offsets can be communicated via a data channel. C. Efficient methods for transferring position information

[0360] Figure 20 is a flowchart of process 2000 for efficiently transferring position information, according to an example of the present disclosure. For example, one or more process blocks in Figure 20 may be executed by a mobile device.

[0361] In block 2005, process 2000 may include storing a grid of reference points for the global reference frame. The grid reference frame may be stored in the memory of each mobile device. For example, a mobile device may store a grid of reference points for the global reference frame as described above.

[0362] In block 2010, process 2000 may include determining the first location of the first mobile device within a global reference frame using measurements made by the first mobile device. In various embodiments, the measurements may be determined using one of several positioning techniques (e.g., GNSS, ranging, RSSI, VIO, PDR, RFID, etc.). The first location of the first mobile device may be stored in the mobile device's memory. For example, the mobile device may determine the first location of the first mobile device within a global reference frame using measurements made by the first mobile device, as described above.

[0363] In block 2015, process 2000 may include detecting a radio signal transmitted from a second mobile device. The radio signal indicates that the two devices are local to each other and can therefore share other types of position information besides GNSS position. The radio signal may be one of various radio signaling protocols (e.g., Bluetooth, BLE, Wi-Fi, UWB, Zigbee, etc.). For example, a device may detect a radio signal transmitted from a second mobile device as described above. The radio signal may also provide position information, ranging information such as signal strength or time of flight.

[0364] In block 2020, process 2000 may include determining the relative position between a first mobile device and a second mobile device based on a radio signal. For example, the devices may determine the relative position between the first mobile device and the second mobile device based on a radio signal, as described above. For example, the relative position may be determined using ranging or odometry information.

[0365] Determining direction or bearing based on radio signals typically involves using signal strength or signal propagation characteristics. These techniques may include RSSI, multiple antenna arrays, magnetic field sensors, and angle of arrival techniques.

[0366] Mobile devices can measure the strength of radio signals from known sources. By comparing RSSI values ​​from multiple sources or antennas, the device can estimate the relative direction or orientation. This technique is commonly used in Wi-Fi-based indoor positioning systems.

[0367] Some mobile devices are equipped with multiple antennas or antenna arrays. By analyzing the signal strength or phase difference between the antennas, the device can determine the direction of arrival (DOA) of a radio signal. This technique is used in beamforming and direction-finding applications.

[0368] Some mobile devices have built-in magnetic field sensors, such as magnetometers. These sensors can detect changes in the Earth's magnetic field caused by nearby radio signals. By analyzing the magnetic field fluctuations, the device can estimate the direction or orientation of the signal source.

[0369] The Angle of Arrival (AoA) technique involves using multiple antennas or antenna arrays to measure the angle from which a radio signal arrives. By analyzing the phase difference or time delay between antennas, a mobile device can estimate the direction or orientation of the signal source.

[0370] In block 2025, process 2000 may include establishing a wireless communication channel with a second mobile device. For example, the device may establish a wireless communication channel with the second mobile device as described above.

[0371] Ultra-wideband (UWB) chips use a specific set of protocols and techniques to open communication channels. UWB chips can establish communication channels as follows:

[0372] UWB chips typically operate in a frequency range of several GHz and use pulses with extremely short durations. When two UWB devices want to establish a communication channel, they begin by performing a process called channel setup. During this process, the devices exchange information necessary to synchronize their timing and frequency characteristics.

[0373] UWB chips excel in accurate ranging and positioning capabilities. Once channel setup is complete, UWB devices can exchange ranging information. This information includes precise timestamps and measurements of the time it takes for UWB pulses to travel between devices. By analyzing the time of flight of these pulses, devices can calculate the distance or range between them.

[0374] After ranging and positioning, the UWB device can proceed to transmit data over the established communication channel. Due to its wide bandwidth, the UWB chip can transmit data at high data rates. The communication can include various types of data, such as voice, audio, video, or sensor data.

[0375] UWB chips can also incorporate security features to ensure secure communication. Encryption algorithms and authentication protocols may be implemented to protect transmitted data from unauthorized access or tampering.

[0376] In various embodiments, the communication channel may be Bluetooth, peer-to-peer Wi-Fi, or cellular, if available.

[0377] In block 2030, process 2000 may include receiving an offset value from a second mobile device via a wireless communication channel, corresponding to the distance between the second mobile device and a first reference point among the reference points, where the offset value is measured by the second mobile device. For example, a device can receive an offset value from a second mobile device via a wireless communication channel, corresponding to the distance between the second mobile device and a first reference point among the reference points, where the offset value is measured by the second mobile device as described above.

[0378] In block 2035, process 2000 may include identifying a stored reference point of a reference point grid corresponding to a first reference point based on a first location of the first mobile device, the relative position between the first mobile device and the second mobile device, and an offset value. For example, the device may identify a stored reference point of a reference point grid corresponding to a first reference point based on a first location of the first mobile device, the relative position between the first mobile device and the second mobile device, and an offset value, as described above.

[0379] To calculate the offset value between two location reference frames, a mobile device may use information about the coordinates or positions of at least three common points in both reference frames. These common points are known as control points or tie points. This process typically involves the following steps:

[0380] Firstly, this process can identify at least three common points that have known coordinates in both reference frames. These points should be easily identifiable and distinguishable in both reference frames.

[0381] Secondly, this process can obtain the coordinates of common points within each reference frame. The coordinates can be in any suitable coordinate system, such as latitude and longitude, Cartesian coordinates, or any other geodetic system.

[0382] Thirdly, the process can transform the coordinates of common points from one reference frame to another using an appropriate transformation method or algorithm. This transformation takes into account the differences in orientation, scale, and translation between the two frames.

[0383] After transforming the coordinates of common points, calculate the difference or offset between corresponding points in the two reference frames. This can be done by subtracting the transformed coordinates of the points in one frame from the coordinates of the other frame.

[0384] This process allows us to calculate the average offset value by taking the average of the offsets calculated for all common points. This provides a single value that represents the overall offset between the two reference frames.

[0385] The specific transformation method or algorithm used to calculate the offset may vary depending on the characteristics of the reference frames, such as their spatial relationships, coordinate systems, and any known geometric transformations. In some cases, more advanced techniques, such as least-squares adjustment or geodetic data transformation models, may be required to obtain accurate results.

[0386] In block 2040, process 2000 may include determining the second location of the second mobile device based on the stored reference point and offset value. For example, the device may determine the second location of the second mobile device based on the stored reference point and offset value as described above.

[0387] Process 2000 may include additional implementations, such as any single implementation or any combination of implementations, relating to one or more other processes described below and / or elsewhere in this specification. In the first implementation, the reference point is separated by at least a first threshold distance, the first threshold distance being less than an offset value.

[0388] In various embodiments, the grid coordinates are less than 5 bytes.

[0389] Figure 20 shows an exemplary block of process 2000, but note that in some implementations, process 2000 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those shown in Figure 20. Additionally or alternatively, two or more blocks of process 2000 may be executed in parallel. VII. Other Techniques A. Signaling method to indicate when the coordinate system has been reset.

[0390] A reset may be required when a person with a mobile device covers the camera or stops walking. The displacement measurement structure can be defined to have an input timestamp indicating that after this time, all data received by the mobile device can be used together. The applicability timestamp is the actual time that the data in question comes in when its horizontal or vertical displacement is applicable and can be used. Based on the reset or moved input timestamp, the device knows that some of the older data is no longer available. If a packet is lost between two packets, the mobile device can still see the two received packets and determine how much the velocity has changed between them, unless a reset has occurred. B. Change the arrows based on the age of the data.

[0391] Over time, GNSS-based positioning arrow data can become outdated. Instead of no longer calculating GNSS-based arrows when this happens, mobile devices are smart enough to still be able to use older location data they received seconds or even tens of seconds earlier.

[0392] In various embodiments, a mobile device can slowly inflate, illustrate, and expand an uncertainty bubble around the location of the person being explored. The uncertainty bubble can be a function of two things: one is the amount of time since the location of the person being explored was last updated, and the other is the amount of time corresponding to our knowledge of the person's movements.

[0393] An uncertainty bubble, also known as an error ellipse or precision circle, is a graphical representation of positional uncertainty associated with a GNSS-based arrow or any location determined using a Global Navigation Satellite System (GNSS). It can provide an indication of potential error or uncertainty in the arrow's position. An uncertainty bubble can function as follows:

[0394] Firstly, a GNSS receiver collects signals from multiple satellites to calculate the receiver's position. These signals are affected by various sources of error, including atmospheric conditions, satellite geometry, clock inaccuracies, and multipath interference.

[0395] Secondly, the GNSS receiver processes the received signal and calculates its position using techniques such as trilateration or polylateration. The receiver estimates its position based on the distance to visible satellites and the known positions of those satellites.

[0396] Thirdly, during the position calculation process, the GNSS receiver also estimates uncertainties or errors associated with its position. These errors may include horizontal and vertical errors, along with other metrics such as the Degradation of Precision (DOP) value.

[0397] Fourth, uncertainty or error estimates are typically represented as uncertainty bubbles or ellipses around the calculated position. The size and shape of the bubble indicate the uncertainty of the expected position. For example, a larger bubble represents a higher degree of uncertainty, while a smaller bubble indicates higher confidence in the calculated position.

[0398] Uncertainty bubbles are often associated with confidence levels, such as a 95% confidence level. This means that the true position is expected to be within the uncertainty bubble with a 95% probability. Confidence levels can be adjusted based on specific requirements or criteria.

[0399] The accuracy and size of uncertainty bubbles can vary based on several factors, including GNSS signal quality, satellite geometry, receiver quality, environmental conditions, and the presence of obstacles. Furthermore, post-processing techniques or differential GNSS methods can be used to further improve accuracy and reduce the size of uncertainty bubbles.

[0400] When the uncertainty becomes sufficiently large, GNSS arrows stop being generated. VIII.Distance measurement

[0401] In some embodiments, a mobile device may include circuitry for performing distance measurement. Such circuitry may include one or more dedicated antennas (e.g., three) and circuitry for processing the measured signals. Distance measurement can be performed using the time of flight of pulses between two mobile devices. In some implementations, round-trip time (RTT) is used to determine distance information for each of the antennas, for example. In other implementations, single-trip time in one direction may be used. The pulses may be formed using ultra-wideband (UWB) radio technology. A. Sequence diagram

[0402] Figure 21 shows a sequence diagram 2100 for performing distance measurement between two mobile devices according to an embodiment of the present disclosure. The two mobile devices may belong to two different users. The two users may know each other and therefore may have each other's phone numbers or other identifiers. As will be described in more detail later, such identifiers may be used for authentication purposes, for example, so that the ranging is not performed using an unknown device. Although Figure 21 shows a single measurement, the process can be repeated to perform multiple measurements over time intervals as part of a ranging session, where such measurements may be averaged or otherwise analyzed to provide a single distance value for each antenna, for example.

[0403] A first mobile device 2110 (e.g., a smartphone) can initiate distance measurement (operation) by sending a distance measurement request 2101 to a second mobile device 2120. The distance measurement request 2101 may include a first set of one or more pulses. Distance measurement can be performed using a distance measurement radio protocol (e.g., UWB). Distance measurement can be triggered in various ways, for example, based on user input and / or authentication using another radio protocol, such as Bluetooth Low Energy (BLE).

[0404] At T1, the first mobile device 2110 transmits a ranging request 2101. At T2, the second mobile device 2120 receives the ranging request 2101. T2 can be the average reception time when multiple pulses are in the first set. The second mobile device 2120 may anticipate the ranging request 2101 within a time window based on previous communications, for example, using another radio protocol. The ranging radio protocol and the other radio protocol can be synchronized so that the mobile device 2120 can turn on one or more ranging antennas and associated circuits over a specified time window, as opposed to keeping them on throughout the entire ranging session.

[0405] Upon receiving the distance measurement request 2101, the mobile device 2120 may transmit a distance measurement response 2102. As shown in the figure, the distance measurement response 2102 is transmitted at time T3, for example, the transmission time of a pulse or the average transmission time of a set of pulses. T2 and T3 may be sets of times for each pulse. The distance measurement response 2102 may include times T2 and T3 so that the mobile device 2110 can calculate the distance information. Alternatively, a delta between the two times (e.g., T3-T2) may be transmitted. The distance measurement response 2102 may also include an identifier for the first mobile device 2110, an identifier for the second mobile device 2120, or both.

[0406] At T4, the first mobile device 2110 can receive the distance measurement response 2102. As with other times, T4 can be a single time value or a set of time values.

[0407] In 2103, the first mobile device 2110 calculates distance information 2130, which can have various units, such as distance units (e.g., meters) or time (e.g., milliseconds). Time may be equivalent to distance with a proportionality constant corresponding to the speed of light. In some embodiments, distance may be calculated from a total round-trip time which may be equal to T2-T1+T4-T3. In some embodiments, processing time for the second mobile device 2120 may also be subtracted from the total round-trip time. More complex calculations may also be used, for example, when time corresponds to a set of time relative to a set of pulses, and when frequency correction is performed. B. Triangulation

[0408] In some embodiments, a mobile device may have multiple antennas, for example, to perform triangulation. Distinct measurements from different antennas can be used to determine a two-dimensional (2D) position, as opposed to a single distance value that could occur anywhere on a circle / sphere around the mobile device. The two-dimensional (2D) position can be specified in various coordinate systems, such as Cartesian or polar coordinates, where polar coordinates may include angular and radial values.

[0409] Figure 22 shows a sequence diagram 2200 of a ranging operation including a mobile device 2210 having three antennas 2211, 2212, and 2213 according to an embodiment of the present disclosure. The antennas 2211, 2212, and 2213 may be arranged to have different orientations, for example, to define a field of view for performing distance measurement.

[0410] In this example in Figure 22, each of the antennas 2211, 2212, and 2213 transmits packets (containing one or more pulses) that are received by the mobile device 2220. These packets may be part of a ranging request 2201. Each packet may be transmitted at time T1, but in other implementations, they may be transmitted at different times.

[0411] In some embodiments, the mobile device 2220 may have multiple antennas itself. In such an implementation, the antenna of the mobile device 2210 can send packets to a specific antenna of the mobile device 2220 (as opposed to broadcasting), and the mobile device 2220 can respond to that specific packet. The mobile device 2220 listens on a designated antenna, so both devices know which antenna is involved, or the packet can indicate which antenna the message is directed to. For example, the first antenna can respond to a received packet, and upon receiving the response, it can send another packet to a different antenna. However, this alternative procedure may require more time and power.

[0412] The three packets of the ranging request 2201 are received at times T2, T3, and T4, respectively. Therefore, the antenna of mobile device 2220 (e.g., UWB antenna) can listen substantially simultaneously and respond independently. Mobile device 2220 provides ranging responses 2202, which are transmitted at times T5, T6, and T7, respectively. Mobile device 2210 receives at times T8, T9, and T 10 At each point, the distance measurement response is received.

[0413] In 2203, the processor 2214 of the mobile device 2210 calculates distance information 2230, for example, as described herein. The processor 2214 can receive time from the antennas, more specifically from a circuit (e.g., a UWB circuit) that analyzes signals from antennas 2211, 2212, and 2213. As will be described later, the processor 2214 may be an always-on processor that uses less power than an application processor that can perform more general functions. The distance information 2230 can be used to determine the two-dimensional (2D) or three-dimensional (3D) position of the mobile device 2220, and such a position can be used to configure the display screen of the mobile device 2210. For example, the position can be used to determine where the corresponding icon should be displayed on the mobile device 2220, for example, which position in a list, which position in a 2D grid, or which cluster in a 1D, 2D, or 3D distance / position range the icon should be displayed in.

[0414] In some embodiments, to determine which ranging response originates from which antenna, mobile device 2220 may inform mobile device 2210 of the order in which the response messages are transmitted, for example, during a ranging setup handshake that may occur using a different radio protocol. In other embodiments, ranging responses may include identifiers that indicate which antenna transmitted the message. These identifiers may be determined during the ranging setup handshake.

[0415] Messages in ranging request 2201 and ranging response 2202 can contain very little data in the payload, for example, by including a small number of pulses. Using fewer pulses may be advantageous. The environment of a mobile device (potentially in a pocket) can make measurements difficult. As another example, the antenna of one device may be pointed in a different direction from the direction in which the other device is approaching. Therefore, it is desirable to use high power for each pulse, but there are government regulations (as well as battery concerns) regarding how much power can be used within a particular time window (e.g., on average over 1 millisecond). Packet frames in these messages can be approximately 150-190 microseconds long. C.UWB

[0416] The radio protocol used for ranging may have a narrower pulse (e.g., a narrower full width at half maximum (FWHM)) than the first radio protocol (e.g., Bluetooth®) used for initial authentication or communication of ranging settings. In some embodiments, the ranging radio protocol (e.g., UWB) can provide a distance accuracy of 5 cm or better. In various embodiments, the frequency range can be 3.1 to 10.6 gigahertz (GHz). Multiple channels can be used, for example, one channel at 6.5 GHz and another at 8 GHz. Thus, in some cases, the ranging radio protocol may not overlap with the frequency range of the first radio protocol (e.g., 2.4 to 2.485 GHz).

[0417] The radio ranging protocol can be specified by IEEE 802.15.4, a type of UWB. Each pulse in a pulse-based UWB system can occupy the entire UWB bandwidth (e.g., 500 MHz), thereby allowing the pulse to be localized temporally (i.e., to a narrow time width, e.g., 0.5 ns to several nanoseconds). In terms of distance, the pulse can be less than 60 cm wide for a 500 MHz wide pulse and less than 23 cm wide for a 1.3 GHz wide pulse. Because the bandwidth is so wide and the width in real space is so narrow, very accurate time-of-flight measurements can be obtained.

[0418] Each of the ranging messages (also called a frame or packet) may contain a sequence of pulses that can represent modulated information. Each data symbol within a frame may be a sequence. A packet may have a preamble containing header information, for example, from the physical layer and media access control (MAC) layer, and may contain a destination address. In some embodiments, a packet frame may contain a synchronization portion and a start frame delimiter, which can line up the timings.

[0419] Packets may include methods of security configuration and may include encrypted information, such as an identifier indicating which antenna is transmitting the packet. Encrypted information can be used for further authentication. However, in the case of ranging operations, the content of the data may not need to be determined. In some embodiments, timestamps for pulses of specific data fragments can be used to track the difference between transmission and reception. The content (e.g., decrypted content) can be used to match pulses so that the correct time difference can be calculated. In some implementations, the encrypted information may include an indicator that authenticates which stage the message corresponds to; for example, ranging request 2201 may correspond to stage 1, and ranging response 2202 may correspond to stage 2. Such use of indicators may be useful when two or more devices are performing ranging operations in close proximity to each other.

[0420] Using narrow pulses (e.g., ~1 nanosecond width), distance can be accurately determined. High bandwidth (e.g., a 500 MHz spectrum) enables narrow pulses and accurate location determination. Pulse cross-correlation can provide timing accuracy, which is a small fragment of pulse width, providing accuracy within hundreds or tens of picoseconds, for example, and providing sub-meter level distance measurement accuracy. The pulses can represent +1 and -1 distance measurement waveforms in several patterns recognized by the receiver. For distance measurement, round-trip time measurement, also called time-of-flight measurement, can be used. As mentioned above, mobile devices can transmit a set of timestamps, which can eliminate the need for clock synchronization between the two devices. IX.UWB device

[0421] Figure 23 is a block diagram of the components of a mobile device 2300 operable to perform ranging according to an embodiment of the present disclosure. The mobile device 2300 includes antennas for at least two different radio protocols, as described above. A first radio protocol (e.g., Bluetooth) may be used for authentication and exchange of ranging settings. A second radio protocol (e.g., UWB) may be used to perform ranging with another mobile device.

[0422] As shown, the mobile device 2300 includes a UWB antenna 2310 for performing ranging. The UWB antenna 2310 is connected to a UWB circuit 2315 for analyzing the signal detected from the UWB antenna 2310. In some embodiments, the mobile device 2300 includes three or more UWB antennas, for example, to perform triangulation. Different UWB antennas may have different orientations, for example, two in one direction and a third in another. The orientation of the UWB antennas can define a field of view for ranging. As an example, the field of view may extend to 120 degrees. Such a restriction can allow the user to determine the direction in which the device is pointed relative to one or more other nearby devices. The field of view may include any one or more of the pitch angle, yaw angle, or roll angle.

[0423] The UWB circuit 2315 can communicate with an always-on-processor (AOP) 2330, which can perform further processing using information from the UWB messages. For example, the AOP 2330 can perform distance calculations using timing data provided by the UWB circuit 2315. The AOP 2330 and other circuits of the device may include dedicated and / or configurable circuits, for example, via firmware or other software.

[0424] As shown, the mobile device 2300 also includes a Bluetooth (BT) / Wi-Fi antenna 2320 for communicating data with other devices. The Bluetooth (BT) / Wi-Fi antenna 2320 is connected to a BT / Wi-Fi circuit 2325 for analyzing detection signals from the BT / Wi-Fi antenna 2320. For example, the BT / Wi-Fi circuit 2325 can parse messages to obtain data (e.g., authentication tags), which can then be transmitted to the AOP 2330. In some embodiments, the AOP 2330 can perform authentication using the authentication tags. Thus, as part of the authentication process, the AOP 2330 can store or retrieve a list of authentication tags for comparison with received tags. In some implementations, such functionality can be achieved by the BT / Wi-Fi circuit 2325.

[0425] In other embodiments, the UWB circuit 2315 and the BT / Wi-Fi circuit 2325 can be connected to an application processor 2340 that can perform functions similar to those of the AOP 2330, either alternatively or additionally. The application processor 2340 typically requires more power than the AOP 2330, and therefore power can be saved by having the AOP 2330 handle certain functions so that the application processor 2340 can remain in a sleep state, e.g., an off state. As an example, the application processor 2340 can be used to communicate audio or video using BT / Wi-Fi, and the AOP 2330 can coordinate the transmission of such content with communication between the UWB circuit 2315 and the BT / Wi-Fi circuit 2325. For example, the AOP 2330 can coordinate the timing of UWB messages for BT advertisements.

[0426] Coordination using AOP2330 can offer various advantages. For example, the first user of a sending device may want to share content with another user, and therefore, distance measurement to this other user's receiving device may be desirable. However, if many people are in the same room, the sending device may need to distinguish a specific device from among multiple devices in the room and potentially determine which device the sending device is referring to. Such functionality can be provided by AOP2330. Furthermore, it is undesirable to wake up the application processors of all other devices in the room, and therefore, the AOPs of other devices can perform some processing on the message and determine that it is for a different device with a different destination address.

[0427] To perform ranging, the BT / Wi-Fi circuit 2325 can analyze an advertisement signal from another device to determine if the other device wishes to perform ranging as part of a process to share content, for example. The BT / Wi-Fi circuit 2325 can communicate this notification to the AOP 2330, which can then schedule the UWB circuit 2315 to be ready to detect UWB messages from the other device.

[0428] For a device that initiates distance measurement, its AOP can perform the distance calculation. Furthermore, the AOP can monitor changes in distance between other devices. For example, AOP2330 can compare distance to a threshold and provide a warning when the distance exceeds the threshold, or potentially provide a reminder when two devices are close enough. An example of the former might be when a parent wants to be warned when a child (and possibly the child's device) is too far away. An example of the latter might be when a person wants to be prompted to present something when speaking to the user of another device. Such monitoring by the AOP can reduce power consumption by the application processor. X. Exemplary device

[0429] Figure 24 is a block diagram of an exemplary device 2400, which may be a mobile device. Device 2400 generally includes a computer-readable medium 2402, a processing system 2404, an input / output (I / O) subsystem 2406, a wireless circuit 2408, and an audio circuit 2410 including a speaker 2450 and a microphone 2452. These components can be connected by one or more communication buses or signal lines 2403. Device 2400 can be any portable mobile device, including handheld computers, tablet computers, mobile phones, laptop computers, tablet devices, media players, personal digital assistants (PDAs), key fobs, car keys, access cards, multifunction devices, mobile phones, portable gaming devices, car display units, etc. (including combinations of two or more of these items).

[0430] The architecture shown in Figure 24 is merely one example of an architecture for device 2400, and it is clear that device 2400 may have more or fewer components, or different configurations than those shown. The various components shown in Figure 24 may be implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0431] The wireless circuit 2408 is used to transmit and receive information to and from the conventional circuits of one or more other devices, such as an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, or memory, via a wireless link or network. The wireless circuit 2408 can use various protocols, such as those described herein.

[0432] The wireless circuit 2408 is coupled to the processing system 2404 via a peripheral device interface 2416. The interface 2416 may include conventional components to establish and maintain communication between the peripheral device and the processing system 2404. The wireless circuit 2408 transmits the received voice and data information (e.g., in a speech recognition application or a voice command application) to one or more processors 2418 via the peripheral device interface 2416. The one or more processors 2418 can be configured to process various data formats for one or more application programs 2434 stored on the medium 2402.

[0433] The peripheral interface 2416 connects the device's input and output peripherals to the processor 2418 and the computer-readable medium 2402. One or more processors 2418 communicate with the computer-readable medium 2402 via the controller 2420. The computer-readable medium 2402 can be any device or medium capable of storing code and / or data for use by one or more processors 2418. The medium 2402 may include a memory hierarchy including a cache, main memory, and auxiliary memory.

[0434] Device 2400 also includes a power system 2442 that supplies power to various hardware components. The power system 2442 may include a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and other components typically associated with the generation, management, and distribution of power in the mobile device.

[0435] In some embodiments, device 2400 includes a camera 2444. In some embodiments, device 2400 includes a sensor 2446. The sensor 2446 may include an accelerometer, compass, gyrometer, pressure sensor, audio sensor, light sensor, barometer, etc. The sensor 2446 can be used to sense aspects of a location, such as an auditory signature or a light signature of the location.

[0436] In some embodiments, device 2400 may include a GPS receiver, sometimes referred to as a GPS unit 2448. The mobile device can obtain position information, timing information, altitude, or other navigation information using a satellite navigation system such as the Global Positioning System (GPS). During operation, the GPS unit can receive signals from GPS satellites orbiting the Earth. The GPS unit analyzes the signals to estimate travel time and distance. The GPS unit can determine the mobile device's current position (current location). Based on these estimates, the mobile device can determine its location lock, altitude, and / or current speed. Location lock can be geographic coordinates such as latitude and longitude information. In other embodiments, device 2400 may be configured to identify GLONASS signals or any other similar type of satellite navigation signal.

[0437] One or more processors 2418 execute various software components stored in the medium 2402 to perform various functions for the device 2400. In some embodiments, the software components include an operating system 2422, a communication module (or set of instructions) 2424, a location module (or set of instructions) 2426, a trigger event module 2428, a predictive app manager module 2430, and other applications (or sets of instructions) 2434 such as vehicle positioning and navigation applications.

[0438] Operating System 2422 can be any suitable operating system, including embedded operating systems such as iOS®, Mac OS, Darwin, RTXC, LINUX, UNIX®, WINDOWS®, OS X, WINDOWS®, or VxWorks. The operating system may include a set of procedures, instructions, software components, and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.

[0439] The communication module 2424 facilitates communication with other devices via one or more external ports 2436 or via the wireless circuit 2408, and includes various software components for handling data received from the wireless circuit 2408 and / or the external ports 2436. The external ports 2436 (e.g., USB, FireWire, Lightning connector, 60-pin connector, etc.) are adapted to connect directly to other devices or indirectly via a network (e.g., the Internet, Wi-Fi, etc.).

[0440] The location / motion module 2426 can help determine the current position (e.g., coordinates or other geographic location identifiers) and movement of device 2400. Modern positioning systems include satellite-based positioning systems such as the Global Positioning System (GPS), cellular network positioning based on "cell IDs," and Wi-Fi positioning technology based on Wi-Fi networks. GPS can also determine position estimates based on the visibility of multiple satellites, but satellites may not be visible (or have weak signals) indoors or in "canyons of buildings." In some embodiments, the location / motion module 2426 receives data from the GPS unit 2448 and analyzes the signal to determine the current position of the mobile device. In some embodiments, the location / motion module 2426 can determine the current location using Wi-Fi or cellular location technology. For example, the location of the mobile device can be estimated using knowledge of nearby cell sites and / or Wi-Fi access points, and knowledge of their locations. Information identifying the Wi-Fi or cellular transmitter is received by the radio circuit 2408 and passed to the location / motion module 2426. In some embodiments, the location module receives one or more transmitter IDs. In some embodiments, a set of transmitter IDs can be compared with a reference database (e.g., a cell ID database, a Wi-Fi reference database), which maps or correlates the transmitter IDs to the position coordinates of the corresponding transmitters and calculates estimated position coordinates for device 2400 based on the position coordinates of the corresponding transmitters. Regardless of the specific location technology used, the location / motion module 2426 receives information from which a location fix can be derived, interprets that information, and returns location information such as geographic coordinates, latitude / longitude, or other location fix data.

[0441] The trigger event module 2428 may include various submodules or systems, for example, as described herein with respect to Figure 2A. Furthermore, the predictive app manager module 2430 may include various submodules or systems, for example, as described herein with respect to Figure 3.

[0442] One or more application programs 2434 on a mobile device may include, without limitation, any applications installed on device 2400, including browsers, address books, contact lists, email, instant messaging, word processing, keyboard emulation, widgets, Java®-enabled applications, encryption, digital rights management, speech recognition, voice duplication, and music players (for playing recorded music stored in one or more files such as MP3 or AAC files).

[0443] Other modules or instruction sets (not shown), such as a graphics module and a time module, may be present. For example, a graphics module may include various conventional software components for rendering, animating, and displaying graphic objects (including, but not limited to, text, web pages, icons, digital images, animations, etc.) on a display surface. In another example, a timer module may be a software timer. A timer module may also be implemented in hardware. A time module may maintain various timers for any number of events.

[0444] The I / O subsystem 2406 can be coupled to a display system (not shown). The display system can be a touch-sensitive display. The display system displays visual output to the user on a GUI. This visual output may include text, graphics, video, and any combination thereof. Some or all of the visual output may correspond to user interface objects. The display may use LED (light-emitting diode), LCD (liquid crystal display) technology, or LPD (polymer light-emitting display) technology, but other display technologies may be used in other embodiments.

[0445] In some embodiments, the I / O subsystem 2406 may include a display, as well as user input devices such as a keyboard, mouse, and / or trackpad. In some embodiments, the I / O subsystem 2406 may include a touch-sensitive display, which may also accept user input based on tactile and / or haptic touch. In some embodiments, the touch-sensitive display forms a touch-sensitive surface that accepts user input. The touch-sensitive display / touch-sensitive surface (together with any associated modules and / or instruction sets in medium 2402) detects contact (and any movement or release of contact) on the touch-sensitive display and translates the detected contact into interaction with user interface objects (e.g., one or more soft keys) that appear on the touchscreen when contact occurs. In some embodiments, the contact points between the touch-sensitive display and the user correspond to one or more of the user's fingers. The user may touch the touch-sensitive display using any preferred object or attachment such as a stylus, pen, or finger. The touch-sensitive display surface can detect contact and any movement or release using any suitable touch sensitivity technology, which includes capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other elements for determining one or more contact points with other proximity sensor arrays or touch-sensitive displays.

[0446] Furthermore, the I / O subsystem can be coupled to one or more other physical control devices (not shown), such as push buttons, keys, switches, locker buttons, dials, slider switches, sticks, and LEDs, to control or perform various functions such as power control, speaker volume control, ringtone volume, keyboard input, scrolling, hold, menu, screen lock, and clearing and ending communications. In some embodiments, in addition to the touchscreen, the device 2400 may include a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display a visual output. The touchpad may be a touch-sensitive surface separate from the touch-sensitive display, or an extension of the touch-sensitive surface formed by the touch-sensitive display.

[0447] In some embodiments, some or all of the operations described herein can be performed using applications running on a user's device. Circuits, logic modules, processors, and / or other components may be configured to perform the various operations described herein. Those skilled in the art will understand that such configurations can be achieved, depending on the implementation, through the design, setup, interconnection, and / or programming of specific components, and that, similarly, depending on the implementation, the configured components may or may not be reconfigurable for different operations. For example, a programmable processor can be configured by providing suitable executable code, and a dedicated logic circuit can be configured by suitably connecting logic gates and other circuit elements.

[0448] Any of the software components or functions described in this application may be implemented as software code to be executed by a processor, for example, using conventional or object-oriented techniques, in any suitable computer language such as Java®, C, C++, C#, Objective-C, Swift, or scripting languages ​​such as Perl or Python. The software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-temporary computer-readable media include random access memory (RAM), read-only memory (ROM), magnetic media such as hard drives or floppy disks, optical media such as compact disks (CDs) or DVDs (digital versatile disks), flash memory, and the like. The computer-readable medium may be any combination of such storage devices or transmission devices.

[0449] Computer programs incorporating various features of this disclosure may be encoded on various computer-readable storage media, including optical storage media such as magnetic disks or tapes, compact discs (CDs) or digital versatile discs (DVDs), and flash memory. The computer-readable storage media encoded with the program code may be packaged with a compatible device or provided separately from other devices. In addition, the program code may be encoded and transmitted over wired optical networks and / or wireless networks compliant with various protocols, including the Internet, thus enabling distribution, for example, via Internet download. Any such computer-readable media may reside on or within a single computer product (e.g., a solid-state drive, hard drive, CD, or an entire computer system), or on or within different computer products within a system or network. The computer system may include a monitor, printer, or other suitable display that provides the user with any of the results described herein.

[0450] As described above, one aspect of the technology involves collecting and using data available from various sources to improve the expectation that users may be interested in communicating with it. This disclosure suggests that in some cases, such collected data may include personal information data that uniquely identifies a particular person, or personal information data that can be used to contact a particular person or locate them. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, Twitter® IDs, home addresses, data or records relating to a user's health or fitness level (e.g., vital signs measurements, medication information, exercise information), birth dates, or any other identifying or personal information.

[0451] This disclosure acknowledges that the use of such personal data in the technology may be for the benefit of the user. For example, personal data may be used to predict which users may want to communicate with at a given time and place. Thus, the use of such personal data in contextual information enables predictions about people who may want to interact with a given user at a given time and place. Furthermore, other uses of personal data that benefit the user are also intended by this disclosure. For example, health and fitness data may be used to provide insights into a user's overall wellness or as positive feedback to individuals using the technology to pursue wellness goals.

[0452] This disclosure is intended to ensure that entities involved in the collection, analysis, disclosure, transmission, storage, or other use of such personal data comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for keeping personal data confidential and secure. Such policies must be readily accessible to users and updated as data collection and / or use changes. Personal data from users should be collected for the lawful and legitimate use of the entity and should not be shared or sold for any other purpose. Furthermore, such collection / sharing should only occur after informing and obtaining the user's consent. In addition, such entities should consider taking all necessary steps to protect and secure access to such personal data and to ensure that others with access to personal data faithfully adhere to those privacy policies and procedures. Furthermore, such entities may undergo third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be adapted to the specific types of personal data collected and / or accessed, and should comply with applicable laws and standards, including jurisdiction-specific considerations. For example, in the United States, the collection or access to certain health data may be subject to federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA). Health data in other countries, on the other hand, may be subject to other regulations and policies and should be addressed accordingly. Therefore, different privacy practices should be maintained in each country with respect to different types of personal data.

[0453] Notwithstanding the foregoing, the Disclosure also intends embodiments that allow a user to selectively prevent the use of or access to personal data. That is, the Disclosure intends that hardware and / or software elements may be provided to prevent or prevent access to such personal data. For example, in the case of a people-centered predictive service, the technology could be configured to allow a user to choose to “opt in” or “opt out” of participating in the collection of personal data during or at any time thereafter when registering for the service. In another example, a user may choose not to provide location information to a recipient suggestion service. In yet another embodiment, a user may choose not to provide precise location information but to allow the transfer of location zone information. In addition to providing “opt-in” and “opt-out” options, the Disclosure intends to provide notices regarding access to or use of personal data. For example, a user may be notified when downloading an app that will access their personal data, and then reminded again immediately before the app accesses the personal data.

[0454] Furthermore, the intent of this disclosure is that personal data should be managed and handled in a manner that minimizes the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data when it is no longer needed. In addition, where applicable in certain health-related applications, data anonymization can be used to protect user privacy. Anonymization can be facilitated, where appropriate, by removing certain identifiers (e.g., birth dates), controlling the amount or specificity of data stored (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data across all users), and / or by other means.

[0455] Therefore, while this disclosure broadly covers the use of personal data to implement one or more different disclosed embodiments, it is also intended that these embodiments can be implemented without requiring access to such personal data. In other words, the various embodiments of the technology will not be rendered inoperable by the absence of all or part of such personal data. For example, it may be predicted for multiple users that a user wishes to communicate at a certain time and place based on a minimum amount of personal information, such as non-personal data or content requested by a device associated with the user, other available non-personal information, or publicly available information.

[0456] While this disclosure has been described in relation to specific embodiments, it should be understood that this disclosure is intended to extend to all modifications and equivalents within the scope of the following claims.

[0457] All patents, patent applications, publications, and descriptions referenced herein are incorporated in their entirety by reference for all purposes. Nothing is considered prior art. In the event of any conflict between this application and the references provided herein, this application shall prevail.

Claims

1. A method executed by the processor of a first mobile device, For each of the multiple ranging sessions that occur during a certain period, The first time period involves transmitting a radio ranging signal, Receiving a wireless response signal from a second mobile device at the second time, Based on the difference between the first time and the second time, the range value between the first mobile device and the second mobile device is determined, and a set of range values ​​is determined accordingly. From first measurements captured during the period using a first sensor on the first mobile device, first odometry information is determined, wherein the first odometry information indicates a first movement of the first mobile device during the period. Receiving second odometry information via a data channel between the first mobile device and the second mobile device, which is determined from second sensor measurements captured during the period using a second sensor on the second mobile device, wherein the second odometry information indicates a second movement of the second mobile device during the period. Using the set of range values, the first odometry information, and the second odometry information, the angle between the first reference frame for the first mobile device and the second reference frame for the second mobile device is solved. A method comprising displaying directional information on the display of the first mobile device indicating the direction from the first current position of the first mobile device to the second current position of the second mobile device.

2. The method according to claim 1, wherein the second sensor is an optical sensor and the measurement value of the second sensor includes odometry information.

3. The method according to claim 1 or 2, wherein the second sensor is an accelerometer, and the measurement value of the second sensor comprises acceleration information for the second mobile device.

4. The method according to any one of claims 1 to 3, wherein solving for the angle between the first reference frame of the first mobile device and the second reference frame of the second mobile device is done using a least-squares equation, on at least the range value, the first odometry information, and the second sensor measurement value.

5. The method according to any one of claims 1 to 4, wherein the first mobile device and the second mobile device are operating.

6. The method according to any one of claims 1 to 5, wherein solving the angle between the first reference frame of the first mobile device and the second reference frame of the second mobile device includes calculating the vertical displacement, horizontal displacement, and direction of travel offset between the first mobile device and the second mobile device.

7. The method according to any one of claims 1 to 6, wherein the first odometry information includes visual inertial odometry information.

8. The method according to any one of claims 1 to 7, wherein the data channel includes a narrowband channel controlled by an ultra-wideband processing chip.

9. Determining whether the relative position between the first and second mobile devices changes, and suppressing the display of the directional information from the first mobile device to the second mobile device based on the angle, The method according to any one of claims 1 to 8, further comprising:

10. The method according to any one of claims 1 to 9, wherein the directional information is an arrow.

11. Determining the first uncertainty in the second current position of the second mobile device, The first location of the first mobile device is determined based on the first GNSS signal acquired by the first mobile device, The second location of the second mobile device is received via the data channel between the first mobile device and the second mobile device based on the second GNSS signal, Determining a second range value between the first location and the second location, Determining the second uncertainty in the second range value, Using the second current position, the first uncertainty, the second range value, and the second uncertainty, the position vector between the first mobile device and the second mobile device is determined. Displaying the directional information based on the position vector, The method according to any one of claims 1 to 10, further comprising:

12. To store the grid of reference points for the global reference frame, Using measurements performed by the first mobile device, the first location of the first mobile device within the global reference frame is determined; To detect a wireless signal transmitted from a second mobile device, The relative position between the first mobile device and the second mobile device is determined based on the radio signal transmitted by the second mobile device during at least one of the plurality of distance measurement sessions. Receiving an offset value from the second mobile device via the data channel, which corresponds to the distance between the second mobile device and the first reference point among the reference points, wherein the offset value is measured by the second mobile device. Based on the first location of the first mobile device, the relative position between the first mobile device and the second mobile device, and the offset value, the stored reference point of the grid corresponding to the first reference point is identified. The second location of the second mobile device is determined based on the stored reference point and the offset value, The method according to any one of claims 1 to 11, further comprising:

13. It is a mobile device, One or more processors, A mobile device comprising: a memory coupled to one or more processors, wherein the memory stores instructions, and the instructions cause the one or more processors to perform one or more of the operations described in any one of claims 1 to 12.

14. A non-temporary computer-readable medium that stores instructions for performing the operations described in any one of claims 1 to 12 when executed on one or more processors.

15. A method executed by the processor of a first mobile device, The first time period involves transmitting a radio ranging signal, Receiving a wireless response signal from a second mobile device at the second time, Based on the aforementioned wireless response signal, the relative position between the first mobile device and the second mobile device is determined. Based on the difference between the first time and the second time, a first range value is determined between the first mobile device and the second mobile device. Determining the first uncertainty in the first range value, The first location of the first mobile device is determined based on the first GNSS signal acquired by the first mobile device, The second location of the second mobile device is received based on a second GNSS signal via a data channel between the first mobile device and the second mobile device, Determining a second range value between the first location and the second location, Determining the second uncertainty in the second range value, Using the first range value, the first uncertainty, the second range value, and the second uncertainty, the position vector between the first mobile device and the second mobile device is determined. A method comprising displaying directional information indicating the direction from the first current position of the first mobile device to the second current position of the second mobile device.

16. The method according to claim 15, wherein the first mobile device and the second mobile device are operating.

17. The method according to claim 15 or 16, wherein the data channel includes a narrowband channel controlled by an ultra-wideband processing chip.

18. The method according to any one of claims 15 to 17, wherein the position vector is determined based on the lower of the first uncertainty or the second uncertainty.

19. The method according to any one of claims 15 to 18, wherein the position vector is determined based on the average of the first range value and the second range value.

20. The method according to claim 19, wherein a plurality of weights are applied to the first range value and the second range value.

21. The method according to claim 20, wherein the plurality of weights are based on the first uncertainty and the second uncertainty.

22. It is a mobile device, One or more processors, A mobile device comprising: a memory coupled to one or more processors, wherein the memory stores instructions, and the instructions cause the one or more processors to perform one or more of the operations described in any one of claims 15 to 21.

23. A non-temporary computer-readable medium that stores instructions that, when executed on one or more processors, perform one or more of the operations described in claims 15 to 21.

24. A method executed by the processor of a first mobile device, To store the grid of reference points for the global reference frame, Using measurements performed by the first mobile device, the first location of the first mobile device within the global reference frame is determined; To detect a wireless signal transmitted from a second mobile device, The relative position between the first mobile device and the second mobile device is determined based on the aforementioned wireless signal, Establishing a wireless communication channel with the second mobile device, Receiving an offset value from the second mobile device via the wireless communication channel, which corresponds to the distance between the second mobile device and the first reference point among the reference points, wherein the offset value is measured by the second mobile device. Based on the first location of the first mobile device, the relative position between the first mobile device and the second mobile device, and the offset value, the stored reference point of the grid corresponding to the first reference point is identified. A method comprising determining the second location of the second mobile device based on the stored reference point and the offset value.

25. The method according to claim 24, wherein the reference points are separated by at least a first threshold distance, and the first threshold distance is less than the offset value.

26. The method according to claim 24 or 25, wherein the file size of the offset value of the grid coordinates is less than 5 bytes.

27. Determining the second grid coordinates of the second location of the second mobile device, The method according to any one of claims 24 to 26, further comprising:

28. The direction from the first mobile device to the second mobile device is determined based at least on the first location of the first mobile device and the second location of the second mobile device in the global reference frame, Displaying a graphical user interface indicating the determined direction, The method according to claim 27, further comprising:

29. To determine whether the distance between the first mobile device and the second mobile device is less than a predetermined distance, The method according to any one of claims 24 to 28, further comprising:

30. Based on a plurality of defined reference points, the offset between the local coordinates of the second mobile device in the local coordinate system and the global reference frame is determined. The method according to any one of claims 24 to 29, further comprising:

31. Determining whether one or more of the reference points of the global reference frame change relative to the global reference frame, Receiving a new offset for a new reference point, The method according to any one of claims 24 to 30, further comprising:

32. It is a mobile device, One or more processors, A mobile device comprising: a memory coupled to one or more processors, wherein the memory stores instructions, and the instructions cause the one or more processors to perform one or more of the operations described in any one of claims 24 to 31.

33. A non-temporary computer-readable medium that stores instructions that, when executed on one or more processors, perform one or more of the operations described in claims 24 to 31.

34. A method executed by the processor of a first mobile device, Determining first inertial odometry information from first inertial measurements captured over a first period of time using an inertial sensor on the first mobile device, Identifying a first reference frame corresponding to the first inertia measurement, Determining first visual odometry information from first visual measurements captured over a first period of time using a visual sensor on the first mobile device, Identifying a second reference frame corresponding to the first visual measurement, Determining a first transformation between the second reference frame and the first reference frame, A method comprising determining the displacement of the first mobile device in the first reference frame during the first period using the first visual odometry information and the transformation.

35. Determining the displacement of the first mobile device is Mapping the first visual odometry information onto the first reference frame based on the first transformation, The error in the first inertial odometry information during the first period is determined by comparing the first inertial odometry information in the first reference frame with the first visual odometry information. The first inertial odometry information is transformed based on the error in the first inertial odometry information, and the transformed first inertial odometry information is obtained. The method according to claim 34, further comprising determining the displacement of the first mobile device in the first reference frame based on the converted first inertial odometry information or the first visual odometry information.

36. Determining second inertial odometry information from second inertial measurements captured over a second period occurring after the first period, and determining second inertial odometry information using the error of the first inertial odometry information, Determining the visual odometry information over the second period, The method according to claim 35, further comprising:

37. Based on a comparison of the second inertial odometry information and the second visual odometry information on the first reference frame, it is determined that the second visual odometry information is unreliable over the second period that occurs after the first period, In response to determining that the second visual odometry information is unreliable over the second period, the use of the second visual odometry information is suspended during the second period. In order to determine the displacement of the first mobile device in the first reference frame, the method involves switching to using the second inertial odometry information during the second period, The method according to claim 36, further comprising:

38. Determining third visual odometry information from third visual measurements taken over a third period occurring after the second period, Determining that the third visual odometry information is reliable, To determine the displacement of the first mobile device in the first reference frame, the third visual odometry information is used during the third period, The method according to claim 37, further comprising:

39. Determining the displacement of the first mobile device in the first reference frame during the third period is: Identifying a third reference frame corresponding to the third visual measurement described above, Determining the second transformation between the third reference frame and the first reference frame, The method according to claim 38, comprising determining the displacement of the first mobile device in the first reference frame during the third period using the first visual odometry information and the second transformation.

40. It is a mobile device, One or more processors, A mobile device comprising: a memory coupled to one or more processors, wherein the memory stores instructions, and the instructions cause the one or more processors to perform one or more of the operations described in any one of claims 34 to 39.

41. A non-temporary computer-readable medium that stores instructions that, when executed on one or more processors, perform one or more of the operations described in claims 34 to 39.

Images