A TinyML-based positioning device for mobile devices within a campus

Abstract

This invention provides a positioning device for mobile devices within a park based on TinyML, belonging to the field of integrated navigation technology. This invention combines Kalman filtering and TinyML concepts, applying them to BDS / SINS integrated navigation. When satellite signal strength is good, BDS and SINS positioning information are fused to obtain high-precision positioning information, and the Kalman filter is trained accordingly. When satellite signal strength is poor, pre-trained TinyML is used to fuse SINS and optical flow information to achieve complementary advantages. Experiments have verified the effectiveness and reliability of the algorithm, overcoming the shortcomings of BDS signal loss in harsh environments and the large fluctuations of Kalman filtering in nonlinear environments, and significantly improving the positioning accuracy of the integrated navigation system.

Description

Technical Field

[0001] This invention relates to the fields of integrated navigation technology, micro machine learning, and multi-source information fusion, and particularly to a positioning device for mobile devices within a campus based on TinyML. Background Technology

[0002] In recent years, users have increasingly demanded precise location data in systems and applications, such as driver assistance systems and autonomous vehicles. Beyond improving performance in various application scenarios, researchers are also seeking low-cost, high-reliability system solutions. Recent advances in sensor technology and sensor fusion algorithms have facilitated the realization of highly integrated, low-cost, high-precision navigation systems. Therefore, increasing research focuses on integrated navigation systems combining the BeiDou Navigation Satellite System (BDS) and Strapdown Inertial Navigation System (SINS), enabling such systems to provide accurate and continuous navigation information (e.g., position, velocity) even in harsh measurement environments. While complex environments present challenges for Global Navigation Satellite Systems (GNSS), GNSS remains a primary means of outdoor navigation. Indeed, the presence of buildings and trees causes signal reflection and attenuation, leading to measurement errors. In severe cases, the number of visible satellites is far from sufficient, and receivers cannot provide accurate position, velocity, and time (PVT) data. The current main solution is to improve system observability by adding more sensors. In most systems, BDS signals are fused with an inertial measurement unit (IMU) consisting of a three-axis accelerometer and a three-axis gyroscope to estimate accurate position and velocity. However, because BDS signals can be interfered with or unavailable in some situations, SINS / BDS is not a completely self-contained system. Optical flow is a device that can estimate the speed, direction, and displacement of a device by utilizing changes in an image. Using optical flow-assisted vehicle-mounted BDS / SINS integrated navigation can provide reliable velocity information when BDS signals are interrupted, thus suppressing the problem of rapid divergence of inertial navigation errors. Summary of the Invention

[0003] To address the aforementioned technical issues, this invention provides a positioning device for mobile devices within a campus based on TinyML. By introducing optical flow to form a multi-source constrained integrated navigation and positioning system, the performance of the integrated navigation system is improved. Simultaneously, TinyML technology is introduced to train the Kalman filter coefficients online, further enhancing the Kalman filter effect.

[0004] The technical solution of this invention is:

[0005] A TinyML-based positioning device for mobile devices within a campus includes an IMU consisting of an accelerometer, a gyroscope, and a magnetometer, an ARM processor, a BDS receiver, and an optical flow meter.

[0006] The gyroscope calculates the actual direction by the deviation from the initial direction; the accelerometer is used to detect the magnitude and direction of the acceleration experienced by the carrier; and the magnetometer measures the strength and direction of the magnetic field. The trajectory of the carrier is obtained by fusing data from the three sensors.

[0007] The optical flow meter uses optical flow-assisted BDS / SINS integrated navigation to provide velocity information when the BDS signal is interrupted, thus suppressing the problem of rapid divergence of inertial navigation errors.

[0008] The ARM processor performs algorithm deployment and inference, performs SINS calculation, and simultaneously performs Kalman filtering for data processing. The Kalman filter parameters are corrected during motion by embedding TinyML.

[0009] The BDS receiver receives BDS satellite signals and outputs the current coordinates of the carrier in the navigation coordinate system. When the outdoor signal is good, it can obtain long-term stable positioning information.

[0010] Furthermore,

[0011] The gyroscope and accelerometer are used for strapdown inertial navigation calculations; the calculated results are combined with BDS positioning information through extended Kalman filtering for navigation calculations; and the Kalman coefficients are corrected using TinyML during the iteration process.

[0012] Furthermore,

[0013] With the assistance of TinyML, when the BDS signal meets the positioning requirements, the Kalman filter algorithm is used for data fusion to obtain an accurate position; the TinyML network is trained online in real time using the integrated navigation output information.

[0014] Furthermore,

[0015] When the BDS signal is lost, the pre-trained TinyML is used to compensate the SINS system and fuse it with optical flow information to solve the problem of rapid decline in integrated navigation accuracy.

[0016] The TinyML network topology consists of an input layer, hidden layers, and an output layer. TinyML adjusts the feedback mechanism of each layer's weights to an error backpropagation method. The filtering performance of the Kalman filter is affected by three parameters: prediction error, gain, and measurement error; these three parameters are trained using the TinyML network.

[0017] The beneficial effects of this invention are

[0018] The accuracy of BDS / SINS integrated navigation was further improved by correcting the Kalman estimation results using a TinyML network. To address the problem of a sharp decline in navigation accuracy caused by BDS signal unlocking under high dynamic and nonlinear environments, an optical flow sensor was introduced. The TinyML network was used to assist the integrated navigation filter, which significantly improved the positioning accuracy of the integrated navigation and solved the problems of BDS signal loss in a short time and rapid divergence of the strapdown inertial navigation system. Results show that optical flow assistance can significantly improve the accuracy of the integrated navigation system when BDS is lost, and the positioning results are reliable. Attached Figure Description

[0019] Figure 1 This is a block diagram of the present invention;

[0020] Figure 2 This is a schematic diagram of the cyclical update process;

[0021] Figure 3 This is a schematic diagram of the TinyML network topology;

[0022] Figure 4 This is a schematic diagram of the block matching algorithm. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0024] This invention primarily addresses the positioning of delivery vehicles within industrial parks (in environments where BeiDou satellite signals are temporarily lost). In such environments with dense high-rise buildings, BeiDou signals are often affected by short-term loss of signal, leading to a significant decrease in the positioning accuracy of traditional BeiDou positioning equipment. This invention proposes a combined positioning method, device, and storage medium based on TinyML and Kalman filtering. It combines the concepts of Kalman filtering and TinyML, applying them to BDS / SINS integrated navigation. When satellite signal strength is good, BDS and SINS positioning information are fused to obtain high-precision positioning information, and the Kalman filter is trained. When satellite signal strength is poor, pre-trained TinyML is used to fuse SINS and optical flow information to achieve complementary advantages. The effectiveness and reliability of the algorithm are verified through experiments.

[0025] This device comprises an IMU (Inertial Measurement Unit) consisting of a high-precision accelerometer, gyroscope, and magnetometer sensor, an ARM processor, a BDS (Browser Detection System) receiver, and an optical flow meter as its hardware platform, establishing a positioning device for mobile devices within a park. This invention proposes a device for combined positioning based on integrated navigation and optical flow information, achieving high-precision continuous positioning within a park. The gyroscope and accelerometer are used for strapdown inertial navigation (SINS) calculations. The calculated results are then used for combined navigation calculations using extended Kalman filtering (EDK) and BDS positioning information. During the iteration process, the Kalman coefficients are corrected using TinyML. With the assistance of TinyML, when the BDS signal meets the positioning requirements, the system uses a Kalman filtering algorithm for data fusion to obtain an accurate position. Simultaneously, the TinyML network is trained online in real-time using the integrated navigation output information; when the BDS signal loses lock, the pre-trained TinyML is used to compensate the SINS system and fuse it with optical flow information to address the problem of rapid decline in integrated navigation accuracy.

[0026] 1) IMU sensor section

[0027] A gyroscope contains an internal gyroscope that effectively detects the angular velocity of the carrier's rotation along its three axes. Integrating this velocity yields the rotation angle, and the actual direction is calculated from the deviation from the initial direction. An accelerometer detects the magnitude and direction of the acceleration experienced by the carrier, while a magnetometer measures the strength and direction of the magnetic field. By fusing data from these three sensors, the carrier's trajectory can be determined.

[0028] 2) Optical Flow Module

[0029] Optical flow is a device that can estimate the vehicle's speed by utilizing changes in an image. Using optical flow-assisted BDS / SINS integrated navigation can provide reliable speed information when the BDS signal is interrupted, thus suppressing the problem of rapid divergence of inertial navigation errors.

[0030] 3) ARM processor

[0031] The ARM processor is mainly used for algorithm deployment and inference, SINS calculation, and Kalman filtering for data processing. It also uses TinyML to correct the Kalman filter parameters during motion.

[0032] 4) BDS

[0033] The BDS receiver receives BDS satellite signals and outputs the current coordinates of the carrier in the navigation coordinate system. When the outdoor signal is good, it can obtain long-term stable positioning information.

[0034] Specific steps are as follows: Figure 2 As shown.

[0035] Kalman filtering, as a recursive algorithm, mainly includes two update processes: time update and measurement update. The iterative update process is illustrated as follows: Figure 2 As shown.

[0036] The time update (state prediction) process is as follows:

[0037]

[0038]

[0039] The measurement update (state correction) process is as follows:

[0040]

[0041]

[0042]

[0043] TinyML network topology is divided into input layer, hidden layer, and output layer, as follows: Figure 3 As shown.

[0044] TinyML network adjusts the feedback mechanism of each layer weight to the error backpropagation method.

[0045] As can be seen from equation (4), the filtering performance of the Kalman filter is affected by the prediction error. Gain K k Measurement error The effect of these three parameters is determined by training the TinyML network. In the TinyML network, the maximum number of iterations is 5000, the learning rate is 0.001, one hidden layer is selected, and the system is trained using the tradingdx function.

[0046] When BDS loses lock, correction is performed using an optical flow sensor. Optical flow methods involve observing pixel changes in the time domain between adjacent frames to calculate the instantaneous velocity of pixel motion. Numerous methods exist for estimating optical flow, among which the most typical include the Lucas and Kanade differential algorithm, phase-based methods, and the Sum of Absolute Errors (SAD) and Block Matching (BMA) algorithm. Considering the hardware platform and computational complexity, the Block Matching (BMA) algorithm based on SAD is chosen for optical flow calculation. With a sampling update frequency of 250Hz, it offers advantages such as fast algorithm convergence, low device power consumption, and high system real-time performance. The general principle and flow of this algorithm are as follows: Figure 4 As shown.

[0047] At a certain moment, a tracking block containing l*l pixels is selected from the acquired frame data. When the next frame image is acquired, a block containing the same l*l pixels is searched within the search area to minimize the sum of the absolute values ​​of the grayscale differences (SAD) of each corresponding pixel between the tracking block and the prediction block, thereby obtaining the optical flow vector u. Here, i and j represent the horizontal and vertical offsets of the tracking block at the current moment from the tracking block at the previous moment, respectively. The velocity estimation model used in this invention is a planar model, with the camera mounted at the bottom of the mobile device and the optical axis perpendicular to the ground to achieve vertical imaging of the ground texture.

[0048] This invention not only overcomes the shortcomings of BDS signal loss in harsh environments and Kalman filtering with large fluctuations in nonlinear environments, but also greatly improves the positioning accuracy of the integrated navigation system.

[0049] The above description is merely a preferred embodiment of the present invention and is used only to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A positioning device for mobile devices within a campus based on TinyML, characterized in that, This includes an IMU consisting of an accelerometer, gyroscope, and magnetometer; an ARM processor; a BDS receiver; and an optical flow meter. The gyroscope calculates the actual direction by measuring the deviation from the initial direction; the accelerometer is used to detect the magnitude and direction of the acceleration experienced by the carrier; and the magnetometer measures the strength and direction of the magnetic field. The trajectory of the carrier is obtained by fusing data from the three sensors. Optical flow meters use optical flow-assisted BDS / SINS integrated navigation to provide velocity information when the BDS signal is interrupted, thus suppressing the problem of rapid divergence of inertial navigation errors. The ARM processor performs algorithm deployment and inference, performs SINS calculation, and performs Kalman filtering for data processing. It also embeds TinyML to correct the Kalman filter parameters during motion. The BDS receiver receives BDS satellite signals and outputs the current coordinates of the carrier in the navigation coordinate system to obtain positioning information; When the BDS signal is lost, the pre-trained TinyML is used to compensate the SINS system and fuse it with optical flow information to solve the problem of rapid decline in integrated navigation accuracy.

2. The apparatus according to claim 1, characterized in that, The gyroscope and accelerometer are used to perform strapdown inertial navigation calculations; the calculated results are then combined with BDS positioning information through extended Kalman filtering for navigation calculations.

3. The apparatus according to claim 2, characterized in that, The Kalman coefficients are corrected using TinyML during the iteration process.

4. The apparatus according to claim 3, characterized in that, With the assistance of TinyML, when the BDS signal meets the positioning requirements, the Kalman filter algorithm is used for data fusion to obtain an accurate position.

5. The apparatus according to claim 4, characterized in that, The TinyML network was trained online in real time using the output information of the integrated navigation system.

6. The apparatus according to claim 4, characterized in that, The TinyML network topology consists of an input layer, a hidden layer, and an output layer. The TinyML network adjusts the feedback mechanism of the weights in each layer to the error backpropagation method.

7. The apparatus according to claim 6, characterized in that, The filtering performance of Kalman filtering is affected by three parameters: prediction error, gain, and measurement error; these three parameters are trained using a TinyML network.

Images