Device positioning method, wearable device and readable storage medium

Abstract

The invention discloses an equipment positioning method, wearable equipment and a readable storage medium, and relates to the technical field of wearable equipment, and the method comprises the following steps: obtaining first measurement data of an ultra-wideband base station and second measurement data of a visual inertial odometer; according to the non-line-of-sight error degree of the ultra-wideband base station, the ultra-wideband measurement noise covariance is dynamically adjusted, and the ultra-wideband measurement noise covariance is in positive correlation with the non-line-of-sight error degree; an observation noise covariance matrix used for Kalman filtering is constructed based on the adjusted ultra-wideband measurement noise covariance, and the ultra-wideband measurement noise covariance is used for representing the uncertainty weight of the first measurement data in the filtering fusion process; and inputting the first measurement data and the second measurement data into a Kalman filter, and fusing the two types of measurement data by using the observation noise covariance matrix to obtain a positioning result of the equipment. According to the invention, under the condition of limited resources, fusion positioning with low time delay and high robustness is realized.

Description

Technical Field

[0001] This application relates to the field of wearable device technology, and in particular to a device positioning method, a wearable device, and a readable storage medium. Background Technology

[0002] In applications requiring high-precision positioning, such as Augmented Reality (AR) and mobile robotics, the accuracy and robustness of device localization are crucial. Ultra-wideband (UWB) technology boasts strong penetration, high temporal resolution, and centimeter-level ranging potential, offering significant advantages in environments without satellite signals. Visual-Inertial Odometry (VIO) achieves continuous six-degree-of-freedom self-localization of devices in unknown environments through the complementarity of visual and inertial sensors. To overcome the limitations of single sensors, fusing UWB and VIO, leveraging UWB's global anti-drift characteristics and VIO's high-frequency relative accuracy, has become an important direction for improving positioning performance in complex environments.

[0003] Traditional data fusion techniques are mainly divided into two categories: loosely coupled and tightly coupled. Loosely coupled schemes are based on filtering frameworks and have high computational efficiency, but they are difficult to dynamically suppress non-line-of-sight errors in UWB, resulting in limited accuracy. Tightly coupled schemes use optimization methods to deeply fuse data, achieving higher accuracy, but they have high computational complexity and face challenges in real-time performance.

[0004] When the aforementioned technologies are applied to resource-constrained platforms such as AR glasses, they face a prominent contradiction: loose coupling, due to unstable accuracy, cannot meet the requirements of precise AR overlay; tight coupling, on the other hand, results in excessive computational burden, affecting real-time response and battery life. Therefore, how to achieve fusion positioning with both low latency and high robustness under limited resource conditions remains an urgent technical problem to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a device positioning method, a wearable device, and a readable storage medium, aiming to solve the technical problem of how to achieve fusion positioning with both low latency and high robustness under limited resource conditions.

[0006] To achieve the above objectives, this application provides a device positioning method, which includes the following steps: Acquire the first measurement data from the ultra-wideband base station and the second measurement data from the visual inertial odometry. The ultra-wideband measurement noise covariance is dynamically adjusted based on the degree of non-line-of-sight error of the ultra-wideband base station, wherein the ultra-wideband measurement noise covariance is positively correlated with the degree of non-line-of-sight error. Based on the adjusted ultra-wideband measurement noise covariance, an observation noise covariance matrix for Kalman filtering is constructed, wherein the ultra-wideband measurement noise covariance is used to characterize the uncertainty weight of the first measurement data in the filtering and fusion process; The first and second measurement data are input into a Kalman filter, and the two types of measurement data are fused using the observation noise covariance matrix to obtain the positioning result of the device.

[0007] In one embodiment, before the step of dynamically adjusting the ultra-wideband measurement noise covariance based on the non-line-of-sight error level of the ultra-wideband base station, the method further includes: The signal strength, current measurement residual, and average measurement residual within a preset time window of the ultra-wideband base station are obtained. Calculate the residual difference between the current measurement residual and the average measurement residual; Based on the difference between the signal strength and the residual, the degree of non-line-of-sight error of the ultra-wideband base station is determined, wherein the degree of non-line-of-sight error is negatively correlated with the signal strength and positively correlated with the residual difference.

[0008] In one embodiment, the step of determining the non-line-of-sight error level of the ultra-wideband base station based on the signal strength and the residual difference includes: If the signal strength is less than a first preset strength threshold and the residual difference is greater than a first preset residual threshold, then the high non-line-of-sight error is determined as the degree of non-line-of-sight error of the ultra-wideband base station. If the signal strength is less than the second preset strength threshold, or the residual difference is greater than the second preset residual threshold, then the non-line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station, wherein the second preset strength threshold is greater than or equal to the first preset strength threshold, and the second preset residual threshold is less than or equal to the first preset residual threshold. If the signal strength is greater than or equal to the second preset strength threshold and less than the third preset strength threshold, or the residual difference is less than or equal to the second preset residual threshold, then the normal line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station, wherein the third preset strength threshold is greater than the second preset strength threshold. If the signal strength is greater than or equal to the third preset strength threshold, then the low non-line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station.

[0009] In one embodiment, the step of dynamically adjusting the ultra-wideband measurement noise covariance based on the non-line-of-sight error level of the ultra-wideband base station includes: Obtain the pre-set baseline measurement noise covariance; If the non-line-of-sight error of the ultra-wideband base station is high non-line-of-sight error, then the reference measurement noise covariance is multiplied by the first preset amplification factor to obtain the adjusted ultra-wideband measurement noise covariance. If the non-line-of-sight error of the ultra-wideband base station is medium non-line-of-sight error, then the reference measurement noise covariance is multiplied by the second preset amplification factor to obtain the adjusted ultra-wideband measurement noise covariance, wherein the second preset amplification factor is smaller than the first preset amplification factor. If the non-line-of-sight error level of the ultra-wideband base station is low non-line-of-sight error, then the reference measurement noise covariance is multiplied by a preset reduction factor to obtain the adjusted ultra-wideband measurement noise covariance. If the non-line-of-sight error of the ultra-wideband base station is a normal non-line-of-sight error, then the reference measurement noise covariance is determined as the adjusted ultra-wideband measurement noise covariance.

[0010] In one embodiment, after the steps of inputting the first measurement data and the second measurement data into a Kalman filter and fusing the two types of measurement data using the observation noise covariance matrix to obtain the positioning result of the device, the method further includes: In response to a high-precision device positioning request, the pre-integration residual of the IMU in the visual inertial odometry, the first measurement residual of the visual inertial odometry, and the second measurement residual of the ultra-wideband base station are obtained. Using the pre-integration residual, the first measurement residual, and the second measurement residual as constraints, the positioning result is optimized using a factor graph algorithm to generate an optimized positioning result.

[0011] In one embodiment, prior to the step of acquiring IMU pre-integration data in response to a high-precision device positioning request, the method further includes: The high-precision device positioning request is triggered once every N keyframes; or... If the non-line-of-sight error of the ultra-wideband base station is detected to be greater than a preset threshold, a high-precision device positioning request is triggered. The keyframe is an image frame selected from the image sequence during the operation of the visual inertial odometry according to a preset selection strategy. The selection strategy is: the visual sensor attitude change is greater than a preset pose change threshold, or the visual feature change significance of the image frame is greater than a preset significance threshold.

[0012] In one embodiment, the step of acquiring the first measurement data of the ultra-wideband base station and the second measurement data of the visual inertial odometry includes: Acquire second measurement data from the visual inertial odometry and the current scene image captured by the visual sensor in the visual inertial odometry, wherein the second measurement data is the current pose of the visual sensor; Based on the current pose and the historical pose measured by the visual inertial odometry, the pose change of the visual sensor is calculated. Based on the current scene image and the historical scene images acquired by the visual sensor, the significance of visual feature changes is calculated. If the change in pose is greater than a preset pose change threshold, or the significance of the change in visual features is greater than a preset significance threshold, then the first measurement data of the ultra-wideband base station is obtained. If the pose change is less than or equal to a preset pose change threshold, and the image change degree is less than or equal to a preset degree threshold, then the device is located using an optical flow tracking algorithm.

[0013] In one embodiment, the positioning result is the device pose. After the steps of inputting the first measurement data and the second measurement data into a Kalman filter and fusing the two types of measurement data using the observation noise covariance matrix to obtain the device positioning result, the method further includes: SLAM maps can be built or updated based on the device pose.

[0014] In addition, to achieve the above objectives, this application also provides a wearable device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the device positioning method as described above.

[0015] In addition, to achieve the above objectives, this application also provides a readable storage medium, which is a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of the device positioning method described above.

[0016] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the device positioning method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: This application embodiment effectively improves the problem of unstable positioning accuracy caused by the inability to dynamically suppress non-line-of-sight errors in loosely coupled schemes by dynamically adjusting the measurement noise covariance of the ultra-wideband base station based on the degree of non-line-of-sight error, and constructing the observation noise covariance matrix of the Kalman filter accordingly. Specifically, a fusion basis is established by acquiring first measurement data from the ultra-wideband base station and second measurement data from the visual inertial odometry system, and the ultra-wideband measurement noise covariance is dynamically adjusted according to the degree of non-line-of-sight error of the ultra-wideband base station, so that the covariance value maintains a positive correlation with the degree of error. This adjustment mechanism enables the Kalman filter observation noise covariance matrix constructed based on this to accurately characterize the real-time uncertainty weight of the first measurement data in the fusion process. When the non-line-of-sight error increases, the corresponding increase in the ultra-wideband measurement noise covariance will be reflected in the matrix, thereby naturally reducing the contribution weight of the first measurement data in the subsequent Kalman filter fusion process. By controlling the observation noise covariance matrix, the Kalman filter achieves adaptive suppression of non-line-of-sight interference during the fusion process of the first and second measurement data. The final device positioning result maintains the computational efficiency advantage of the filtering framework while improving the positioning accuracy and stability in complex environments, thus achieving fusion positioning with both high robustness and low latency under resource-constrained conditions. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the first embodiment of the device positioning method of this application; Figure 2 This is a schematic diagram illustrating the process for determining the degree of non-line-of-sight error in an embodiment of the device positioning method of this application; Figure 3 This is a schematic diagram of the ultra-wideband measurement noise covariance adjustment process involved in an embodiment of the device positioning method of this application; Figure 4 This is a schematic diagram of the factor graph optimization process involved in one embodiment of the device positioning method of this application; Figure 5 This is a schematic diagram of the hardware operating environment of the device positioning method apparatus in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] In industrial augmented reality (AR) applications, achieving high-precision device positioning for AR glasses in complex indoor environments often relies on Simultaneous Localization and Mapping (SLAM) technology. However, traditional visual-inertial odometry (VIO) solutions accumulate errors over long periods of operation, and even with the integration of an IMU, drift cannot be completely avoided. Pure visual / inertial positioning, lacking absolute reference, struggles to guarantee global accuracy in large spaces and long-distance scenarios. On the other hand, ultra-wideband (UWB) indoor positioning technology has attracted attention due to its advantages such as low power consumption and high interference resistance. UWB active beacons can provide global coordinate references in the absence of GPS, but their ranging data is susceptible to non-line-of-sight (NLOS) occlusion and multipath effects, resulting in significant errors and fluctuations. A single UWB positioning system has limited accuracy in complex indoor environments, necessitating the integration of other sensors for improvement.

[0024] Current research attempts to fuse UWB and VIO, with traditional methods mainly falling into two categories: loosely coupled fusion based on Kalman filtering, which offers good real-time performance but limited accuracy; and tightly coupled fusion based on optimization (such as factor graphs), which offers high accuracy but high computational cost. In the field of mobile robotics, some researchers have used adaptive Kalman filtering to fuse UWB and IMU, and then combined it with visual SLAM for optimization, significantly improving positioning accuracy in complex environments. However, directly applying these technologies to AR glasses faces specific challenges: limited device computing power and battery capacity (requiring low power consumption), the need for real-time response (low latency and high frame rate), and the need for rapid identification and registration of critical equipment in industrial settings (image anchoring to support AR information overlay). Existing solutions are optimized for the platform characteristics of AR glasses; if multi-sensor fusion algorithms process all data indiscriminately, it may lead to excessive system latency or rapid battery depletion.

[0025] In summary, there is an urgent need for a multi-source sensor fusion positioning method for industrial AR glasses, capable of achieving high real-time accuracy with low power consumption on a Linux embedded platform. This method should combine the advantages of UWB and VIO: utilizing the global reference provided by UWB to suppress visual-inertial drift, and combining visual / inertial details to improve the stability of UWB ranging. Simultaneously, a data sparsity strategy is needed to balance data volume and response speed, avoiding unnecessary redundant computation.

[0026] Based on this, the main solution of this application is as follows: First measurement data from an ultra-wideband base station and second measurement data from a visual inertial odometry system are acquired; the ultra-wideband measurement noise covariance is dynamically adjusted according to the degree of non-line-of-sight error of the ultra-wideband base station, wherein the ultra-wideband measurement noise covariance is positively correlated with the degree of non-line-of-sight error; based on the adjusted ultra-wideband measurement noise covariance, an observation noise covariance matrix for Kalman filtering is constructed, wherein the ultra-wideband measurement noise covariance is used to characterize the uncertainty weight of the first measurement data in the filtering fusion process; the first measurement data and the second measurement data are input into a Kalman filter, and the two types of measurement data are fused using the observation noise covariance matrix to obtain the positioning result of the device.

[0027] This application establishes a fusion foundation by acquiring first measurement data from an ultra-wideband (UWB) base station and second measurement data from a visual-inertial odometry (VIO). It dynamically adjusts the UWB measurement noise covariance based on the degree of non-line-of-sight (NOS) error of the UWB base station, ensuring a positive correlation between the covariance value and the error level. This adjustment mechanism allows the Kalman filter observation noise covariance matrix constructed based on this mechanism to accurately characterize the real-time uncertainty weight of the first measurement data during the fusion process. When the NOS error increases, the corresponding increase in the UWB measurement noise covariance is reflected in the matrix, naturally reducing the contribution weight of the first measurement data in the subsequent Kalman filter fusion process. Through the adjustment of this observation noise covariance matrix, the Kalman filter achieves adaptive suppression of NOS interference during the fusion process of the first and second measurement data. The final device positioning result maintains the computational efficiency advantage of the filtering framework while improving positioning accuracy and stability in complex environments, thus achieving highly robust and low-latency fusion positioning under resource-constrained conditions.

[0028] It should be noted that the execution subject of each embodiment of the device positioning method of this application can be a wearable device capable of realizing the above functions, such as a VR (Virtual Reality) headset, an AR (Augmented Reality) headset, etc., and the embodiments of the device positioning method of this application do not impose specific limitations on this.

[0029] Based on this, this application proposes a device positioning method according to a first embodiment. In this embodiment, referring to... Figure 1 As shown, the device positioning method includes the following steps S10~S40: Step S10: Obtain the first measurement data of the ultra-wideband base station and the second measurement data of the visual inertial odometry. During device localization, initial measurement data is acquired from Ultra-Wideband (UWB) base stations. This data typically represents the distance information between the UWB base station and the tag to be located (also known as the tag anchor or anchor device), calculated by measuring the Time of Flight (ToF) or Time Difference of Arrival (TDoA) of the UWB signal. These distance measurements provide the localization system with absolute positional constraints within the global coordinate system of the UWB network.

[0030] Simultaneously, secondary measurement data is acquired from a visual-inertial odometry (VIO) system. The VIO system integrates image sequences acquired by a visual sensor (such as a camera) with acceleration and angular velocity data output by an inertial measurement unit (IMU). By tracking feature points in the images and optimizing them with tight coupling with IMU data, VIO can continuously output high-frequency relative pose changes (including position and attitude) of the device in a local coordinate system, providing a continuous motion trajectory for localization.

[0031] To facilitate subsequent fusion processing, the output time of one data source (e.g., VIO) is typically used as a benchmark to time-align the measurements from the other data source (e.g., UWB). A common practice is to unify the two types of asynchronous data to the same time point by interpolation or selecting the measurement data with the closest timestamps, thus providing time-synchronized observation input for fusion algorithms such as Kalman filtering.

[0032] Step S20: Dynamically adjust the ultra-wideband measurement noise covariance according to the degree of non-line-of-sight error of the ultra-wideband base station, wherein the ultra-wideband measurement noise covariance is positively correlated with the degree of non-line-of-sight error; After acquiring the measurement data, the UWB measurement noise covariance is dynamically adjusted. This adjustment process is based on the degree of non-line-of-sight (NLS) error of the UWB base station. NLS is a common problem in UWB positioning, which can lead to deviations in measurement data and thus affect positioning accuracy. The degree of NLS can be assessed in various ways, such as by detecting multipath effects of the UWB signal, signal strength attenuation, or comparison with a known environmental map.

[0033] The initial value of the UWB measurement noise covariance is preset based on the hardware performance and environmental conditions of the UWB base station. During the positioning process, the UWB measurement noise covariance is dynamically adjusted based on the assessed non-line-of-sight (NLS) error level, ensuring it is positively correlated with the NLS error level. This adjustment can be achieved through empirical formulas, machine learning models, or adaptive algorithms, aiming to reflect changes in the reliability of UWB measurement data in real time, thereby providing a more accurate basis for weight allocation in the subsequent filtering and fusion process.

[0034] Step S30: Based on the adjusted ultra-wideband measurement noise covariance, construct the observation noise covariance matrix for Kalman filtering, wherein the ultra-wideband measurement noise covariance is used to characterize the uncertainty weight of the first measurement data in the filtering and fusion process; After dynamically adjusting the ultra-wideband measurement noise covariance, the next step is to construct the observation noise covariance matrix for Kalman filtering based on the adjusted covariance values. The observation noise covariance matrix is ​​a diagonal matrix whose diagonal elements include the adjusted ultra-wideband measurement noise covariance and the visual inertial odometry measurement noise covariance.

[0035] The ultra-wideband measurement noise covariance in this matrix is ​​used to characterize the uncertainty weight of the first measurement data in the filtering and fusion process. A larger value indicates lower reliability of the measurement data and a smaller weight in the fusion process. The measurement noise covariance of the visual inertial odometry (VIO) can be pre-set according to the accuracy characteristics of its sensor and remains relatively stable during the fusion process. By constructing such an observation noise covariance matrix, a more accurate weight allocation framework can be provided for the Kalman filter, thereby better balancing the contributions of the two measurement data during the fusion process.

[0036] Step S40: Input the first measurement data and the second measurement data into the Kalman filter, and use the observation noise covariance matrix to fuse the two types of measurement data to obtain the positioning result of the device.

[0037] The preprocessed and weighted first and second measurement data are input into the Kalman filter. The Kalman filter performs recursive estimation based on the preset system model and observation model, combined with the input measurement data and the constructed observation noise covariance matrix. At each time step, the Kalman filter updates the target device's position and attitude state estimates based on the state estimate from the previous time step and the measurement data from the current time step.

[0038] This fusion method fully leverages the advantages of both measurement data types while mitigating their respective shortcomings. The final positioning result includes the target device's position coordinates and attitude information in the global coordinate system. This result boasts high accuracy and robustness, meeting the positioning needs of devices in complex environments and providing reliable positioning support for various application scenarios.

[0039] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. On this basis, refer to Figure 2 As shown, before the step of dynamically adjusting the ultra-wideband measurement noise covariance based on the non-line-of-sight error level of the ultra-wideband base station, the method further includes: Step A10: Obtain the signal strength of the ultra-wideband base station, the current measurement residual, and the average measurement residual within a preset time window; Obtain the real-time received signal strength indication from the ultra-wideband base station. This strength value reflects the comprehensive attenuation of the signal during propagation due to path loss, obstruction, and multipath effects, and is a fundamental indicator for judging signal propagation conditions and link quality. Simultaneously, calculate the ultra-wideband measurement residual at the current moment. This residual is defined as the difference between the currently measured distance observation and the corresponding distance estimate calculated based on the current system state prediction (usually obtained through a Kalman filter prediction step). It intuitively represents the instantaneous deviation between a single observation and the system's expectation.

[0040] Furthermore, a sliding time window mechanism is employed to continuously collect the values ​​of all ultra-wideband measurement residuals within a preset historical time period. By calculating their arithmetic mean or weighted average, a residual benchmark that evolves over time is dynamically generated. This benchmark can reflect the overall level and trend of recent measurement errors, providing a reliable reference for identifying whether abnormal fluctuations have occurred in the current measurement.

[0041] Step A20: Calculate the residual difference between the current measurement residual and the average measurement residual; After obtaining the above parameters, calculate the difference between the current measurement residual and the average measurement residual. This difference reflects the degree of deviation between the current measurement and the historical measurement trend. If the difference is large, it may indicate that the current measurement has been affected by non-line-of-sight propagation, causing the measured value to deviate from the true value. By calculating this difference, the reliability of the current measurement can be assessed more accurately.

[0042] Step A30: Based on the signal strength and the residual difference, determine the degree of non-line-of-sight error of the ultra-wideband base station, wherein the degree of non-line-of-sight error is negatively correlated with the signal strength and positively correlated with the residual difference.

[0043] Typically, an error severity evaluation function can be constructed, with signal strength and the residual difference as inputs. In this function model, the output value of the non-line-of-sight (NOS) error severity is negatively correlated with the received signal strength (weaker signal suggests greater path loss and a higher error severity evaluation); simultaneously, it is positively correlated with the residual difference (the greater the deviation of the current measurement from historical normal levels, the higher the error severity evaluation). Ultimately, the function outputs a quantified NOS error severity value, which will be used in subsequent steps to dynamically adjust the ultra-wideband measurement noise covariance.

[0044] This embodiment constructs a multi-dimensional, dynamic error assessment foundation by acquiring the signal strength of an ultra-wideband base station, the current measurement residual, and the average measurement residual obtained based on a sliding time window. Signal strength directly reflects the instantaneous quality of the channel, the current measurement residual captures the instantaneous anomalies of a single observation, while the historical average residual establishes a dynamic error benchmark adapted to the environment. Calculating the difference between the current measurement residual and the average measurement residual effectively isolates abnormal deviations caused by sudden environmental changes (such as sudden obstruction), thereby separating instantaneous interference from background noise. Finally, by constructing an evaluation function that makes the degree of non-line-of-sight error negatively correlated with signal strength and positively correlated with the residual difference, the physical layer indicators characterizing channel quality and the algorithm layer indicators characterizing observation consistency are fused and quantified. This dual-parameter fusion evaluation mechanism ensures that the determination of the degree of non-line-of-sight error is no longer based on a simple judgment of a single threshold, but comprehensively considers the potential impact of signal attenuation and the historical deviation of observations, thereby improving the accuracy and environmental adaptability of error assessment and providing a reliable and quantitative decision-making basis for the subsequent precise adaptive adjustment of noise covariance.

[0045] In one possible implementation, the step of determining the non-line-of-sight error level of the ultra-wideband base station based on the signal strength and the residual difference includes: Step B10: If the signal strength is less than the first preset strength threshold and the residual difference is greater than the first preset residual threshold, then the high non-line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station. Determine signal strength Is it less than the first preset intensity threshold? Meanwhile, residual difference Is it greater than the first preset residual threshold? ,in, This represents the current measurement residual of the ultra-wideband base station. This represents the average measurement residual. If both conditions are met simultaneously, it indicates that there may be a serious non-line-of-sight propagation problem in the current measurement environment. A low signal strength means that the signal has undergone significant attenuation during transmission, possibly due to signal obstruction or reflection; while a large residual difference further indicates a significant deviation between the current measurement value and the historical measurement trend. Therefore, in this case, the non-line-of-sight error level is defined as high, indicating that the reliability of the current measurement data is low, and it needs to be given a lower weight in the subsequent fusion process.

[0046] In a preferred embodiment, ,in, The standard deviation of the measurement residuals of the ultra-wideband base station within a preset time window. A pre-set residual multiple threshold is introduced, which dynamically adjusts based on the standard deviation derived from historical statistics, according to the actual fluctuation level of recent measurement noise. When the environment is stable and measurement accuracy is high, the standard deviation is small, and the corresponding first preset residual threshold is also small, making the system more sensitive to errors. Conversely, in environments with high inherent noise, the threshold is automatically widened, thereby reducing misjudgments of normal fluctuations. This adaptive mechanism enables the judgment condition of "whether the residual difference is greater than the first preset residual threshold" to more intelligently and accurately identify abnormal measurements that truly exceed the typical fluctuation range of the current environment, thus corresponding to the "high" level in the non-line-of-sight error assessment, ensuring that the decisions in subsequent weight adjustment stages are both robust and accurate.

[0047] Step B20: If the signal strength is less than the second preset strength threshold, or the residual difference is greater than the second preset residual threshold, then the non-line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station, wherein the second preset strength threshold is greater than or equal to the first preset strength threshold, and the second preset residual threshold is less than or equal to the first preset residual threshold. If signal strength Less than the second preset intensity threshold or residual difference Greater than the second preset residual threshold If the signal strength or residual difference does not reach the standard for high non-line-of-sight error, then the degree of non-line-of-sight error is determined to be moderate. Here, the second preset strength threshold is greater than or equal to the first preset strength threshold, while the second preset residual threshold is less than or equal to the first preset residual threshold. This means that even if the signal strength or residual difference does not reach the standard for high non-line-of-sight error, as long as one of these conditions is met, it indicates that a certain degree of non-line-of-sight error may exist. In this case, the degree of non-line-of-sight error is determined to be moderate, indicating that the reliability of the current measurement data is between high non-line-of-sight error and normal line-of-sight error, and appropriate weighting adjustments are needed during the fusion process.

[0048] Similarly, in a preferred embodiment, ,in, Less than or equal to .

[0049] Step B30: If the signal strength is greater than or equal to the second preset strength threshold and less than the third preset strength threshold, or the residual difference is less than or equal to the second preset residual threshold, then the normal line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station, wherein the third preset strength threshold is greater than the second preset strength threshold. When signal strength Greater than or equal to the second preset intensity threshold And less than the third preset intensity threshold or residual difference Less than or equal to the second preset residual threshold In this case, the degree of non-line-of-sight error is defined as normal line-of-sight error. The third preset intensity threshold here is greater than the second preset intensity threshold. In this situation, the combination of signal strength and residual difference indicates that the current measurement environment is relatively stable, and the impact of non-line-of-sight error is small. Therefore, the reliability of the current measurement data can be considered high, close to normal line-of-sight measurement conditions. In this case, the degree of non-line-of-sight error is defined as normal, indicating that it can be given a higher weight in the fusion process.

[0050] Step B40: If the signal strength is greater than or equal to the third preset strength threshold, then the low non-line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station.

[0051] Finally, if the signal strength Greater than or equal to the third preset intensity threshold If the non-line-of-sight (NOS) error level is low, then the signal strength in the current measurement environment is high, the signal transmission conditions are good, and the impact of NOS errors is minimal. In this case, the reliability of the current measurement data can be considered very high, with almost no NOS error. Therefore, a low NOS error level means that it can be given the highest weight in the fusion process, making full use of these high-precision measurement data to improve the overall positioning accuracy.

[0052] In one possible implementation, refer to Figure 3 As shown, the step of dynamically adjusting the ultra-wideband measurement noise covariance based on the non-line-of-sight error level of the ultra-wideband base station includes: Step C10: Obtain the preset benchmark measurement noise covariance; The reference measurement noise covariance can be obtained from the system's pre-configured parameters. This benchmark value is pre-set based on the calibration test results of ultra-wideband base stations in ideal or standard line-of-sight environments. It characterizes the inherent, relatively stable level of uncertainty in distance measurement under conditions of no significant interference, and serves as the initial reference benchmark for all dynamic adjustments.

[0053] Step C20: If the non-line-of-sight error of the ultra-wideband base station is high non-line-of-sight error, then the reference measurement noise covariance is multiplied by the first preset amplification factor to obtain the adjusted ultra-wideband measurement noise covariance. If the evaluation results show that the non-line-of-sight error level of the ultra-wideband base station is high, it means that the current measurement environment is severely affected by non-line-of-sight propagation, resulting in low reliability of the measurement data. In this case, the benchmark measurement noise covariance is compared with the first preset amplification factor. Multiplication. The first preset amplification factor is a value greater than 1, used to significantly increase the noise covariance, thereby reducing the weight of these measurement data in the subsequent Kalman filtering process and reducing their impact on the positioning results.

[0054] Step C30: If the non-line-of-sight error of the ultra-wideband base station is medium non-line-of-sight error, then the reference measurement noise covariance is multiplied by the second preset amplification factor to obtain the adjusted ultra-wideband measurement noise covariance, wherein the second preset amplification factor is smaller than the first preset amplification factor. When the non-line-of-sight error level is moderate, it indicates that there is some, but not particularly severe, non-line-of-sight error in the measurement environment. In this case, the reference measurement noise covariance is compared with the second preset amplification factor. Multiply. The second preset amplification factor is less than the first preset amplification factor, but still greater than 1. This adjustment method can moderately increase the noise covariance to reflect the moderate reliability of the measurement data, while providing appropriate weight adjustments during the fusion process to balance positioning accuracy and robustness.

[0055] Step C40: If the non-line-of-sight error of the ultra-wideband base station is low non-line-of-sight error, then the reference measurement noise covariance is multiplied by a preset reduction factor to obtain the adjusted ultra-wideband measurement noise covariance. When the evaluation results show that the non-line-of-sight error of the ultra-wideband base station is low, it indicates that the signal transmission conditions of the current measurement environment are very ideal, and the impact of non-line-of-sight error is minimal. In this case, to further optimize positioning accuracy, it is necessary to adjust the reference measurement noise covariance. Specifically, this involves adjusting the reference measurement noise covariance with a preset reduction factor. Multiplication. This reduction factor is a positive number less than 1, and its function is to moderately reduce the value of the noise covariance. In this way, the weight of the measurement data in the Kalman filter fusion process can be increased, thereby making full use of these high-precision measurement data to improve the accuracy and reliability of the positioning results.

[0056] Step C50: If the non-line-of-sight error level of the ultra-wideband base station is normal non-line-of-sight error, then the reference measurement noise covariance is determined as the adjusted ultra-wideband measurement noise covariance.

[0057] If the non-line-of-sight error of the ultra-wideband (UWBS) base station is assessed as normal, it means that the signal transmission conditions of the current measurement environment are at a standard level, with neither significant non-line-of-sight error nor particularly ideal signal conditions. In this case, no additional adjustment to the baseline measurement noise covariance is required. The pre-set baseline measurement noise covariance is directly used as the adjusted UWBS measurement noise covariance. This approach preserves the original weights of the measurement data, is suitable for most conventional measurement scenarios, and ensures that the positioning system maintains stable performance under normal conditions.

[0058] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. On this basis, refer to Figure 4 As shown, after the steps of inputting the first measurement data and the second measurement data into a Kalman filter and fusing the two types of measurement data using the observation noise covariance matrix to obtain the positioning result of the device, the method further includes: Step D10: In response to the high-precision device positioning request, obtain the pre-integration residual of the IMU in the visual inertial odometry, the first measurement residual of the visual inertial odometry, and the second measurement residual of the ultra-wideband base station. In response to a high-precision device positioning request initiated by the system or application layer (e.g., when precise AR content registration or key waypoints are required), a backend optimization process is initiated. First, various residual data are acquired to construct optimization constraints. Specifically, this includes: extracting the pre-integrated residual of the IMU from the visual inertial odometry system, which reflects the consistency error between inertial measurements and motion state estimations between adjacent image frames; simultaneously, acquiring the first measurement residual of the visual inertial odometry, which can be visual reprojection error or relative pose error, respectively measuring the deviation between the position obtained by projecting 3D map points onto the current image plane and the actual observed feature point pixel position, and the deviation between the relative pose transformation estimated between adjacent keyframes based on visual or inertial data and the relative transformation inferred through feature matching, direct alignment, or IMU pre-integration; and the second measurement residual of the ultra-wideband base station, which can be the difference between the current UWB distance (or angle) observation and the corresponding value predicted based on the system state.

[0059] Step D20: Using the pre-integration residual, the first measurement residual, and the second measurement residual as constraints, the positioning result is optimized using a factor graph algorithm to generate an optimized positioning result.

[0060] Based on the acquired residual data, a unified factor graph model is constructed for optimization. Specifically, the device's pose state can be used as the optimization variable. The pre-integration residual, the first measurement residual, and the second measurement residual are modeled as IMU factors, visual factors, and UWB factors connected to the factor graph, respectively, serving as nonlinear constraints on the state variables. Subsequently, by executing a factor graph optimization algorithm (e.g., using a nonlinear least squares solver such as the Levenberg-Marquardt method), the pose state estimation is iteratively adjusted to minimize the sum of the constraint residuals represented by all factors. Finally, the optimized positioning result after global consistency optimization is output. This step effectively utilizes multi-source information for smoothing and optimization over a longer time window, further suppressing cumulative errors or instantaneous disturbances that may remain in the Kalman filtering stage, thereby providing higher accuracy and consistency in positioning output at critical nodes or when needed.

[0061] This embodiment incorporates three types of constraints—IMU pre-integration residual, visual reprojection error (or relative pose error), and ultra-wideband ranging residual—into a unified optimization framework. This approach fully leverages the complementary characteristics of data from different sensors: IMU constraints provide a continuous high-frequency motion dynamics model between frames, effectively maintaining trajectory smoothness and physical plausibility; visual constraints utilize environmental geometric features to provide high-precision local relative pose constraints, effectively suppressing cumulative drift; and UWB constraints contribute global, absolute ranging (or angle) information, providing an "anchor point" for drift suppression in the optimization problem. The combined effect of these three constraints enables the optimization process to not only utilize information within a longer time window for global trajectory smoothing and correction, reducing potential instantaneous noise or hysteresis errors in the front-end Kalman filter, but more importantly, enhances the system's robustness and fault tolerance in scenarios where a single sensor fails, such as missing visual textures, rapid motion, or UWB non-line-of-sight interference, through mutual verification and supplementation of multi-source information. Ultimately, when responding to high-precision requests (such as AR registration), this optimization mechanism can further converge to a consistent, accurate, and reliable positioning result over a longer spatiotemporal dimension, based on real-time filtered positioning. This allows for the provision of location output that meets higher application requirements on demand without affecting real-time performance.

[0062] In one possible implementation, prior to the step of acquiring IMU pre-integration data in response to a high-precision device positioning request, the method further includes: Step E10: Trigger the high-precision device positioning request once every N keyframes; During the operation of Visual Inertial Odometry (VIO), keyframes are selected from the image sequence according to a preset strategy. These keyframes are crucial image frames for localization and mapping. The selection strategy includes two scenarios: first, the pose change of the visual sensor exceeds a preset pose change threshold, indicating significant movement or rotation of the visual sensor in space, requiring a reassessment of localization accuracy; second, the visual feature change of the image frame exceeds a preset saliency threshold, typically indicating the appearance of new feature points or significant changes in scene structure. Based on these keyframes, a high-precision device localization request is automatically triggered every N keyframes. This periodic triggering mechanism ensures optimization of localization accuracy at critical moments, especially in scenarios with complex visual sensor motion or rapidly changing environments.

[0063] Alternatively, in step E20, if the non-line-of-sight error of the ultra-wideband base station is detected to be greater than a preset threshold, a high-precision device positioning request is triggered. In addition to periodic triggering based on keyframes, the non-line-of-sight (NFS) error level of the ultra-wideband (UWB) base station is monitored in real time. If the detected NFS error level exceeds a preset threshold, it indicates that the current measurement environment may be severely affected by NFS propagation, leading to a decrease in positioning accuracy. In this case, a high-precision device positioning request will be triggered immediately. This dynamic triggering mechanism can respond promptly to environmental changes, ensuring that the positioning results are corrected through optimized algorithms when the NFS error is large, thereby improving the robustness and reliability of the positioning system.

[0064] These two triggering mechanisms work together to ensure that the optimization process can be initiated promptly at critical moments requiring high-precision positioning. Periodic triggering of keyframes is suitable for routine positioning optimization needs, while dynamic triggering based on the degree of non-line-of-sight error can quickly respond to sudden environmental changes, thereby maintaining positioning accuracy under various complex conditions.

[0065] In one possible implementation, the step of acquiring the first measurement data from the ultra-wideband base station and the second measurement data from the visual inertial odometry includes: Step F10: Obtain the second measurement data of the visual inertial odometry and the current scene image collected by the visual sensor in the visual inertial odometry, wherein the second measurement data is the current pose of the visual sensor; Simultaneously acquire the second measurement data output by the visual inertial odometry (i.e., the six-degree-of-freedom pose estimation of the visual sensor at the current moment), as well as the current scene image acquired in real time by the visual sensor.

[0066] Step F20: Calculate the pose change of the visual sensor based on the current pose and the historical pose measured by the visual inertial odometry. Based on the obtained current pose, it is compared with the historical pose recorded by the visual inertial odometry in the previous keyframe or the previous moment. The pose change of the visual sensor is calculated through relative pose transformation, which is used to quantify the motion amplitude of the device since the last reference moment.

[0067] Step F30: Calculate the significance of visual feature changes based on the historical scene images acquired by the visual sensor from the current scene image; Feature extraction and matching algorithms can be used to analyze the matching rate of visual features, the emergence ratio of new feature points, or the change in overall image similarity between the current scene image and the stored historical scene image (usually the previous keyframe image), thereby calculating a quantitative visual feature change significance to assess the degree of updating of the environment appearance.

[0068] Step F40: If the pose change is greater than a preset pose change threshold, or the visual feature change significance is greater than a preset significance threshold, then the first measurement data of the ultra-wideband base station is obtained. The system determines whether the calculated pose change exceeds a preset pose change threshold and whether the calculated visual feature change significance exceeds a preset significance threshold. If either condition is met, the system is identified as being in a critical state of intense motion or dramatic scene change, where the short-term error of the visual inertial odometry (VIO) may increase or tracking risks may arise. In this case, the system actively acquires first measurement data (such as distance observations) from the ultra-wideband (UWB) base station and inputs this UWB data along with the current pose data from the VIO into the subsequent fusion positioning process (e.g., Kalman filtering) to introduce absolute position constraints for stabilization and correction.

[0069] Step F50: If the pose change is less than or equal to a preset pose change threshold, and the image change degree is less than or equal to a preset degree threshold, then the device is located using an optical flow tracking algorithm.

[0070] If the calculated pose change is less than or equal to the pose change threshold, and the calculated visual feature change significance is less than or equal to a preset significance threshold, it indicates that the device is in a low-uncertainty state with relatively static or stable motion and a stable visual environment. In this case, it is determined that there is no need to activate the high-power UWB ranging and complex fusion, and instead, the more computationally efficient optical flow tracking algorithm is called to directly estimate the incremental motion of the device based on the continuous image sequence, thereby achieving low-overhead continuous positioning.

[0071] This embodiment uses real-time motion and scene change indicators generated within the VIO to intelligently decide when to introduce global UWB measurements for fusion correction. This mechanism precisely focuses the use of UWB resources during "critical periods" when the VIO may be unreliable or at high risk of error accumulation, while employing extremely low-power optical flow tracking during "stable periods" when the VIO is sufficiently robust. This achieves a dynamic and optimal balance between positioning accuracy, robustness, device power consumption, and computational load.

[0072] In one possible implementation, the positioning result is the device pose. After the steps of inputting the first measurement data and the second measurement data into a Kalman filter and fusing the two types of measurement data using the observation noise covariance matrix to obtain the device positioning result, the method further includes: Step G10: Build or update the SLAM map based on the device pose.

[0073] After obtaining the current device pose (position and orientation) estimate by fusing UWB and VIO data through a Kalman filter, the Simultaneous Localization and Mapping (SLAM) mapping stage is performed based on this pose. Specifically, the current device pose is used as the pose estimate for keyframes, and visual feature points (such as ORB, SIFT, etc.) extracted from the image acquired by the vision sensor at this time, or their corresponding 3D point clouds, along with the pose information of the keyframe, are inserted or updated into the system's SLAM map. If it is the first time building, a global map is initialized using this; if the map already exists, the current observation is matched and associated with existing 3D points in the map to correct the position of map points, add new map areas, or optimize the pose constraints between keyframes, thereby achieving incremental expansion of the map and maintenance of global consistency.

[0074] For example, to help understand the technical concept or principle of the device positioning method after combining this embodiment with the first and second embodiments described above, a specific embodiment is now listed. In this specific embodiment, the device positioning process includes: Step 1, Sensor Calibration and Initial Alignment During system startup, spatial alignment and temporal synchronization between the UWB coordinate system and the camera-IMU coordinate system are completed. First, extrinsic parameters of the camera and IMU, as well as the positional offset of the UWB antenna relative to the camera / IMU coordinate system, are obtained through offline calibration. Then, in an environment with known UWB base station coordinates, the AR glasses are placed at a reference position. An initial global position estimate is obtained by ranging from UWB tags. This position is compared with the relative pose calculated by the VIO at the same moment, and the transformation matrix from the VIO initial origin to the global coordinate system is calculated. Subsequently, the relative poses output by all VIOs can be mapped to the global pose through this transformation. This process can be expressed by the following formula: Let the initialization time be The UWB solution yields the following position for the label in the global coordinate system: ; Obtain the extrinsic parameters of the UWB tag relative to the IMU through offline calibration: ; in, This indicates the offset of the label in the IMU coordinate system. This indicates the attitude offset of the tag in the IMU coordinate system; The position of the IMU in the global coordinate system at the initialization time is: ; in, Set by the direction of gravity and the initial orientation; At the same moment, the visual inertial odometry (VIO) outputs the IMU in the VIO coordinate system. The relative pose below: ; Aligning the VIO coordinate system with the UWB global coordinate system allows for the calculation of fixed transformations: ; At any time thereafter The relative pose output by VIO All of these can be mapped to global poses: .

[0075] To align the timelines of each sensor, a timestamp interpolation synchronization triggering mechanism is employed, ensuring that IMU sampling, image frames, and UWB ranging correspond to the most recent time points at the fusion time. After initialization, the system possesses a unified spatiotemporal reference framework, laying the foundation for subsequent fusion.

[0076] Step 2, Front-end data sparse sampling and preprocessing: IMU pre-integration: The front end employs a pre-integration method to numerically integrate the IMU's accelerometer and gyroscope data between two keyframe images, obtaining the relative motion (attitude increment and velocity change) over that time interval. This pre-integration result serves as the input to the IMU factor during tightly coupled optimization (i.e., optimization via factor graph algorithm). Pre-integration significantly reduces the frequency of IMU data that the back end needs to process, while preserving high-frequency motion information.

[0077] Visual keyframe selection: The front-end reduces the frequency of visual odometry calculations through a keyframe selection strategy. For example, a frame is selected as a keyframe for back-end SLAM mapping only when the camera pose change exceeds a certain threshold or the scene texture shows significant changes (i.e., the scene change saliency is greater than a preset saliency threshold); otherwise, it is skipped. This ensures that feature extraction and matching are performed only on frames with sufficient disparity changes, reducing unnecessary computation. For skipped frames, optical flow tracking algorithms can be used to determine their pose change relative to the previous keyframe, obtaining intermediate poses at a lower cost for smooth transitions in AR displays.

[0078] UWB ranging filtering: UWB tags send ranging data at a low frequency (10Hz). If the UWB refresh rate is higher than the visual keyframe rate, the front end down-frequency the UWB data; otherwise, the UWB data is up-frequency, and only the nearest frame of UWB data near the visual keyframe time is used for fusion. If UWB signal anomalies occur (e.g., RSSI anomalies or ranging value jumps), the front end can temporarily refrain from using the data to avoid passing obviously erroneous observations to the back end.

[0079] Data buffering and time alignment: A sensor data buffer is set up at the front end, and the IMU pre-integration results and the latest keyframe image pose estimation (i.e., the second measurement data) are buffered according to the timestamp. (obtained through a fast VIO algorithm) and the corresponding UWB ranging data (i.e., the first measurement data). The data is packaged into synchronous data packets. This ensures that the data delivered to the backend fusion module is aligned within the same time window, avoiding fusion errors caused by misalignment of multiple data sources.

[0080] Step 3: Back-end Kalman filter fusion The backend first runs an improved extended Kalman filter (EKF) to estimate and update the state in real time, defining the system state vector as: ; in, For location, For speed, For attitude quaternions (from IMU coordinate system) To the global coordinate system ), These are the zero bias values ​​for the gyroscope and accelerometer, respectively.

[0081] When new observation data (i.e., the new first and second measurement data) arrives, the observation residuals are calculated and the state is corrected. The observation vector includes measurement information from UWB and vision. The gain and state are calculated according to the Kalman filter standard formula to obtain the filtered position and attitude, i.e., the device positioning result. Specifically: For the UWB ranging observation model, let the position of the UWB tag in the IMU coordinate system be... Then its position in the global coordinate system is: ; No. Each tag anchor point The position in the global coordinate system is The corresponding theoretical distance measurement is: ; The actual UWB ranging observation model is as follows: ; Its residual is defined as: ; For the visual odometry pose observation model, the visual front end outputs the camera pose in the VIO coordinate system. From camera external parameters: ; The pose of the IMU in the VIO coordinate system can be recovered. ; Then obtained from initialization The measured pose of the IMU in the global coordinate system is obtained: ; Define visual observation vector

[0082] The observation model is

[0083] The position residual and attitude residual can be written as

[0084] in, It is a quaternion logarithmic mapping.

[0085] Combine visual pose observation with UWB ranging observation into a unified observation vector.

[0086] definition:

[0087] The observation noise covariance matrix is

[0088] in, For adaptively adjusted UWB measurement noise covariance, Measure the noise covariance matrix for VIO.

[0089] Specifically, The adjustment method is as follows: Within the sliding time window, the mean and standard deviation of the UWB measurement residuals are statistically analyzed:

[0090] Let the current RSSI of the ultra-wideband base station be... The soft threshold, hard threshold, and high-quality conditions are defined as follows:

[0091] , The RSSI threshold, The residual multiple threshold; Adaptive adjustment of UWB covariance based on observation quality :

[0092] in, The variance is measured as a baseline. As the amplification factor, This is the reduction factor.

[0093] Step 4, SLAM Map and Backend Optimization As the EKF continues to operate, the system constructs an incremental sparse SLAM map. The map consists of keyframe poses and 3D landmarks (from visual feature triangulation). When higher accuracy or map optimization is required, the system enters a second fusion path—the factor graph global optimization mode. In this mode, the backend collects IMU pre-integration constraints, visual observation constraints (relative poses or loop closure constraints formed by matching features between keyframes), and UWB observation constraints from several recent keyframes, constructing an optimization problem together. For example, a factor graph is constructed using the GTSAM library: nodes include keyframe poses and IMU biases, and factors include IMU pre-integration residuals, the first measurement residual of VIO, and the second measurement residual of UWB. The factor graph is solved using nonlinear optimization (Levenberg-Marquardt algorithm), yielding the optimized poses of all keyframes within the window. The optimization results are used to correct previously accumulated drift and update the keyframe poses in the SLAM map. In this specific implementation, factor graph optimization is not performed on every frame but is triggered intermittently according to a strategy: for example, every N keyframes or when… This process is performed once per cycle. This ensures the optimization effect while avoiding frequent and time-consuming calculations. After optimization, the latest keyframe pose is fed back to the front end as a reference to prevent the filtered estimate from gradually deviating from the true value.

[0094] Step 5, Image Anchoring and Device Recognition In the SLAM map building process, certain target devices in the industrial environment are added to the map as anchor points to enhance the stable overlay of real-world information. Specifically, this can be implemented in two ways: Anchoring based on manual tags: Visual tags (such as QR codes, April Tags, etc.) are attached to the surface of the target device, and the system pre-stores the ID of each tag and the corresponding device information.

[0095] Anchoring Based on Natural Features: For devices where labeling is inconvenient, an image anchoring algorithm based on natural appearance features is provided. First, in the offline phase, multi-angle photos of the target device are acquired, feature points are extracted, and a device feature template library is established. During runtime, when the camera captures the scene, a device feature matching step is added to the visual SLAM feature points: the current frame features are matched with device templates in the library. If a high-confidence device feature match is found in a certain area, the device is considered identified. Next, the relative pose between the camera and the device is calculated using the matched feature points (PnP solution). This pose is then converted to global coordinates to generate device anchor points. Once the anchor points are inserted into the map, even if the device is temporarily not in the center of the image, the system can use SLAM tracking to maintain an estimate of its position, thus ensuring that the superimposed virtual information is stably "attached" to the real device and does not drift with changes in viewing angle.

[0096] It should be noted that the above examples are only used to help understand this embodiment and do not constitute a limitation on the device positioning process of this embodiment. Any simple modifications based on this technical concept are within the protection scope of this application.

[0097] Furthermore, embodiments of this application also propose a wearable device, the wearable device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method described above.

[0098] refer to Figure 5 The diagram illustrates a structural schematic suitable for implementing the embodiments of this application. The wearable devices in the embodiments of this application may also include, but are not limited to, mobile terminals such as mobile phones, VR headsets, laptops, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The wearable device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0099] like Figure 5 As shown, the wearable device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the wearable device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the wearable device to communicate wirelessly or wiredly with other devices to exchange data. While wearable devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0100] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0101] The wearable device provided in this application, employing the device positioning method described in the above embodiments, can solve the technical problem of achieving both low latency and high robustness in fusion positioning under limited resource conditions. Compared with the prior art, the beneficial effects of the wearable device provided in this application are the same as those of the device positioning method described in the above embodiments, and other technical features of the wearable device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0102] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0104] In addition, to achieve the above objectives, embodiments of this application also provide a readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the device positioning method described in the above embodiments.

[0105] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0106] The aforementioned computer-readable storage medium may be included in the wearable device; or it may exist independently and not assembled into the wearable device.

[0107] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by a wearable device, cause the wearable device to perform the process steps of any embodiment of the device positioning method.

[0108] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0110] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the modules themselves.

[0111] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described device positioning method. This solves the technical problem of achieving both low latency and high robustness in fused positioning under limited resource conditions. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the device positioning method provided in the above embodiments, and will not be repeated here.

[0112] Furthermore, embodiments of this application also propose a computer program product, including a computer program that, when executed by a processor, implements the steps of the device positioning method described above.

[0113] The specific implementation of the computer program product in this application is basically the same as the embodiments of the above-mentioned device positioning method, and will not be repeated here.

[0114] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0115] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software sensor. This computer software sensor is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a wearable device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0117] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for positioning equipment, characterized in that, The device positioning method includes the following steps: Acquire the first measurement data from the ultra-wideband base station and the second measurement data from the visual inertial odometry. The ultra-wideband measurement noise covariance is dynamically adjusted based on the degree of non-line-of-sight error of the ultra-wideband base station, wherein the ultra-wideband measurement noise covariance is positively correlated with the degree of non-line-of-sight error. Based on the adjusted ultra-wideband measurement noise covariance, an observation noise covariance matrix for Kalman filtering is constructed, wherein the ultra-wideband measurement noise covariance is used to characterize the uncertainty weight of the first measurement data in the filtering and fusion process; The first and second measurement data are input into a Kalman filter, and the two types of measurement data are fused using the observation noise covariance matrix to obtain the positioning result of the device.

2. The equipment positioning method as described in claim 1, characterized in that, Before the step of dynamically adjusting the ultra-wideband measurement noise covariance based on the non-line-of-sight error level of the ultra-wideband base station, the method further includes: The signal strength, current measurement residual, and average measurement residual within a preset time window of the ultra-wideband base station are obtained. Calculate the residual difference between the current measurement residual and the average measurement residual; Based on the difference between the signal strength and the residual, the degree of non-line-of-sight error of the ultra-wideband base station is determined, wherein the degree of non-line-of-sight error is negatively correlated with the signal strength and positively correlated with the residual difference.

3. The equipment positioning method as described in claim 2, characterized in that, The step of determining the non-line-of-sight error level of the ultra-wideband base station based on the difference between the signal strength and the residual includes: If the signal strength is less than a first preset strength threshold and the residual difference is greater than a first preset residual threshold, then the high non-line-of-sight error is determined as the degree of non-line-of-sight error of the ultra-wideband base station. If the signal strength is less than the second preset strength threshold, or the residual difference is greater than the second preset residual threshold, then the non-line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station, wherein the second preset strength threshold is greater than or equal to the first preset strength threshold, and the second preset residual threshold is less than or equal to the first preset residual threshold. If the signal strength is greater than or equal to the second preset strength threshold and less than the third preset strength threshold, or the residual difference is less than or equal to the second preset residual threshold, then the normal line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station, wherein the third preset strength threshold is greater than the second preset strength threshold. If the signal strength is greater than or equal to the third preset strength threshold, then the low non-line-of-sight error is determined as the non-line-of-sight error level of the ultra-wideband base station.

4. The equipment positioning method as described in claim 3, characterized in that, The step of dynamically adjusting the ultra-wideband measurement noise covariance based on the non-line-of-sight error level of the ultra-wideband base station includes: Obtain the pre-set baseline measurement noise covariance; If the non-line-of-sight error of the ultra-wideband base station is high non-line-of-sight error, then the reference measurement noise covariance is multiplied by the first preset amplification factor to obtain the adjusted ultra-wideband measurement noise covariance. If the non-line-of-sight error of the ultra-wideband base station is medium non-line-of-sight error, then the reference measurement noise covariance is multiplied by the second preset amplification factor to obtain the adjusted ultra-wideband measurement noise covariance, wherein the second preset amplification factor is smaller than the first preset amplification factor. If the non-line-of-sight error level of the ultra-wideband base station is low non-line-of-sight error, then the reference measurement noise covariance is multiplied by a preset reduction factor to obtain the adjusted ultra-wideband measurement noise covariance. If the non-line-of-sight error of the ultra-wideband base station is a normal non-line-of-sight error, then the reference measurement noise covariance is determined as the adjusted ultra-wideband measurement noise covariance.

5. The equipment positioning method as described in claim 1, characterized in that, After the steps of inputting the first measurement data and the second measurement data into a Kalman filter and fusing the two types of measurement data using the observation noise covariance matrix to obtain the positioning result of the device, the method further includes: In response to a high-precision device positioning request, the pre-integration residual of the IMU in the visual inertial odometry, the first measurement residual of the visual inertial odometry, and the second measurement residual of the ultra-wideband base station are obtained. Using the pre-integration residual, the first measurement residual, and the second measurement residual as constraints, the positioning result is optimized using a factor graph algorithm to generate an optimized positioning result.

6. The equipment positioning method as described in claim 5, characterized in that, Before the step of acquiring IMU pre-integration data in response to a high-precision device positioning request, the method further includes: The high-precision device positioning request is triggered once every N keyframes; or... If the non-line-of-sight error of the ultra-wideband base station is detected to be greater than a preset threshold, a high-precision device positioning request is triggered. The keyframe is an image frame selected from the image sequence during the operation of the visual inertial odometry according to a preset selection strategy. The selection strategy is: the visual sensor attitude change is greater than a preset pose change threshold, or the visual feature change significance of the image frame is greater than a preset significance threshold.

7. The equipment positioning method as described in claim 1, characterized in that, The steps of acquiring the first measurement data of the ultra-wideband base station and the second measurement data of the visual inertial odometry include: Acquire second measurement data from the visual inertial odometry and the current scene image captured by the visual sensor in the visual inertial odometry, wherein the second measurement data is the current pose of the visual sensor; Based on the current pose and the historical pose measured by the visual inertial odometry, the pose change of the visual sensor is calculated. Based on the current scene image and the historical scene images acquired by the visual sensor, the significance of visual feature changes is calculated. If the change in pose is greater than a preset pose change threshold, or the significance of the change in visual features is greater than a preset significance threshold, then the first measurement data of the ultra-wideband base station is obtained. If the pose change is less than or equal to a preset pose change threshold, and the image change degree is less than or equal to a preset degree threshold, then the device is located using an optical flow tracking algorithm.

8. The equipment positioning method according to any one of claims 1 to 7, characterized in that, The positioning result is the device pose. After the steps of inputting the first measurement data and the second measurement data into a Kalman filter and fusing the two types of measurement data using the observation noise covariance matrix to obtain the device positioning result, the method further includes: SLAM maps can be built or updated based on the device pose.

9. A wearable device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the device positioning method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a device control program, which, when executed by a processor, implements the steps of the device positioning method as described in any one of claims 1 to 8.

Images