Camera-assisted UWB NLOS identification and positioning method

Abstract

The invention relates to a UWB NLOS identification and positioning method based on camera assistance, and the method comprises the steps: arranging a plurality of UWB base stations in a positioning region, and enabling the UWB base stations to be provided with visual identifications; collecting an image of the surrounding environment of the UWB base station, identifying a visual identifier corresponding to each UWB base station, and judging whether the UWB base station is in a line-of-sight LOS state or a non-line-of-sight NLOS state according to an identification result; and optimizing the link weight according to the state of the UWB base station, and carrying out position calculation to obtain a final position estimation coordinate of the UWB tag. The method provided by the invention has relatively strong robustness and higher positioning precision.

Description

Technical Field

[0001] This invention relates to the field of indoor positioning technology, and in particular to a camera-assisted UWB NLOS identification and positioning method. Background Technology

[0002] With the continuous development of positioning technology, the demand for high-precision positioning continues to grow. People's requirements for positioning services are no longer limited to specific scenarios, but rather they hope to obtain stable and accurate positioning results in various complex environments. Among many positioning technologies, the Global Navigation Satellite System (GNSS), with its advantages of wide coverage and high accuracy, can basically meet the positioning needs of most outdoor scenarios. However, inside buildings or in complex, obstructed environments, GNSS signals are easily blocked, making reliable positioning difficult. Therefore, researching high-precision, low-cost positioning technologies suitable for indoor environments has become an important research direction in recent years.

[0003] In recent years, wireless sensor-based positioning methods, such as Wi-Fi, Bluetooth, RFID, and Ultra-Wideband (UWB), have received widespread attention. Among them, UWB, with its high accuracy, low power consumption, and robustness to multipath interference, is considered one of the most promising technologies for positioning robots, drones, and smart terminals. A typical UWB positioning network consists of multiple fixed anchor points and one or more mobile tags, which communicate point-to-point to achieve ranging. Under line-of-sight (LOS) conditions, UWB systems can achieve centimeter-level ranging accuracy. However, when signal propagation is affected by obstacles, reflection, or diffraction, i.e., under non-line-of-sight (NLOS) conditions, the ranging error increases significantly, seriously affecting positioning accuracy and system stability. To address this issue, researchers have proposed various UWB NLOS signal recognition and suppression methods. Currently, the mainstream methods can be divided into two main categories: one is recognition methods based on ranging data characteristics. Another approach is a deep learning-based recognition method based on Channel Impulse Response (CIR). While these methods have made significant progress both theoretically and experimentally, they still face several challenges in practical applications. On the one hand, recognition based on ranging features relies on experience and statistical regularities, making the model prone to failure in complex environments.

[0004] In summary, existing NLOS identification methods still have shortcomings in terms of environmental adaptability, algorithm robustness, and engineering feasibility, making it difficult to achieve stable and reliable positioning performance in complex indoor scenes. Therefore, a simpler and more direct method is needed to identify UWB NLOS signals and reduce the impact of NLOS signals on the final positioning results. Summary of the Invention

[0005] The purpose of this invention is to provide a camera-assisted UWB NLOS identification and localization method. This method directly determines the link visibility status by detecting the AprilTag identifier of the base station through a multi-view camera, and constructs a comprehensive weight model by fusing visual judgment probability and CIR channel quality. Under the TDOA (Time Difference of Arrival) localization framework, adaptive weight optimization and localization of NLOS signals are performed.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A camera-assisted UWB NLOS identification and localization method includes:

[0008] Several UWB base stations are deployed in the positioning area, wherein the UWB base stations are equipped with visual identifiers;

[0009] Collect images of the surrounding environment of the UWB base station, identify the visual identifiers corresponding to each UWB base station, and determine whether the UWB base station is in line-of-sight (LOS) or non-line-of-sight (NLOS) state based on the identification results.

[0010] The link weights are optimized based on the status of the UWB base station, and the location is calculated to obtain the final estimated coordinates of the UWB tag's location.

[0011] Optionally, identify the visual identifiers corresponding to each UWB base station, and determine whether the UWB base station is in line-of-sight (LOS) or non-line-of-sight (NLOS) state based on the identification results, including:

[0012] The environmental image is preprocessed by grayscale conversion and Gaussian blurring, and the Canny operator is used to extract the image edge information;

[0013] Candidate label regions are selected by quadrilateral contour detection. Binarization encoding matching and ID decoding are performed within the candidate regions. If the visual identifier of the UWB base station is correctly identified, it is marked as line-of-sight (LOS) state; otherwise, it is marked as non-line-of-sight (NLOS) state.

[0014] Optionally, optimizing the link weight based on the state of the UWB base station includes:

[0015] Calculate the channel impulse response (CIR) signal weights of the UWB base station, and calculate the camera-assisted decision weights based on the state of the UWB base station;

[0016] The optimized weights are obtained by multiplying the channel impulse response (CIR) signal weights and the auxiliary decision weights.

[0017] Optionally, the camera-assisted determination weight for:

[0018] ;

[0019] Where i is the index of the base station. This represents the probability of the camera correctly detecting the LOS link. This represents the false detection probability of the NLOS link.

[0020] Optionally, the probability of the camera correctly detecting the LOS link. for:

[0021] ;

[0022] False detection probability of NLOS link for:

[0023] .

[0024] Optionally, the location calculation to obtain the final estimated coordinates of the UWB tag includes: linearizing the hyperbolic equations of TDOA using the Taylor series expansion method, and combining it with the least squares method to find local solutions. The iteration is continued until the error meets the set threshold, at which point the iteration process is stopped, and the final estimated coordinates are obtained.

[0025] Optionally, using the least squares method to find local solutions includes:

[0026] ;

[0027] in, This is the position correction value. This is the weight matrix. Let covariance matrix be the variance matrix. For the final label position, and These are the correction values ​​for the x and y coordinates, respectively. For Jacobian matrices, To measure the residual vector, t is the epoch number, and the superscript T indicates transpose.

[0028] Optionally, acquiring environmental images around the UWB base station includes: installing a camera system consisting of four wide-angle cameras facing different directions on a mobile positioning platform, and acquiring environmental images through the camera system, wherein the mobile positioning platform is also equipped with UWB tags.

[0029] The beneficial effects of this invention are as follows: Addressing the problems of existing UWB positioning methods based on channel impulse response (NLOS) relying on large amounts of training data and struggling to generalize across scenarios, as well as the tendency for single CIR weight adjustments to misjudge and decrease positioning accuracy in complex occlusion environments, this invention proposes a camera-assisted UWB NLOS identification and positioning method. This method directly determines the link visibility status by detecting the base station's AprilTag identifier using a multi-view camera, integrates visual judgment probability and CIR channel quality to construct a comprehensive weight model, and performs adaptive weight optimization and positioning of NLOS signals within the TDOA positioning framework.

[0030] Compared to traditional UWB NLOS identification and localization methods, the camera-assisted UWB NLOS identification and localization method employed in this invention can effectively suppress localization errors caused by NLOS signals resulting from obstacle occlusion. It exhibits stronger robustness and higher localization accuracy. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is an example diagram illustrating false detections in an embodiment of the present invention;

[0033] Figure 2 This is a flowchart illustrating the AprilTag recognition process performed on each frame of image captured by the camera system according to an embodiment of the present invention.

[0034] Figure 3 This is an example diagram illustrating the identification process according to an embodiment of the present invention;

[0035] Figure 4 This is a flowchart illustrating a camera-assisted UWB NLOS identification and localization method according to an embodiment of the present invention;

[0036] Figure 5 This is a physical diagram of the camera system according to an embodiment of the present invention;

[0037] Figure 6 This is a floor plan of the experimental site according to an embodiment of the present invention;

[0038] Figure 7 This is a map of the experimental field in an embodiment of the present invention;

[0039] Figure 8 This is an error diagram at static test point 1 in an embodiment of the present invention, wherein (a) is the X-direction error at static test point 1, and (b) is the Y-direction error at static test point 1.

[0040] Figure 9 This is an error diagram at static test point 2 in an embodiment of the present invention, wherein (a) is the X-direction error at static test point 2, and (b) is the Y-direction error at static test point 2;

[0041] Figure 10 This is a comparison diagram of the dynamic trajectories in an embodiment of the present invention;

[0042] Figure 11 This is a dynamic test positioning error diagram according to an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] This embodiment proposes a camera-assisted UWB NLOS identification and localization method, including:

[0046] Several UWB base stations are deployed in the positioning area, and visual identifiers are installed on the UWB base stations.

[0047] Collect images of the surrounding environment of UWB base stations, identify the visual identifiers corresponding to each UWB base station, and determine whether the UWB base station is in line-of-sight (LOS) or non-line-of-sight (NLOS) state based on the identification results.

[0048] The link weights are optimized based on the status of the UWB base station, and the location is calculated to obtain the final estimated coordinates of the UWB tag's location.

[0049] Furthermore, the visual identifiers corresponding to each UWB base station are identified, and the UWB base station is determined to be in a line-of-sight (LOS) or non-line-of-sight (NLOS) state based on the identification results, including:

[0050] The environmental image is preprocessed by grayscale conversion and Gaussian blurring, and the Canny operator is used to extract the image edge information;

[0051] Candidate label regions are selected by quadrilateral contour detection. Binarization encoding matching and ID decoding are performed within the candidate regions. If the visual identifier of the UWB base station is correctly identified, it is marked as line-of-sight (LOS) state; otherwise, it is marked as non-line-of-sight (NLOS) state.

[0052] Specifically, the system first acquires information about the surrounding environment of the current carrier through a camera system. AprilTag recognition is then performed on each frame of image acquired by the camera system: grayscale conversion and Gaussian blur preprocessing are performed first, followed by edge extraction using the Canny operator. Candidate tag regions are then selected through quadrilateral contour detection. Within these candidate regions, binary encoding matching and ID decoding are performed. When the tag code matches a preset base station ID, the corresponding base station is temporarily considered to be in a visible state. The specific detection process is as follows: Figure 2 As shown.

[0053] Define a binary observation variable. If the AprilTag of the i-th base station is correctly identified, it is marked as LOS state; otherwise, it is temporarily marked as NLOS state.

[0054] ;

[0055] Recognition examples Figure 3 As shown. Figure 3 As shown, in a UWB four-base station system, the camera system detects base stations 0, 1, and 3 in the environment. Since base station 3 is not present, its link status is temporarily marked as NLOS. The NLOS signal link is confirmed by combining the received UWB signals. Specifically, if the received CIR information contains base station information marked as NLOS by the camera system, then the status of this base station is determined to be NLOS.

[0056] Furthermore, optimizing link weights based on the status of UWB base stations includes:

[0057] Calculate the channel impulse response (CIR) signal weights of the UWB base station, and calculate the camera-assisted decision weights based on the state of the UWB base station.

[0058] The optimized weights are obtained by multiplying the channel impulse response (CIR) signal weights and the auxiliary decision weights.

[0059] Specifically, after determining the UWB base station status based on the camera system and UWB information, the link weights are re-optimized by combining the information from both. Specifically, the CIR signal weight of base station i is first calculated using a formula. :

[0060] ;

[0061] in, The difference between the first path power and the minimum power for the base station to receive tag messages. This is the difference between the total received power of the base station and the minimum received power of the tag. The difference between the noise power and the minimum noise power. The first path power for the base station to receive tag messages. The total received power of the tag messages received by the base station. Based on this, camera-assisted decision weights are introduced:

[0062] ;

[0063] Where i is the index of the base station. This represents the probability of the camera correctly detecting the LOS link. This represents the false detection probability of the NLOS link.

[0064] To quantify the uncertainty of visual inspection, the probability of correct detection of the LOS link by the camera is statistically obtained during the offline calibration stage. False detection probability of NLOS link During online detection, Bayesian inference transforms the observation results into link state probabilities: when the camera does not detect the i-th base station tag, the probability that the link is NLOS is... When a tag is detected, the probability that the link is a LOS is: .

[0065] ;

[0066] ;

[0067] Examples of false positives are as follows Figure 1 .

[0068] The final overall weight, or optimized weight, is:

[0069] .

[0070] Further, position calculation is performed to obtain the final estimated coordinates of the UWB tag. This includes: linearizing the hyperbolic equations of TDOA using the Taylor series expansion method, and finding local solutions using the least squares method. The process is iterated until the error meets the set threshold, at which point the iteration process stops, and the final estimated coordinates are obtained.

[0071] Specifically, based on the Taylor series expansion method, the hyperbolic equations of TDOA are linearized, and local solutions are obtained using the least squares method. Through continuous iteration, the positioning accuracy of the estimated tag node position is improved until the error meets the set threshold, at which point the iteration process stops, and the final estimated coordinates are obtained. Ignoring second-order and higher-order components in the Taylor expansion, we get:

[0072] ;

[0073] in:

[0074] ;

[0075] ;

[0076] ;

[0077] in The initial values ​​are substituted for the Taylor expansion. This indicates the distance from the base station to the tag node. This represents the distance difference between base station i and base station 1 and the tag node. Represented as base station coordinates. and These are the x and y coordinate correction values, respectively. The final position calculation formula is:

[0078] ;

[0079] in, This is the position correction value. The weight matrix is ​​composed of the combined weights of the signals from the aforementioned base stations. Let covariance matrix be the variance matrix. For the final label position, and Correction values ​​for x and y coordinates respectively. For Jacobian matrices, To measure the residual vector, t is the epoch number, and the superscript T indicates transpose.

[0080] Furthermore, the acquisition of environmental images around the UWB base station includes: installing a camera system consisting of four wide-angle cameras facing different directions on the mobile positioning platform, and acquiring environmental images through the camera system. The mobile positioning platform is also equipped with UWB tags.

[0081] The effectiveness of this method is illustrated below with experimental results:

[0082] like Figure 4As shown, a camera-assisted UWB NLOS recognition and localization method consists of two subsystems: a UWB subsystem and a camera subsystem. The two subsystems are synchronized in time before system startup. After system startup, the camera captures environmental information around the rover and detects and identifies the corresponding AprilTag. Simultaneously, the system receives CIR data from the UWB subsystem. If CIR data is received and the corresponding AprilTag for a base station is detected, it is determined to be a LOS link. If CIR data is received but the camera subsystem does not detect the AprilTag number corresponding to the base station, the link state of that base station is determined to be NLOS. Subsequently, the camera recognition weight for the current camera-identified link state is determined based on the link determination result. Finally, the weight of the link participating in the localization calculation is determined based on the camera recognition weight and CIR data, and the localization calculation is performed. The UWB subsystem consists of multiple base stations with known locations and AprilTags for the rover. The camera subsystem is as follows... Figure 5 As shown, four wide-angle cameras are installed on the rover station to acquire information about the surrounding environment.

[0083] This embodiment was tested in the Beidou+ Intelligent Navigation and Positioning Laboratory inside the National Key Laboratory Building of Henan Polytechnic University. Figure 6 and Figure 7 These are the experimental plan view and the experimental scene view, respectively.

[0084] The experimenters operated the experimental equipment to collect data along a preset path within the experimental site. The Taylor method, the Chan method, and the method proposed in this embodiment were compared. The three methods... Figure 8-11 They are referred to as Taylor UWB, Chan UWB, and CA UWB, respectively.

[0085] (1) Static testing:

[0086] like Figure 8 (a)-(b) and Figure 9As shown in (a)-(b), the effects of NLOS on the three algorithms in the NLOS environment are illustrated. It can be seen that the method proposed in this embodiment has a significantly lower error than the other two algorithms in most cases. Tables 1 and 2 show the root mean square error (RMSE) and mean error (ME) at two static test points. From Tables 1-2, it can be seen that the RMSE values ​​of this method in the X direction at the two static test points are approximately 0.03 and 0.02, respectively, which are 81.25% and 62.5% higher than the other two algorithms at static test point 1, and 33.33% and 88.24% higher than the other two algorithms. The RMSE values ​​in the Y direction at the two static test points are approximately 0.02 and 0.02, respectively, which are 94.74% and 93.94% higher than the other two algorithms at static test point 1, and 77.78% and 98.35% higher than the other two algorithms.

[0087] Table 1

[0088]

[0089] Table 2

[0090]

[0091] (2) Dynamic testing:

[0092] from Figure 10 As can be seen, the trajectory based on this method closely follows the preset trajectory and is more complete and orderly, while the Taylor and Chan methods exhibit excessively large solution errors in the boxed areas of the graph compared to this method. Data with excessively large errors were not statistically analyzed in this experiment; therefore, from... Figure 10 This shows that the trajectories obtained using both Taylor's and Chan's methods are incomplete in this part. Furthermore, from... Figure 10 As can be seen, the Chan method's solution results are quite chaotic, fluctuating wildly, and the trajectory data is almost unusable. It is clearly evident that this method effectively suppresses trajectory drift compared to the other two methods. Figure 11 The diagram shows the positioning error of the three methods. Figure 11 The positioning error graphs of the three methods are shown, displaying statistical results for errors below 20 meters. The graphs demonstrate that the proposed method is more stable than the Taylor and Chan methods. Furthermore, the number of error jumps in the proposed method is significantly fewer than the other two methods, and the positioning error of the proposed method is lower than that of the other two methods for most of the time. This dynamic experiment further demonstrates the accuracy and reliability of the proposed method.

[0093] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A camera-assisted UWB NLOS recognition and localization method, characterized in that, include: Several UWB base stations are deployed in the positioning area, wherein the UWB base stations are equipped with visual identifiers; Collect images of the surrounding environment of the UWB base station, identify the visual identifiers corresponding to each UWB base station, and determine whether the UWB base station is in line-of-sight (LOS) or non-line-of-sight (NLOS) state based on the identification results. The link weights are optimized based on the status of the UWB base station, and the location is calculated to obtain the final estimated coordinates of the UWB tag's location.

2. The camera-assisted UWB NLOS identification and localization method according to claim 1, characterized in that, Identify the visual identifiers corresponding to each UWB base station, and determine whether the UWB base station is in line-of-sight (LOS) or non-line-of-sight (NLOS) state based on the identification results, including: The environmental image is preprocessed by grayscale conversion and Gaussian blurring, and the Canny operator is used to extract the image edge information; Candidate label regions are selected by quadrilateral contour detection. Binarization encoding matching and ID decoding are performed within the candidate regions. If the visual identifier of the UWB base station is correctly identified, it is marked as line-of-sight (LOS) state; otherwise, it is marked as non-line-of-sight (NLOS) state.

3. The camera-assisted UWB NLOS identification and localization method according to claim 1, characterized in that, Optimizing link weights based on the status of the UWB base stations includes: Calculate the channel impulse response (CIR) signal weights of the UWB base station, and calculate the camera-assisted decision weights based on the state of the UWB base station; The optimized weights are obtained by multiplying the channel impulse response (CIR) signal weights and the auxiliary decision weights.

4. The camera-assisted UWB NLOS identification and localization method according to claim 3, characterized in that, Camera-assisted decision weight for: ； Where i is the index of the base station. This represents the probability of the camera correctly detecting the LOS link. This represents the false detection probability of the NLOS link.

5. The camera-assisted UWB NLOS identification and localization method according to claim 4, characterized in that, Probability of correct detection of LOS link by the camera for: ； False detection probability of NLOS link for: 。 6. The camera-assisted UWB NLOS identification and localization method according to claim 1, characterized in that, The process of calculating the final estimated coordinates of the UWB tag's location includes: linearizing the hyperbolic equations of TDOA using the Taylor series expansion method, and finding local solutions using the least squares method. This process is iterated until the error meets a set threshold, at which point the iteration process stops, and the final estimated coordinates are obtained.

7. The camera-assisted UWB NLOS identification and localization method according to claim 1, characterized in that, Finding local solutions using the least squares method includes: ； in, This is the position correction value. This is the weight matrix. Let covariance matrix be the variance matrix. For the final label position, and These are the correction values ​​for the x and y coordinates, respectively. For Jacobian matrices, To measure the residual vector, t is the epoch number, and the superscript T indicates transpose.

8. The camera-assisted UWB NLOS identification and localization method according to claim 1, characterized in that, The process of acquiring environmental images around the UWB base station includes: installing a camera system consisting of four wide-angle cameras facing different directions on a mobile positioning platform, and acquiring environmental images through the camera system. The mobile positioning platform is also equipped with UWB tags.

Images