Indoor BLE equipment three-dimensional positioning and visualization method

Abstract

The invention relates to the field of augmented reality, in particular to a three-dimensional positioning and visualization method for indoor BLE equipment. The invention discloses a three-dimensional positioning and visualization method for indoor BLE (Bluetooth Low Energy) equipment. The method comprises the following steps: step S10, determining Bluetooth equipment global coordinates of Bluetooth equipment in a base station space coordinate system based on position information of a signal base station; s20, based on the position information, obtaining a visual equipment global coordinate of the visual equipment in a base station space coordinate system; and S30, according to the visual global coordinate and the Bluetooth global coordinate, obtaining a view field space coordinate of the Bluetooth device in a view field of the visual device. According to the invention, the global coordinate of the Bluetooth device in the base station space coordinate system and the global coordinate of the visual device in the base station space coordinate system are obtained; the global coordinate of the Bluetooth is converted into the space coordinate of the view field by utilizing the global coordinate of the visual equipment and the global coordinate of the Bluetooth equipment, so that the decoupling of the positioning end and the visual end is realized, and the deployment and the use of the equipment are more flexible.

Description

Technical Field

[0001] This invention relates to the field of augmented reality, and more particularly to a method for three-dimensional positioning and visualization of indoor BLE devices. Background Technology

[0002] With the development of the Internet of Things (IoT) and mobile computing, the demand for indoor positioning is growing rapidly. Compared with solutions such as UWB, Wi-Fi, and RFID, Bluetooth has advantages such as low power consumption, low cost, mature ecosystem, and high terminal penetration, making it one of the important technical routes for indoor positioning. Current Bluetooth positioning mainly relies on RSSI fingerprinting or simple polygonal positioning, and the display format is usually limited to 2D planar heatmaps or top-view trajectories. These methods struggle to express spatial information such as three-dimensional height differences and occlusion relationships. Users still need additional information to find targets in complex three-dimensional environments, resulting in a heavy interactive burden. Based on these limitations, the industry urgently needs a new positioning method that can output robust three-dimensional positions and overlay device status and spatial location in real time from the user's current perspective; it should also support multi-target matching, occlusion detection, and out-of-view cues to reduce the user's cognitive burden and improve retrieval and inspection efficiency. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method for three-dimensional positioning and visualization of indoor BLE devices.

[0004] The technical solution adopted by this invention to solve its technical problem is: a method for three-dimensional positioning and visualization of indoor BLE devices, comprising the following steps: Step S10: Determine the global coordinates of the Bluetooth device in the base station's spatial coordinate system based on the location information of the signal base station; Step S20: Based on the location information, obtain the global coordinates of the visualization device in the base station spatial coordinate system; Step S30: Based on the visualized global coordinates and the Bluetooth global coordinates, obtain the field-of-view spatial coordinates of the Bluetooth device in the field of view of the visualized device.

[0005] Optionally, step S30 includes: Establish a coordinate system for the visualization device based on its field of view. Based on the visualized base station coordinates and the Bluetooth device's global coordinates, the field-of-view spatial coordinates of the Bluetooth device in the visualized device coordinate system are obtained.

[0006] Optionally, step S30 further includes: The field-of-view spatial coordinates are updated according to the orientation of the visualization device.

[0007] Optionally, updating the field-of-view spatial coordinates according to the pose of the visualization device includes: Acquire inertial sensor data from the visualization device; The field of view spatial coordinates are updated based on the inertial sensor data.

[0008] Optionally, it includes: The pixel coordinates of the field of view spatial coordinates are calculated based on the visualization parameters of the visualization device; Determine whether the pixel coordinates are within the field of view of the visualization device; if so, visualize the Bluetooth device based on the field of view spatial coordinates and the field of view information of the visualization device.

[0009] Optionally, the step of visualizing the Bluetooth device based on the field-of-view spatial coordinates and the field-of-view image information of the visualization device includes: Acquire the field of view image information; Based on the field of view image information, obtain the target mask and the corresponding pixel-level depth map; Based on the pixel-level depth map, the average depth of the target mask is obtained; The relative distance between the visualization device and the Bluetooth device is obtained based on wireless positioning; Calculate the distance confidence level based on the average depth and the relative distance; The Bluetooth device is visualized based on the distance confidence level.

[0010] Optionally, obtaining the target mask and the corresponding pixel-level depth map based on the field-of-view image information includes: The field-of-view image information is processed using monocular depth technology to obtain the target mask and the pixel-level depth map.

[0011] Optionally, calculating the distance confidence score based on the average depth and the relative distance includes: Calculate the absolute value of the difference between the average depth and the relative distance to obtain the absolute depth difference; The distance confidence level is obtained by calculating the absolute depth difference.

[0012] Optionally, visualizing the Bluetooth device based on the distance confidence level includes: The image confidence score of the field of view image information is obtained; Multiply the distance confidence score by the image confidence score to obtain the fusion confidence score of the target mask; The Bluetooth device corresponding to the highest fusion confidence score within the field of view is visualized.

[0013] Optionally, obtaining the average depth based on the target mask and the pixel-level depth map includes: The average depth is obtained by matching the pixel-level depth map with the target mask.

[0014] The implementation of this invention has the following beneficial effects: This invention obtains the global coordinates of the Bluetooth device in the base station spatial coordinate system and the global coordinates of the visualization device in the base station spatial coordinate system. Using the global coordinates of the visualization device and the Bluetooth device, the Bluetooth global coordinates are converted into field-of-view spatial coordinates, thereby decoupling the positioning end and the visualization end, making the deployment and use of the device more flexible. Attached Figure Description

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 A flowchart of a method for 3D positioning and visualization of indoor BLE devices in one embodiment; Figure 2 This is a schematic diagram illustrating the location determination of dual base stations in one embodiment; Figure 3 This is a schematic diagram of the field of view spatial coordinate transformation in one embodiment. Detailed Implementation

[0016] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0018] This invention provides a method for three-dimensional positioning and visualization of indoor BLE devices, such as... Figure 1 As shown, it includes the following steps: Step S10: Determine the global coordinates of the Bluetooth device in the base station spatial coordinate system based on the location information of the signal base station.

[0019] Based on the known location information of signal base stations, and by analyzing the signal transmission characteristics between the Bluetooth device and multiple base stations, the global coordinates of the Bluetooth device in the spatial coordinate system constructed by the base stations can be calculated. Utilizing the distances between base stations to calculate the global coordinates of the Bluetooth device ensures the accuracy and reliability of the positioning results, thereby effectively determining the specific location of the Bluetooth device in space.

[0020] In some scenarios, two fixed signal base stations are deployed indoors, each equipped with a vertical antenna array that is orthogonal to each other. The AOA components of the arrays aligned with the X and Y axes are then measured. , And based on this, the elevation angle is obtained. With azimuth Place the secondary base station on the X-axis of the primary base station with a spacing of d, and calculate the three-dimensional coordinates of the target in the spatial coordinate system of the primary base station based on geometric relationships. .

[0021] Step S20: Based on location information, obtain the global coordinates of the visualization device in the base station spatial coordinate system.

[0022] Based on the location information of the signal base station, the accurate global coordinates of the visualization device in the base station spatial coordinate system are calculated through the positioning system and coordinate transformation algorithm, thereby realizing the real-time accurate positioning and visualization of the device's spatial location.

[0023] Step S30: Based on the visualized global coordinates and the Bluetooth global coordinates, obtain the field-of-view spatial coordinates of the Bluetooth device in the field of view of the visualized device.

[0024] Based on the correspondence between the visualization global coordinate system and the Bluetooth global coordinate system, the field-of-view spatial coordinates of the Bluetooth device within the field of view of the visualization device are obtained through a coordinate transformation algorithm.

[0025] This invention decouples the positioning and visualization ends by converting Bluetooth global coordinates into field-of-view spatial coordinates, making the deployment and use of the device more flexible.

[0026] Furthermore, the present invention allows for the one-time deployment of a unified high-precision positioning signal base station, which can support the concurrent use of a massive number of heterogeneous terminals (such as AR glasses, mobile phones and other visual devices); each terminal obtains personalized spatial guidance based on its own pose, realizing "one source, multiple terminals, and independent perspective".

[0027] In some scenarios, a dual-base station approach is used to determine the target location. For example... Figure 2As shown, the signal base stations include a first signal base station 01 and a second signal base station 02. The first signal base station 01 is the main base station, and a spatial rectangular coordinate system is established with the main base station as the main base station. The second signal base station 02 is placed as the secondary base station on the X-axis of the main base station, and the distance between the second and the main base station is d. According to geometric relationships, the spatial coordinates of the Bluetooth device 03 to be located and the signal angles measured by the two base stations have the following relationship: ,in The signal azimuth angle is measured by the secondary base station. Based on the above relationship, the Bluetooth global coordinates of the Bluetooth device 03 to be located in the spatial coordinate system of the primary base station can be calculated as follows: , , .

[0028] By using dual anchor points and three-dimensional geometric solutions, the azimuth / elevation estimation error is significantly reduced.

[0029] In some executable embodiments, step S30 includes: Establish the visualization device coordinate system based on the field of view of the visualization device.

[0030] To accurately describe and analyze the image information captured by visualization devices, it is necessary to first establish a coordinate system corresponding to the device's field of view. Based on the device's field of view and viewing angle, the reference direction, origin, and coordinate axis distribution of the coordinate system are determined to ensure the accuracy and consistency of subsequent data processing and spatial positioning.

[0031] Based on the visualized base station coordinates and the Bluetooth device's global coordinates, the field-of-view spatial coordinates of the Bluetooth device in the visualized device coordinate system are obtained.

[0032] Based on existing visualized base station coordinate information and Bluetooth device location data in the global coordinate system, spatial transformation and coordinate mapping algorithms are used to calculate and obtain the specific field-of-view spatial coordinates of the Bluetooth device in the visualized device coordinate system, thereby achieving accurate visualized positioning and spatial relationship presentation of the device location.

[0033] Taking a smartphone as an example for visualization, its coordinate system is defined as follows: the phone's center of mass is the origin. When the phone is facing the user, the direction extending to the right from the center of mass is the positive X-axis, the direction extending towards the top of the phone is the positive Y-axis, and the direction pointing towards the user from the center of mass is the positive Z-axis. A detailed illustration follows. Figure 3 As shown, the signal base stations include a first signal base station 01 and a second signal base station 02. The global coordinates of the positioning visualization device 04 and the Bluetooth device 03, located in the base station spatial coordinate system of the first signal base station 01 and the second signal base station 02, are respectively... and The field-of-view coordinates of the Bluetooth device relative to the visualization device can be obtained through affine transformation. The affine transformation matrix A from the base station spatial coordinate system to the visualization device coordinate system is: The field-of-view coordinates of the Bluetooth device relative to the visualization device are: Based on the field-of-view coordinates, the two-dimensional coordinates of the Bluetooth device on the visual device can be calculated: In some executable embodiments, step S30 further includes: Update the field-of-view spatial coordinates based on the orientation of the visualization device.

[0034] Based on the real-time changes in the orientation of the visualization device, the relationship between the visualization device's coordinate system and the base station's spatial coordinate system also changes with the orientation. The coordinate positions within the field of view are dynamically updated and recalculated following the orientation, ensuring the real-time nature, accuracy, and naturalness of spatial guidance.

[0035] In some executable embodiments, updating the field-of-view spatial coordinates according to the pose of the visualization device includes: Acquire inertial sensor data from visualization devices.

[0036] Update the field of view spatial coordinates based on inertial sensor data.

[0037] The system acquires and reads raw data collected by the inertial sensors built into the visualization device, including key parameters such as acceleration, angular velocity, and direction. Subsequently, based on this real-time acquired inertial sensor data, the system dynamically and precisely adjusts and updates the position of the observation point and the viewing angle in the current field of view using a corresponding spatial coordinate transformation algorithm.

[0038] In some executable embodiments, the following are included: The pixel coordinates of the field of view spatial coordinates are calculated based on the visualization parameters of the visualization device.

[0039] Specifically, based on the horizontal / vertical field of view of the visualization device With resolution Calculate the pixel coordinates by analyzing the field of view spatial coordinates. .

[0040] Determine whether the pixel coordinates are within the field of view of the visualization device; if so, visualize the Bluetooth device based on the field of view spatial coordinates and the field of view information of the visualization device.

[0041] If pixel coordinates If the target crosses the boundary, it is determined that the target is not within the field of view; when the attitude of the visualization device changes, the field of view spatial coordinates are updated in real time based on inertial sensor data to maintain visualization consistency.

[0042] In some executable embodiments, the Bluetooth device is visualized based on the field-of-view spatial coordinates and the field-of-view image information of the visualization device, including: Acquire field of view image information.

[0043] Understandably, this involves acquiring image data information from within the field of view.

[0044] Based on the field of view image information, the target mask and the corresponding pixel-level depth map are obtained.

[0045] Based on the field-of-view image information, image processing and computer vision techniques are used to obtain a mask of the target object (which may be a Bluetooth device within the field of view) and its corresponding pixel-level depth map. This process involves accurate segmentation and recognition of the target region in the image to generate a high-quality target mask. At the same time, a depth estimation algorithm is used to assign a corresponding depth value to each pixel, ultimately obtaining pixel-level depth information that is strictly aligned with the mask.

[0046] The average depth of the target mask is obtained based on the pixel-level depth map.

[0047] Furthermore, the depth value of each pixel in the pixel-level depth map is precisely extracted, and then the target region is accurately identified and separated using image segmentation technology. Based on this, for the acquired target mask, the depths of all pixels belonging to that region are summed, and their arithmetic mean is calculated to obtain the average depth value of the target mask.

[0048] The relative distance between the visualized device and the Bluetooth device is obtained from wireless positioning.

[0049] Specifically, wireless positioning refers to the distance value between a visual device and a Bluetooth device calculated using a base station spatial coordinate system or a field-of-view spatial coordinate system.

[0050] In some scenarios, dual-signal base stations connect to a visualization device A and a Bluetooth device B. The primary base station obtains the angles of the two devices relative to it via wireless positioning (the secondary base station does the same). The primary and secondary base stations then broadcast these positioning results (at this stage, they are only angles, not coordinates). The visualization device can then acquire this broadcast information. The visualization device further converts this angle information into coordinate information (first calculating the 3D coordinates of A and B in the base station's spatial coordinate system, and then converting them to the visualization device's coordinate system (the field-of-view spatial coordinate system)). This allows them to obtain the relative distance between A and B (the distance from the Bluetooth device to the visualization device).

[0051] Understandably, visualization devices and signal base stations can also connect via Bluetooth.

[0052] Data obtained through wireless positioning technology can clearly and intuitively display the relative spatial distance between the visualized device and the Bluetooth device.

[0053] Distance confidence is calculated based on average depth and relative distance.

[0054] By calculating distance confidence using average depth and relative distance, the system can proactively identify and filter out unreliable distance estimates caused by noise, occlusion, or calculation errors, thereby significantly improving the accuracy and robustness of localization, mapping, or 3D recognition. The system can dynamically adjust data fusion weights or trigger re-detection based on the confidence level, achieving adaptive optimization.

[0055] The Bluetooth devices are visualized based on distance confidence.

[0056] Visualizing Bluetooth devices based on distance confidence significantly improves the efficiency and reliability of human-computer interaction. This reduces the cognitive load on users dealing with abstract location data, enabling them to quickly identify highly reliable targets and understand the range of uncertainty. It also enhances the system's transparency and adaptability. Users can optimize their behavior based on visual feedback (such as adjusting their position to improve signal strength), while the system can prioritize allocating attention resources to the most reliable spatial information. Ultimately, this leads to more accurate and efficient human-computer collaborative operations in scenarios such as warehouse item retrieval and equipment inspection.

[0057] In some executable embodiments, based on the field-of-view image information, the target mask and the corresponding pixel-level depth map are obtained, including: By using monocular depth technology to process the field-of-view image information, a target mask and the corresponding pixel-level depth map are obtained.

[0058] A lightweight single-stage detection network is used to process the field-of-view image information, creating a mask region for each device in the image to obtain a target mask. Each target mask corresponds to one device. The Monodepth2 algorithm is then used to process the target masks, resulting in a pixel-level depth map output by the monocular depth estimation algorithm.

[0059] In some executable embodiments, calculating distance confidence based on average depth and relative distance includes: Calculate the absolute value of the difference between the average depth and the relative distance to obtain the absolute depth difference.

[0060] The absolute depth difference is obtained by calculating the difference between the average depth and the relative distance, and taking the absolute value of this difference. This eliminates the influence of possible negative values, ensuring that the final absolute depth difference is always a non-negative value, accurately reflecting the degree of difference between the two.

[0061] The distance confidence level is obtained by calculating the absolute depth difference.

[0062] The absolute depth difference of the target mask is precisely calculated to obtain the confidence score of the target mask.

[0063] In some executable embodiments, the Bluetooth device is visualized based on distance confidence, including: Image confidence score for acquiring field-of-view image information.

[0064] Multiply the distance confidence score by the image confidence score to obtain the fusion confidence score of the target mask.

[0065] The Bluetooth device corresponding to the highest fusion confidence score within the field of view is visualized.

[0066] Understandably, the fusion confidence of all mask regions within the field of view is divided by clustering (K-means with absolute depth), and selection is made within the cluster. The highest mask area is used as the corresponding Bluetooth device and is overlaid with visualization.

[0067] In some executable embodiments, the average depth is obtained based on the target mask and the pixel-level depth map, including: The average depth is obtained by matching the pixel-level depth map with the target mask.

[0068] A lightweight single-stage detection network (YOLOv8) is used to generate the target mask and detection confidence. The Monodepth2 monocular depth estimation outputs a pixel-level depth map, and the average depth of the mask region is calculated. .Will Relative distance obtained from wireless positioning Perform a consistency metric and obtain Based on this, the distance confidence level is calculated. and with Multiply to obtain the fusion confidence level Candidate sets are divided by clustering (K-means with absolute depth), and selection is made within each cluster. The highest target is the corresponding Bluetooth device and overlaid with visualization.

[0069] In some feasible embodiments, Bluetooth devices in connectionless mode transmit device information via broadcast packets. Each broadcast packet contains a unique identifier for distinguishing the device: the MAC address. Additionally, the broadcast packet may carry other supplementary information (such as device status, data type, counters, etc.) to provide the base station with more information for identification and processing. When listening to these broadcast packets, the base station can identify all broadcasts from the same device based on the MAC address and combine this with the broadcast timestamp and AOA (Aspect-Oriented Analysis) to further track and locate the device. After dual-base station positioning is completed, each base station broadcasts its own positioning results.

[0070] Both the primary and secondary base stations broadcast their location results, which are then received by the visualization device for subsequent coordinate transformations. It's understandable that the location results from the signal base stations include information from both the Bluetooth device and the visualization device. Only the visualization device is configured to receive these results and perform further processing.

[0071] The above embodiments only illustrate preferred embodiments of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can freely combine the above technical features without departing from the concept of the present invention, and can also make several modifications and improvements, all of which fall within the protection scope of the present invention. Therefore, all equivalent transformations and modifications made with respect to the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A method for three-dimensional positioning and visualization of indoor BLE devices, characterized in that, Includes the following steps: Step S10: Determine the global coordinates of the Bluetooth device in the base station's spatial coordinate system based on the location information of the signal base station; Step S20: Based on the location information, obtain the global coordinates of the visualization device in the base station spatial coordinate system; Step S30: Based on the visualized global coordinates and the Bluetooth global coordinates, obtain the field-of-view spatial coordinates of the Bluetooth device in the field of view of the visualized device.

2. The three-dimensional positioning and visualization method according to claim 1, characterized in that, Step S30 includes: Establish a coordinate system for the visualization device based on its field of view. Based on the visualized base station coordinates and the Bluetooth device's global coordinates, the field-of-view spatial coordinates of the Bluetooth device in the visualized device coordinate system are obtained.

3. The three-dimensional positioning and visualization method according to claim 2, characterized in that, Step S30 further includes: The field-of-view spatial coordinates are updated according to the orientation of the visualization device.

4. The three-dimensional positioning and visualization method according to claim 3, characterized in that, Updating the field-of-view spatial coordinates according to the pose of the visualization device includes: Acquire inertial sensor data from the visualization device; The field of view spatial coordinates are updated based on the inertial sensor data.

5. The three-dimensional positioning and visualization method according to claim 1, characterized in that, include: The pixel coordinates of the field of view spatial coordinates are calculated based on the visualization parameters of the visualization device; Determine whether the pixel coordinates are within the field of view of the visualization device; If present, the Bluetooth device is visualized based on the field of view spatial coordinates and the field of view information of the visualization device.

6. The three-dimensional positioning and visualization method according to claim 5, characterized in that, The step of visualizing the Bluetooth device based on the field-of-view spatial coordinates and the field-of-view image information of the visualization device includes: Acquire the field of view image information; Based on the field of view image information, obtain the target mask and the corresponding pixel-level depth map; Based on the target mask and the pixel-level depth map, the average depth of the target mask is obtained; The relative distance between the visualization device and the Bluetooth device is obtained based on wireless positioning; Calculate the distance confidence level based on the average depth and the relative distance; The Bluetooth device is visualized based on the distance confidence level.

7. The three-dimensional positioning and visualization method according to claim 6, characterized in that, The step of obtaining the target mask and the corresponding pixel-level depth map based on the field-of-view image information includes: The field-of-view image information is processed using monocular depth technology to obtain the target mask and the pixel-level depth map.

8. The three-dimensional positioning and visualization method according to claim 6, characterized in that, The step of calculating the distance confidence score based on the average depth and the relative distance includes: Calculate the absolute value of the difference between the average depth and the relative distance to obtain the absolute depth difference; The distance confidence level is obtained by calculating the absolute depth difference.

9. The three-dimensional positioning and visualization method according to claim 8, characterized in that, The step of visualizing the Bluetooth device based on the distance confidence level includes: The image confidence score of the field of view image information is obtained; Multiply the distance confidence score by the image confidence score to obtain the fusion confidence score of the target mask; The Bluetooth device corresponding to the highest fusion confidence score within the field of view is visualized.

10. The three-dimensional positioning and visualization method according to claim 7, characterized in that, The step of obtaining the average depth based on the target mask and the pixel-level depth map includes: The average depth is obtained by matching the pixel-level depth map with the target mask.

Images