Underground space AR navigation system based on visual identification feature point technology

Abstract

The invention relates to the technical field of route navigation, in particular to an underground space AR (Augmented Reality) navigation system based on a visual identification feature point technology, which comprises a spatial modeling and data processing module, an identification system management module, a client application module, a real-time positioning module, a navigation engine module and a dynamic data management module. According to the method, aiming at underground multi-layer space design, a centimeter-level three-dimensional base drawing is pre-constructed by utilizing a completed BIM / CAD drawing, a three-dimensional anchor point is formed through an anti-shielding code mark pasted on the surface of a fixed structure such as a beam column, accurate distinguishing and positioning between an upper floor and a lower floor can be realized under a complete off-line condition, and when traditional GPS and Bluetooth signals fail, the positioning accuracy of the upper floor and the lower floor can be improved. The system can still complete cross or triangular positioning by visually identifying the anchor points, and the problem of navigation errors caused by unclear upstairs and downstairs division is solved.

Description

Technical Field

[0001] This invention relates to the field of route navigation technology, and in particular to an AR navigation system for underground spaces based on visual recognition feature point technology. Background Technology

[0002] With the acceleration of urban vertical development, the scale of large-scale, multi-level underground spaces, such as underground commercial complexes, subway transfer hubs, underground parking garages, and civil defense projects, is rapidly expanding. Statistics show that the total operational underground commercial area in just four cities—Beijing, Shanghai, Guangzhou, and Shenzhen—exceeds 1.8 × 10⁻⁶. 7 m 2 And it is still growing at a rate of about 15% per year. These areas generally have the following characteristics: satellite signals are completely blocked by floors, soil and steel bars, making GPS / BeiDou unusable; traditional Bluetooth Beacon, UWB or Wi-Fi RTT require dense deployment of base stations, which not only has high initial wiring costs, but also causes the error to increase rapidly after the points are offset due to decoration, ceiling, and equipment maintenance; the underground environment has a high degree of structural similarity, with escalators, elevators, fireproof roller shutters and columns of uniform specifications, making it very easy for passengers to experience "spatial confusion" between B1, B2 and B3 floors, resulting in "same-floor illusion" or "misjudgment of different floors".

[0003] To address the aforementioned pain points, the industry needs a fully offline underground space navigation solution that requires no additional base stations, can distinguish floors, achieves centimeter-level positioning accuracy, and can synchronize with real-time changes in merchants and construction. The ideal technical approach should fully leverage the stable physical structure of underground spaces, using as-built BIM / CAD drawings as a base map to construct a centimeter-level 3D mesh in one go via vehicle-mounted or handheld LiDAR. Encoded markers should be affixed to fixed structures such as beams and columns as "3D anchor points," and zero-latency relocation should be achieved by combining mobile visual recognition with triangulation / intersection positioning algorithms. Simultaneously, a lightweight incremental hot update mechanism should be used to push information such as store name changes, relocations, and temporary fencing to user terminals in real time, enabling users to navigate to the new location even after searching for the old name. This ensures low-cost operation and maintenance while continuously providing highly reliable and available underground navigation services. Summary of the Invention

[0004] To achieve the above objectives, this invention proposes an underground space AR navigation system based on visual recognition feature point technology, including a spatial modeling and data processing module, a signage system management module, a client application module, a real-time positioning module, a navigation engine module, and a dynamic data management module;

[0005] The spatial modeling and data processing module is used to generate 3D base maps with centimeter-level accuracy based on as-built drawings, and to construct static models of static elements in underground space. At the same time, it uses LiDAR for dynamic scanning and reconstruction, and filters dynamic objects.

[0006] The signage system management module is used to affix anti-obstruction signs to fixed structural surfaces and to create different signs through area division and coding;

[0007] The client application module is used to create an APP or mini-program, providing storage, search and navigation functions for underground space models, and calling cameras to determine the surrounding environment;

[0008] The real-time positioning module is used to acquire information about the surrounding environment through a camera and to locate the user's position using markers, including cross-location and triangulation methods.

[0009] The navigation engine module is used to calculate the optimal path based on the user's current location and the target point, and to overlay virtual elements on the client in real time for navigation;

[0010] The dynamic data management module is used for updating merchant data and construction areas, and retains historical name indexes when merchants change their names or relocate.

[0011] In one example, the spatial modeling and data processing module uses an onboard LiDAR for dynamic scanning and forms a centimeter-level 3D mesh through four steps: point cloud registration, noise culling, surface reconstruction, and texture mapping.

[0012] In one example, the spatial modeling and data processing module uses the deep learning instance segmentation network MaskR-CNN to filter dynamic objects that appear during the scanning process in real time on the GPU.

[0013] In one example, the signage system management module affixes anti-obstruction signs around the beams and columns in the underground space. The signs are divided into different areas and coded to form different signs.

[0014] In one example, the client application module allows users to enter the client by scanning a QR code and access the camera to rotate 360° to obtain surrounding information.

[0015] In one example, when the real-time positioning module obtains information from the surrounding tags, it selects the four nearest tags and determines the user's location range by connecting the tags at diagonal angles. When there are only tags on one side, it determines the user's location range by calculating using trigonometric functions.

[0016] In one example, the real-time positioning module triggers single-sided triangulation under boundary conditions, uses the mobile phone IMU and ARCore / ARKit to obtain angle information, calculates the user's position using the sine theorem, and uses the RANSAC algorithm to cluster 200 triangulation measurement results generated within 30 consecutive frames, removes outliers, and takes the center of the density peak subset as the final positioning coordinates.

[0017] In one example, the navigation engine module overlays virtual elements such as directional arrows and distance prompts on the client in real time and provides voice navigation functionality.

[0018] In one example, when a merchant changes its name or relocates, the dynamic data management module overwrites the original records with new data while retaining the historical name index, supporting old name searches. In construction area updates, the dynamic data management module imports 3D models of temporary fencing and passageway changes and adjusts navigation paths in real time.

[0019] In one example, a method for an AR navigation system for underground space based on visual recognition feature point technology includes the following steps:

[0020] Step 1: Establish an underground space model, generate a 3D base map with centimeter-level accuracy based on the as-built BIM / CAD drawings, and construct a static model for static elements;

[0021] Step 2: Set labels and affix anti-obstruction signs around the beams and columns in the underground space, and divide and code the areas;

[0022] Step 3: Create a client application, transmit the underground space model map to the client, and allow users to access it by scanning a QR code;

[0023] Step 4: Determining the location of the person. The client accesses the user's camera permissions, obtains surrounding information through 360° rotation, and uses tag information to determine the user's location range.

[0024] Step 5: Set the destination for navigation. Users select the target location through the client and plan the route based on the determined location.

[0025] Step 6: Map updates, updating the map after merchant changes and name changes, and retaining the historical name index.

[0026] The underground space AR navigation system based on visual recognition feature point technology proposed in this invention can bring the following beneficial effects:

[0027] 1. This invention is designed for multi-story underground spaces. It utilizes as-built BIM / CAD drawings to pre-construct a centimeter-level 3D base map and forms 3D anchor points by affixing anti-obstruction coded markings to the surfaces of fixed structures such as beams and columns. This allows for accurate differentiation and positioning between upper and lower floors even when completely offline. When traditional GPS and Bluetooth signals fail, the system can still visually identify these anchor points to complete cross or triangulation positioning, thus solving the navigation error problem caused by not being able to distinguish between upper and lower floors.

[0028] 2. By setting up a dynamic environment where merchants frequently change names, relocate, or are temporarily under construction, the system uses a dynamic data management module to perform local hot updates on the 3D base map, pushing the changed merchant information and construction site model to the client in real time; at the same time, it retains the historical name index, so that users searching for the original store can still be correctly guided to the new location, significantly reducing operation and maintenance costs and continuously ensuring navigation availability. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0030] Figure 1 A schematic diagram of the overall system architecture of this underground space AR navigation system based on visual recognition feature point technology;

[0031] Figure 2 A schematic diagram of the cross-positioning method in an underground space AR navigation system based on visual recognition feature point technology;

[0032] Figure 3 This diagram illustrates the triangulation method in an underground space AR navigation system based on visual recognition feature point technology. Detailed Implementation

[0033] To more clearly illustrate the overall concept of the present invention, a detailed description will be provided below with reference to the accompanying drawings and examples.

[0034] In the description of this invention, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0035] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0036] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a communication connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0037] In this invention, unless otherwise expressly specified and limited, the first feature "on" or "below" the second feature may be in direct contact with the first and second features, or indirect contact through an intermediate medium. In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0038] like Figures 1 to 3 As shown, this invention proposes an underground space AR navigation system based on visual recognition feature point technology, including a spatial modeling and data processing module, a signage system management module, a client application module, a real-time positioning module, a navigation engine module, and a dynamic data management module.

[0039] The spatial modeling and data processing module generates centimeter-level precision 3D base maps based on as-built BIM / CAD drawings for static model construction. This includes static elements such as beams, columns, partitions, ceilings, ventilation shafts, electrical shafts, escalators, elevators, shop fronts, fire hydrants, electrical boxes, advertising light boxes, and trash cans, which rarely shift after completion. Dynamic scanning and reconstruction are performed using scanning equipment such as vehicle-mounted LiDAR. Dynamic objects such as pedestrians, vehicles, and temporary stacks of goods appearing during the scanning process are deleted to ensure the model is established in the underground space.

[0040] The signage system management module affixes anti-obstruction signs to the surfaces of fixed structures such as beams and columns. The signs are divided into different areas and coded on different structures to form different signs. The location of the corresponding fixed structure can be determined by the sign points.

[0041] The client application module creates an APP or mini-program. Customers enter the client by scanning a QR code or other means. The client stores the underground space model created in the space modeling and data processing module and provides search and navigation functions. At the same time, it needs to access the customer's camera to determine the surrounding environment.

[0042] The real-time positioning module acquires the surrounding environment through the camera in the client application module and uses tags to locate the user's position. This includes cross-location and triangulation methods. Cross-location uses diagonal lines to intersect the tags around the user to determine the user's location range. Triangulation is used when only one side of the tag exists and a line cannot be connected. It uses trigonometric function calculations to determine the user's location range.

[0043] The navigation engine module calculates the optimal path based on the user's current location and target point, and overlays virtual elements such as directional arrows and distance prompts on the client in real time to guide the user's movement. It also adds voice navigation functionality for user convenience.

[0044] The dynamic data management module updates merchant data and construction areas. When a merchant changes its name or relocates, the new data overwrites the original record and retains the historical name index, supporting old name searches (e.g., "original route navigation store" can still navigate to the new location). In construction area updates, 3D models of temporary fences and passageway changes are imported, and navigation paths are adjusted in real time.

[0045] The specific usage method of the above system includes the following steps:

[0046] Step 1: Establish an underground space model. Underground spaces are physically highly stable. Static elements such as beams, columns, partitions, ceilings, ventilation shafts, electrical shafts, escalators, elevators, shop facades, fire hydrants, electrical boxes, advertising light boxes, and trash cans rarely shift after completion. Therefore, a BIM model or as-built CAD drawings can be generated during the design phase as the original base map. During the operation phase, vehicle-mounted LiDAR or handheld SLAM devices are used to continuously scan along main vehicular and pedestrian roads at a density of 0.1m / point, simultaneously acquiring high dynamic range texture images. A centimeter-level 3D mesh containing geometric, semantic, and textural information is formed through four steps: point cloud registration, noise removal, surface reconstruction, and texture mapping. For dynamic objects such as pedestrians, vehicles, and temporary storage areas encountered during the scanning process, a deep learning instance segmentation network, MaskR-CNN, is used for real-time filtering on the GPU. The segmentation results are manually checked and then reinjected into the training set to ensure model purity.

[0047] Step 2: Set labels. Affix labels around the beams and columns in the underground space, ensuring that the labels are not obscured and can be identified. The labels can be used to divide the underground space into areas. For example, the underground space can be divided into four parts, namely A, B, C and D, and the columns in each area can be labeled in the manner of A1, A2...An.

[0048] Step 3: Create a client application. Build a navigation client and transfer the underground space model map to the client. The client can be accessed by scanning a QR code. This client can be a mini-program or an app, allowing users to access the underground space map.

[0049] Step 4: Determining the location of the person. After scanning the code to enter the client, the client accesses the user's camera permission, obtains information about the surroundings by rotating the user 360° around their location, and determines their specific location by obtaining the tag information in the information.

[0050] In determining a specific location, there are several scenarios, including when the acquired tag information is present all around the location, combined with... Figure 2 Given the obtained tag information as A1, A2, A3, A4, B1, B2, B3, and B4, select the four tags closest to the camera, such as A1, A2, B1, and B2. These four points are located around the camera position. Connect the tags at diagonal angles, and the intersection point is the location of the existing camera point. This determines the range of the camera point's location, which is the user's location range. This range can be used to determine the user's route location.

[0051] Establish a rule for determining "points from the last four weeks".

[0052] The absolute coordinates of the pillar labels in the 3D grid are known quantities. When the client is shooting a 360° video stream, it extracts frames at 2fps and performs AprilTag decoding frame by frame to obtain the coordinates (u,v) of the center of each label in the 2D pixel coordinate system and the corresponding world coordinates (X,Y,Z).

[0053] Judgment rules:

[0054] Convert all labels in the current frame to polar coordinates (ρ, θ) with the image center as the origin, based on pixel coordinates.

[0055] Only retain labels with ρ≤0.6*min(width, height), and exclude candidates that are too far away or have excessive edge distortion.

[0056] The labels are sorted in ascending order of θ, and the difference Δθ between adjacent labels is calculated. If Δθ < 15°, the labels are merged to prevent duplicate labels on multiple sides of the same column.

[0057] Find the four labels closest to the centers of the four quadrants (45°, 135°, 225°, 315°) in the θ0–360° range; these are the “nearest four quadrant points”.

[0058] For example, at a crossroads:

[0059] The captured image shows A1 (θ = 40°), A2 (θ = 50°), B1 (θ = 130°), B2 (θ = 140°), C1 (θ = 220°), C2 (θ = 230°), D1 (θ = 310°), and D2 (θ = 320°).

[0060] After merging according to rule 3, four points A1, B1, C1, and D1 are retained, which fall exactly within ±10° of the center of the four quadrants. The system directly uses these four points as the nearest four points, and the intersection of the lines obtained is the user's location.

[0061] In a T-shaped channel:

[0062] The image only shows A1 (θ = 30°), A2 (θ = 45°), B1 (θ = 110°), B2 (θ = 125°), C1 (θ = 210°), and C2 (θ = 220°).

[0063] The system determines the situation as a "boundary case" due to the lack of a fourth quadrant label, triggering single-sided triangulation and not entering the logic for the four surrounding points.

[0064] In the empty hall

[0065] A1 (θ = 35°), A2 (θ = 40°), B1 (θ = 125°), B2 (θ = 130°), C1 (θ = 215°), C2 (θ = 220°), D1 (θ = 305°), and D2 (θ = 310°) appear on the screen.

[0066] After merging, we get A1, B1, C1, and D1, but the ρ value shows that A1 is 320px away from the center pixel, while the other three points are...

[0067] If the pixel size is less than 280px, the system will remove A1 based on the distance threshold and replace it with A2. Then, cross-location will be performed to ensure that the pixel size is the closest and that the pixel size is balanced around the perimeter.

[0068] Combination Figure 3When a tag point in the video has only one direction, it indicates that the user is currently at the boundary of the underground space. Since the tag on the other side is missing, the tags are A1, A2, B1, and B2 on the same side, and it is impossible to connect them diagonally. A triangle is established between two tags and the camera point. Since the tag point is on a pillar, the position of the pillar is fixed, and the distance is fixed. Therefore, the length of one side of the triangle is known. At the same time, a straight line parallel to the line between the two pillars is established. AR technology is used to calculate the angle between this line and the side of the triangle, and then the angle inside the triangle is determined. The length of the triangle side can be calculated using trigonometric functions. This length is the distance between the camera point and the tag position, and the angle is the angle between the camera point and the tag position. Therefore, the position of the camera point is determined. To avoid the discrepancy between calculations, multiple triangles are randomly formed between the tags and the camera point, and the positions of multiple camera points are calculated. Through the distribution of the points, the user's position is in the area with the densest distribution.

[0069] The trigonometric function calculation formula is as follows: Let the distance between the two label pillars be d, and let the camera point P and labels A and B form triangle PAB, where AB = d. Use the phone's IMU to measure the angle θ between the PAB plane and the horizontal plane, and use ARCore / ARKit to obtain the angle α between the camera's optical axis and pillar AB.

[0070] ∠APB=180°-∠PAB-∠PBA

[0071] Given that ∠PAB = α, and AB makes an angle β with the horizontal line, we can obtain ∠PBA = 90° - β - θ

[0072] According to the Law of Sines:

[0073] |PA| / sin∠PBA=|PB| / sin∠PAB=d / sin∠APB

[0074] From this, |PA| and |PB| can be solved, which are the distances from the camera point to the two pillars. To improve accuracy, the RANSAC algorithm is used to cluster the 200 triangulation results generated within 30 consecutive frames, removing 3σ field values, and taking the center of the density peak subset as the final positioning coordinates, with a horizontal error of <0.3m. If only one-sided tags are detected, height constraints are used: the pillar height H is known, and the mobile phone camera height h is obtained from the barometer and pedestrian dead reckoning. The horizontal distance d′=h·d / (Hh) is calculated by reverse calculation using the principle of similar triangles, realizing single-tag ranging. After positioning is completed, the client completes coordinate transformation within 1 second, aligns the visual positioning result to the origin of the navigation grid, and then starts path planning.

[0075] Step 5: Set the destination for navigation. The user selects the target location through the client and plans the route based on the location determined in Step 4, and arrives at the destination through the route planning.

[0076] Step 6: Map Updates. Since there are shops and other areas in the underground space, and most navigation routes are based on these shops, the map needs to be updated after shops relocate or change their names. Map updates should also be performed during construction. The updated map is then entered into the client. To prevent navigation failures after a shop name change, an overwrite process is performed during the update, backing up the original shop names to ensure navigation is still possible when searching for those names.

[0077] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0078] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. An underground space AR navigation system based on visual recognition feature point technology, characterized in that: It includes a spatial modeling and data processing module, a signage system management module, a client application module, a real-time positioning module, a navigation engine module, and a dynamic data management module; The spatial modeling and data processing module is used to generate 3D base maps with centimeter-level accuracy based on as-built drawings, and to construct static models of static elements in underground space. At the same time, it uses LiDAR for dynamic scanning and reconstruction, and filters dynamic objects. The signage system management module is used to affix anti-obstruction signs to fixed structural surfaces and to create different signs through area division and coding; The client application module is used to create an APP or mini-program, providing storage, search and navigation functions for underground space models, and calling cameras to determine the surrounding environment; The real-time positioning module is used to acquire information about the surrounding environment through a camera and to locate the user's position using markers, including cross-location and triangulation methods. The navigation engine module is used to calculate the optimal path based on the user's current location and the target point, and to overlay virtual elements on the client in real time for navigation; The dynamic data management module is used for updating merchant data and construction areas, and retains historical name indexes when merchants change their names or relocate.

2. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: The spatial modeling and data processing module uses vehicle-mounted LiDAR for dynamic scanning and forms a centimeter-level 3D mesh through four steps: point cloud registration, noise culling, surface reconstruction, and texture mapping.

3. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: The spatial modeling and data processing module uses the deep learning instance segmentation network MaskR-CNN to filter dynamic objects that appear during the scanning process in real time on the GPU.

4. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: The signage system management module affixes anti-obstruction signs around the beams and columns in the underground space. The signs are divided into different areas and coded to form different signs.

5. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: The client application module allows users to enter the client by scanning a QR code and access the camera to rotate 360° to obtain surrounding information.

6. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: When the real-time positioning module obtains information from tags around the four sides, it selects the four nearest tags and determines the user's location range by connecting the tags at diagonal angles. When there are only tags on one side, it determines the user's location range by calculating using trigonometric functions.

7. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: The real-time positioning module triggers single-sided triangulation under boundary conditions, uses the mobile phone IMU and ARCore / ARKit to obtain angle information, calculates the user's position using the sine theorem, and uses the RANSAC algorithm to cluster 200 triangulation results generated within 30 consecutive frames, removes outliers, and takes the center of the density peak subset as the final positioning coordinates.

8. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: The navigation engine module overlays directional arrows and distance prompts as virtual elements on the client side in real time and provides voice navigation functionality.

9. The underground space AR navigation system based on visual recognition feature point technology according to claim 1, characterized in that: When a merchant changes its name or relocates, the dynamic data management module overwrites the original records with new data and retains the historical name index, supporting old name searches. During construction area updates, the dynamic data management module imports 3D models of temporary fencing and passageway changes and adjusts navigation paths in real time.

10. A method for implementing the underground space AR navigation system based on visual recognition feature point technology according to any one of claims 1-9, characterized in that: Includes the following steps: Step 1: Establish an underground space model, generate a 3D base map with centimeter-level accuracy based on the as-built BIM / CAD drawings, and construct a static model for static elements; Step 2: Set labels and affix anti-obstruction signs around the beams and columns in the underground space, and divide and code the areas; Step 3: Create a client application, transmit the underground space model map to the client, and allow users to access it by scanning a QR code; Step 4: Determining the location of the person. The client accesses the user's camera permissions, obtains surrounding information through 360° rotation, and uses tag information to determine the user's location range. Step 5: Set the destination for navigation. Users select the target location through the client and plan the route based on the determined location. Step 6: Map updates, updating the map after merchant changes and name changes, and retaining the historical name index.

Images