Indoor AR (Augmented Reality) navigation system and method for fusing sparse space map and dynamic path

Abstract

The invention discloses a sparse space map and dynamic path fused indoor AR navigation system and method, and relates to the technical field of augmented reality navigation, and the system comprises a sparse map construction module, a man-machine interaction adaptation module, a landmark mark management module, a dynamic path fusion module and an AR navigation execution module. The sparse map construction module is used for realizing three-dimensional reconstruction and persistent storage of an indoor scene, the man-machine interaction adaptation module is used for resolving ground height parameters in real time and dynamically correcting a space coordinate system, and the landmark marking management module is used for generating an editable marking object containing space coordinates and attribute data. The dynamic path fusion module is used for carrying out topological association on a user-defined path and an algorithm planning path, the AR navigation execution module is used for realizing path visualization and path dynamic optimization through a path matching algorithm and realizing AR navigation, and the defects that an existing indoor navigation technology is low in positioning precision, high in mapping cost and poor in dynamic adaptability can be overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of augmented reality navigation, and particularly to an indoor AR navigation system and method for fusing sparse spatial maps and dynamic paths. Background Art

[0002] AR indoor navigation is one of the important application fields of indoor positioning technology. Indoor navigation has long faced three core challenges: low positioning accuracy, high mapping cost, and poor dynamic adaptability. Traditional satellite navigation (such as GPS) completely fails indoors due to signal attenuation, while positioning solutions based on Bluetooth beacons and Wi-Fi fingerprints require the deployment of a large number of hardware devices, with high costs and complex maintenance.

[0003] In recent years, indoor augmented reality navigation technology has developed rapidly, and its core depends on the combination of environmental perception, positioning algorithms, and virtual-real fusion capabilities. Traditional solutions mostly use high-precision dense three-dimensional maps (based on lidar or photogrammetry technology) as the basis for navigation, but dense three-dimensional reconstruction consumes a large amount of computing resources and is difficult to run in real time on mobile devices. Although visual SLAM technology can achieve autonomous positioning through cameras and IMUs, it is prone to feature point loss in textureless areas (such as solid-color walls), resulting in positioning drift. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an indoor AR navigation system for fusing sparse spatial maps and dynamic paths, including A sparse map construction module, which is based on visual inertial odometry cameras and multi-sensor fusion technology, constructs a three-dimensional sparse map of the indoor scene and locates the mobile device synchronously through the SLAM algorithm, and uses the RANSAC algorithm to remove abnormal feature points to construct a map database containing environmental semantic information; A human-computer interaction adaptation module, which is based on the ray detection method and ground height dynamic calculation technology, and corrects the three-dimensional space coordinate system in real time, where the ground height parameter is updated by triggering the position clicked by the user; A landmark marking management module, which aligns with the cloud sparse spatial map coordinates through feature point matching to generate an editable marking object, and the marking object includes spatial coordinates, attribute data, and semantic labels; A dynamic path fusion module, which uses a topological association mechanism to fuse the user-defined path and the algorithm-planned path, and performs dynamic optimization by calculating multi-weight metrics through a path optimization unit. The multi-weight metrics include path length, traffic status, and user preference; the path optimization unit compares and optimizes the path by calculating the basic weight metrics. The basic weight metrics include path length weight, traffic obstruction rate weight, and user preference weight, and the user preference weight increases with the increase in the number of times the user selects this path; The AR navigation execution module integrates a map location detection unit and a path matching algorithm to perform path visualization and dynamic path optimization, thereby performing AR navigation.

[0005] The further limited technical solution of the present invention is: Further, in the sparse map construction module, multi-sensor fusion includes timestamp alignment visual error, IMU, and depth sensor data fusion; the specific method of using the RANSAC algorithm to eliminate abnormal feature points is: randomly select the minimum number of sample points to fit the model, calculate the error between the data points and the model, and set a basic threshold. Points with an error less than the basic threshold are classified as inliers, and the rest are outliers; repeat the above process, and finally select the model with the most inliers as the optimal model to eliminate abnormal feature points.

[0006] In the indoor AR navigation system that fuses the sparse spatial map and the dynamic path as described above, in the human-computer interaction adaptation module, the ray detection method is specifically as follows: the system emits a virtual ray from the starting point of the camera in a specified direction, and detects the intersection points with the collision bodies on the ray path. If the ray intersects with the collision body, the detected value is returned; the ground height dynamic calculation technology is used to calculate the height of the mobile device from the ground. By combining the parameter values obtained from the ray detection with the three-dimensional space coordinates through the ground height calculation algorithm in the system, the relative height of the mobile device from the ground is finally calculated.

[0007] In the indoor AR navigation system that fuses the sparse spatial map and the dynamic path as described above, in the landmark marking management module, feature point matching is performed through ORB combined with FAST key point detection and BRIEF descriptor. FAST is used to detect corner points in the image, and corner points are determined by detecting the pixel brightness changes around a circle of pixels. ORB estimates the direction of the key points detected by FAST; BRIEF generates a descriptor by comparing the intensities of pixel pairs in the image; ORB improves BRIEF, and the generated descriptor is a binary vector; in AR navigation, feature point matching is performed by matching the ORB feature points in the real map recognized by the camera with the virtual scene feature points stored in advance.

[0008] In the indoor AR navigation system that fuses the sparse spatial map and the dynamic path as described above, in the dynamic path fusion module, the specific method of the topology association mechanism for fusing the user-defined path and the algorithm-planned path is: form a topology graph by topologically associating the nodes in the map with the nodes of the user-defined path, and find the optimal path from the starting point to the ending point in the topology graph through a search algorithm.

[0009] In the indoor AR navigation system that fuses the sparse space map and the dynamic path as described above, in the AR navigation execution module, when the map location detection unit performs AR navigation and after the map matching is completed, the system compares the feature points of the real map with the feature points of the saved sparse space map in the spatial coordinate system, and calculates the relative direction and position of the locations existing in the real map according to the relative positions.

[0010] The present invention also provides an indoor AR navigation method that fuses a sparse space map and a dynamic path, which is applied to the indoor AR navigation system that fuses the sparse space map and the dynamic path as described above, and includes the following steps: S1. Create a map, synchronously collect environmental data through multiple sensors, construct a three-dimensional sparse map of the indoor scene using the SLAM algorithm, and store the feature points and semantic labels in the cloud database; S2. Based on the ray detection method, solve the ground height in real time. When the user interaction behavior triggers the coordinate system correction, update the spatial coordinate axis offset and reprojection of the virtual object; S3. Match the pose of the current mobile device in the cloud map, generate an AR location information marker that supports dynamic binding and modification of data, so as to set the locations required for navigation in the map; S4. Between the set navigation locations, fuse the user-defined path and the algorithm-planned path, and generate a multi-objective optimization path through the dynamic path fusion and path comparison and optimization algorithm; According to the selected starting point and ending point by the user, combine the predefined path data and the path data obtained from the algorithm analysis, and dynamically plan an optimal path; S5. Execute AR indoor navigation. When the user selects a target location in the current map, the system will generate a visual multi-objective optimization path to guide the user to the target location.

[0011] Further, in step S1, capture the feature points in the environment through the camera. The feature points include corner points and edges, and combine the visual inertial odometer or the pure visual odometer to estimate the pose of the mobile device in real time, that is, the position and direction, and at the same time construct a sparse three-dimensional space map; The motion model of the mobile device is used to describe the change of the pose over time, expressed as: p t =f(p t-1 ,u t )+w t where f(p t-1 ,u t ) is the motion function, which describes the transformation of the mobile device from p t-1 to p t ; u t is the control input, including speed and angular velocity; w t is the motion noise; The observation model of the mobile device is used to describe the observation of the feature points in the environment by the mobile device, which is expressed as: z t =h(p t ,m)+v t Among them, h(p t ,m) is the observation function, describing the position p of the mobile device t The process from z to the observed feature point m; t is the observed value, i.e. the coordinate of the feature point; v t is the observation noise.

[0012] In the indoor AR navigation method of the sparse spatial map and dynamic path fusion described above, in step S3, the feature points of the real map obtained by the camera are compared with the feature points of the cloud map to achieve pose matching; the feature point matching is achieved by minimizing the reprojection error: Among them, z ti is the time t for feature point m i The observed value, p t is the position of the mobile device, h(p t ,m i ) is to transform the feature point m i Predicted values ​​projected onto the image plane.

[0013] As described above, in the indoor AR navigation method that integrates sparse spatial maps and dynamic paths, in step S3, according to the matched sparse map, based on this map, the target location for the required navigation is set in the space, and the system visualizes the location information through the information visualization placement algorithm, generates an AR location information tag in the map scene that supports dynamic binding and modification of attribute data, and sets its position and name.

[0014] The beneficial effects of the present invention are: (1) In the present invention, a 3D reconstruction method for indoor scenes is realized by using sparse spatial map construction technology, which effectively solves the current high-efficiency and real-time problems of SLAM in complex dynamic environments; for areas with low texture, high dynamics, and large areas without feature points; (2) In the present invention, existing AR navigation technologies generally have problems such as low positioning accuracy, low path planning efficiency, and even large deviations. The present invention constructs a visual landmark marking system to generate augmented reality location information tags that support dynamic binding and modification of attribute data, records the relative coordinates of areas without significant features in the system, sets virtual landmarks and dynamic path fusion mechanisms for them, and topologically associates user-defined paths with algorithm-planned paths, thereby effectively solving the above problems; (3) In the present invention, through the dual data fusion architecture of the sparse map and the dynamic path, combined with the path correction mechanism of human-machine collaboration, the collaborative improvement of navigation accuracy and path flexibility is achieved, which is particularly applicable to indoor continuous navigation scenarios with multiple floors and across regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic structural diagram of the AR navigation system in Embodiment 1 of the present invention; Figure 2 It is a schematic flowchart of the AR navigation method in Embodiment 2 of the present invention; Figure 3 It is a schematic diagram of the spatial map construction in Embodiment 3 of the present invention; Figure 4 It is a schematic diagram of the ground height and location settings in the sample area of Embodiment 3 of the present invention; Figure 5 It is a schematic diagram of indoor AR navigation in the sample area of Embodiment 3 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS Embodiment 1:

[0016] An indoor AR navigation system that fuses a sparse spatial map and a dynamic path provided in this embodiment, as Figure 1 shown, includes a sparse map construction module, a human-computer interaction adaptation module, a landmark marking management module, a dynamic path fusion module, and an AR navigation execution module.

[0017] The sparse map construction module, based on the Visual-Inertial Odometry (VIO) camera and multi-sensor fusion technology, synchronously constructs a three-dimensional sparse map of the indoor scene and locates the mobile device (mobile phone) through the SLAM (Simultaneous Localization and Mapping) algorithm, and uses the RANSAC (Random Sample Consensus) algorithm to eliminate abnormal feature points, and constructs a persistent map database containing environmental semantic information.

[0018] Multi-sensor fusion refers to aligning the visual error, IMU (Inertial Measurement Unit), and depth sensor data through timestamp alignment to improve the robustness of SLAM mapping.

[0019] RANSAC abnormal feature point elimination refers to randomly selecting the minimum sample points to fit the model, calculating the error between the data points and the model, and setting a basic threshold. The points with an error less than the basic threshold are classified as inliers, and the rest are outliers; repeating the above process, finally selecting the model with the most inliers as the optimal model, so as to eliminate abnormal feature points and achieve robust fitting of the data.

[0020] The sparse map construction module uses the VIO camera and fuses with multiple sensors to calculate the device pose. According to the device movement, it synchronously scans the feature points of the real environment map, records them in the spatial map and eliminates errors, and finally uploads them to the cloud database.

[0021] The human-computer interaction adaptation module, based on the ray detection method and the ground height dynamic calculation technology, real-time corrects the three-dimensional space coordinate system, and the ground height parameter is triggered to update by the position clicked by the user.

[0022] The ray detection method means that the system emits a virtual ray along a specified direction from the starting point of the camera to detect the intersection point with the collider on the ray path. If the ray intersects the collider, the detected value is returned; the ground height calculation technology is used to calculate the height of the mobile device from the ground. By combining the parameter values obtained from the ray detection with the three-dimensional space coordinates through the ground height calculation algorithm in the system, the relative height of the mobile device from the ground is finally calculated.

[0023] The human-computer interaction adaptation module helps the user control the settings in the system through the UI interface button. When the user clicks the button related to editing the ground height in the system menu, the system will apply to open the device camera. Then the system will scan the environment map. After the scanning is completed, the system will emit a virtual ray to detect the intersection point with the collider on the ray path to set the height of the device from the ground to determine the initial height value.

[0024] The triggering methods for ground height update include: the user clicks the ground height setting button, the system generates an AR entity, and the user clicks and places it in the scene. The system will detect the height difference between the device and it to set the ground height.

[0025] The landmark marking management module aligns with the cloud sparse space map coordinates through feature point matching to generate an editable marking object, which includes spatial coordinates, attribute data, and semantic labels.

[0026] Feature point matching refers to using ORB (Oriented FAST and Rotated BRIEF) combined with FAST (Features from Accelerated Segment Test) key point detection and BRIEF (Binary Robust Independent Elementary Features) descriptor for feature point matching. FAST is used to detect corner points in the image by detecting the pixel brightness changes around a circle of pixels. ORB estimates the direction of the key points detected by FAST to make it rotationally invariant.

[0027] BRIEF is a method based on binary descriptors that generates descriptors by comparing the intensities of pixel pairs in an image; ORB improves BRIEF to make it more suitable for rotational invariance. The generated descriptor is a binary vector, which has a fast calculation speed and small storage space. In AR navigation, feature point matching is achieved by matching the ORB feature points recognized by the camera in the real map with the pre-stored virtual scene feature points.

[0028] The coordinate alignment method includes: an online relocalization technique based on the ORB-SLAM3 framework, which realizes real-time matching of the current device pose and the cloud map through virtual-real feature point matching; and uses the semantic segmentation algorithm U-Net to classify and label the attribute data of the marked object.

[0029] The landmark marking management module enables users to add location landmark information in the cloud map through the interface with the cloud database, fuses the feature point information with the three-dimensional coordinates, and realizes the persistent storage of the landmark marking and the space. Figure 1 The core of landmark marking management is feature point matching, which uses ORB combined with FAST key point detection and BRIEF descriptors to achieve high-precision and high-efficiency feature point matching.

[0030] The dynamic path fusion module uses a topological association mechanism to fuse the user-defined path and the algorithm-planned path, and realizes dynamic optimization through the path optimization unit calculating multiple weight metrics (path length, traffic status, user preference); the path optimization unit compares and optimizes the paths by calculating the basic weight metrics, and the basic weight metrics include the path length weight, the traffic obstruction rate weight, and the user preference weight, and the user preference weight will increase as the user selects this path multiple times.

[0031] The specific method for the topological association mechanism to fuse the user-defined path and the algorithm-planned path is: form a topological graph by topologically associating the nodes in the map with the nodes of the user-defined path, and find the optimal path from the starting point to the ending point in the topological graph through a search algorithm.

[0032] The topological association mechanism includes: users can place virtual identifiers in the virtual map and customize the path through the connection lines between the virtual identifiers; the nodes of the constructed path map include three-dimensional coordinates, traffic status identifiers, and dynamic weight parameters; the user-defined path is injected into the map through a priority evaluation model, and the priority weight coefficient is higher than that of the algorithm-planned path; the user-defined paths with repeated starting points are optimized by comparing the weight values.

[0033] Dynamic path fusion is based on the weights calculated by the path optimization unit to dynamically fuse the user-defined path and the path formulated by the system algorithm. The user-defined path specifically lies in the user-defined path function of the system, which sets a path that meets the user's needs based on the set landmark markings in the map.

[0034] The AR navigation execution module integrates a map location detection unit and a path matching algorithm to perform path visualization and dynamic path optimization, realizing AR navigation.

[0035] When the map location detection unit is in AR navigation, after the map is matched, the system compares the feature points of the real map with the feature points of the saved sparse spatial map in the spatial coordinate system, and calculates the relative direction and position of the locations existing in the real map according to the relative position.

[0036] The path matching algorithm specifically refers to a navigation path calculation and display algorithm that combines virtual and real. The optimal path data is imported and the path matching algorithm is used. This algorithm is based on virtual path data, combines the virtual path with the real world, and displays it on the map through the mobile phone camera, and performs real-time detection and dynamic adjustment on the path; in this embodiment, the dynamic window method (Dynamic Window Approach, DWA) is used to sample the current position and speed state of the user, formulate speed samples, and perform real-time obstacle avoidance optimization on the path.

[0037] The user enables AR navigation by scanning the real map and matching it with the cloud. When the user is about to reach the navigation destination, the navigation ends. Embodiment Two:

[0038] An indoor AR navigation method for fusing a sparse spatial map and a dynamic path provided in this embodiment is applied to the indoor AR navigation system for fusing a sparse spatial map and a dynamic path proposed in Embodiment One, as Figure 2 shown, and includes the following steps: S1. Create a map, synchronously collect environmental data through multiple sensors, use the SLAM algorithm to construct a three-dimensional sparse map of the indoor scene, and store the feature points and semantic labels in the cloud database.

[0039] The process of creating and saving the sparse map is as follows: Creating the sparse spatial map mainly relies on the SLAM technology. Feature points (such as corner points, edges, etc.) in the environment are captured through the camera, and the pose (position and direction) of the mobile device is estimated in real time by combining visual inertial odometry or pure visual odometry (Visual Odometry, VO), while constructing a sparse three-dimensional spatial map.

[0040] The sparse spatial map constructed in this embodiment ensures the accuracy and real-time performance of the map by optimizing the feature point matching and pose estimation algorithms, and at the same time supports persistent storage, providing a stable environment understanding and positioning ability for AR applications.

[0041] The motion model of the mobile device describes the change of pose over time, expressed as: pt = f(p t-1 , u t ) + w t where f(p t-1 , u t ) is the motion function that describes the transformation of the mobile device from p t-1 to p t ; u t is the control input, including speed and angular velocity; w t is the motion noise (assumed to be Gaussian noise).

[0042] The observation model of the mobile device describes the observation of the feature points in the environment by the mobile device and is expressed as: z t = h(p t , m) + v t where h(p t , m) is the observation function that describes the process from the pose p t of the mobile device to the observed feature point m; z t is the observation value (coordinates of the feature point); v t is the observation noise.

[0043] S2. Real-time calculation of the ground height based on the ray detection method. When the user interaction behavior triggers the coordinate system correction, update the spatial axis offset and reprojection of the virtual object.

[0044] The process of calculating the ground height is as follows: Based on the spatial perception technology of the user interaction behavior, combine the initial value of the ground height stored in the variable, and use ray detection to detect the position clicked by the user, and calculate the ground height parameter in real time and dynamically correct the spatial coordinate system.

[0045] S3. Match the current pose of the mobile device in the cloud map, generate an AR location information marker that supports dynamic binding and modification of data, and set the location required for navigation in the map.

[0046] The process of matching the current device pose is as follows: Compare the real-world map feature points obtained by the camera with the cloud map feature points to achieve pose matching; feature point matching is a key step in SLAM and is achieved by minimizing the reprojection error: where z ti is the observation value of the feature point m i at time t, p t is the pose of the mobile device, and h(p t , m i ) is to transform the feature point mi The predicted value projected onto the image plane.

[0047] The process of generating an augmented reality location information marker that supports dynamic binding and modification of attribute data is as follows: Based on the matched sparse map, set the target location for the required navigation within the space. The system visualizes the location information through an information visualization placement algorithm, generates an AR location information marker that supports dynamic binding and modification of attribute data in the map scene, and sets its position and name, providing users with an intuitive location management function. Users can set locations for multiple saved maps.

[0048] S4. Between the set navigation locations, fuse the user-defined path and the algorithm-planned path, and generate a multi-objective optimized path through a dynamic path fusion and path comparison and optimization algorithm.

[0049] The process of generating a multi-objective optimized path is as follows: Create a custom path through the dynamic path creation and management function. Provide users with an intuitive and flexible way to define the connection relationship between locations. The innovation lies in dynamic path creation, location-based path definition, data persistence and real-time update, and modular design. This method of creating a path aims to efficiently and conveniently solve the navigation problem in large areas without feature points.

[0050] The navigation path is calculated through a dynamic path fusion and path comparison and optimization algorithm. The system dynamically plans an optimal path based on the starting point and ending point selected by the user, combining the predefined path data and the path data obtained from algorithm analysis; during the path planning process, the system will avoid duplicate paths and dead loops to ensure the effectiveness of the path.

[0051] S5. Execute AR indoor navigation. When the user selects a target location within the current map, the system will generate a visualized multi-objective optimized path to guide the user to the target location.

[0052] The process of AR navigation is as follows: The system renders the navigation path in real time and superimposes the virtual path onto the display scene. The starting point and ending point of the path are dynamically adjusted according to the user's position and the target location. During the navigation process, the system will detect the distance between the user and the target location in real time, and based on the path priority evaluation model, achieve precise navigation within the scene and dynamic optimization of the navigation path through a path matching algorithm. When the user approaches the target location (the distance from the target point is less than the preset distance), the navigation is completed. Example Three:

[0053] This example is for AR indoor navigation in a large scene. In a university library, the following steps are adopted: Step One. AsFigure 3 As shown, first use the map creation function in the menu to scan the characteristic areas in the library, not limited to Figure 3 the areas shown.

[0054] Step 2: As Figure 4 shown, click the location setting button in the menu, match the map saved in the cloud based on the feature points. After successful matching, first set the ground height in the map, and then set the target location in the map. If necessary, customize the special path between locations. The example shown is the second floor of this university library.

[0055] Step 3: Start navigation. Click the start navigation button in the menu. The system will match the feature points in the area where the user is located with the feature points of the sparse map in the cloud. After successful matching, select the target location to start indoor AR navigation. After navigation starts, according to the formulated function, judge the location of the user and use a visual virtual path to guide the user to the destination. As Figure 5 shown, the navigation path shown is the area from borrowing area -1 to borrowing area -2 on the second floor of this university library. If the user deviates from the navigation path, the system will recalculate and match the optimal path according to the coordinates and pose of the user's location.

[0056] The present invention realizes a three-dimensional reconstruction method for indoor scenes through sparse space map construction technology, and efficiently solves the problems of high efficiency and real-time in the current SLAM in complex dynamic environments; for areas with low texture, high dynamics, and large areas without feature points, existing AR navigation technologies generally have problems such as low positioning accuracy, low path planning efficiency, and even large deviations. The present invention constructs a visual landmark marking system, generates augmented reality location information marks that support dynamic binding and modification of attribute data, records the relative coordinates of areas without significant features in the system and sets a virtual landmark and dynamic path fusion mechanism for them, and topologically associates the user-defined path with the algorithm-planned path. These two methods effectively solve the above problems.

[0057] The present invention realizes the coordinated improvement of navigation accuracy and path flexibility through a dual data fusion architecture of sparse maps and dynamic paths, combined with a human-machine collaborative path correction mechanism, and is particularly suitable for indoor continuous navigation scenarios with multiple floors and cross-regions.

[0058] In addition to the above embodiments, the present invention may also have other implementation manners. All technical solutions formed by equivalent replacement or equivalent transformation fall within the protection scope required by the present invention.

Claims

1. An indoor AR navigation system that fuses sparse spatial maps and dynamic paths, characterized in that: including a sparse map construction module, which, based on visual-inertial odometry camera and multi-sensor fusion technology, synchronously constructs a three-dimensional sparse map of the indoor scene and locates the mobile device through the SLAM algorithm, and uses the RANSAC algorithm to eliminate abnormal feature points, and constructs a map database containing environmental semantic information; a human-computer interaction adaptation module, which, based on the ray detection method and the ground height dynamic calculation technology, corrects the three-dimensional space coordinate system in real time, and the ground height parameter is updated by triggering the position clicked by the user; a landmark marking management module, which aligns the feature points with the coordinates of the sparse space map in the cloud through feature point matching, and generates an editable marking object, which contains spatial coordinates, attribute data and semantic labels; a dynamic path fusion module, which uses a topological association mechanism to fuse the user-defined path and the algorithm-planned path, and performs dynamic optimization by calculating multiple weight indicators through a path optimization unit. The multiple weight indicators include path length, traffic status and user preference; the path optimization unit compares and optimizes the path by calculating the basic weight indicators. The basic weight indicators include path length weight, traffic obstruction rate weight and user preference weight, and the user preference weight increases with the increase in the number of times the user selects this path; an AR navigation execution module, which integrates a map location detection unit and a path matching algorithm to perform path visualization and path dynamic optimization, so as to perform AR navigation.

2. The indoor AR navigation system for fusing sparse spatial maps and dynamic paths according to claim 1, wherein: In the sparse map construction module, the multi-sensor fusion includes timestamp alignment visual error, IMU and depth sensor data fusion; the specific method of using the RANSAC algorithm to eliminate abnormal feature points is: randomly select the minimum sample points to fit the model, calculate the error between the data points and the model, and set the basic threshold, and divide the points with errors less than the basic threshold into inliers, and the rest are outliers; Repeat the above process, and finally select the model with the most inliers as the optimal model, so as to eliminate abnormal feature points.

3. The indoor AR navigation system for fusing a sparse spatial map and a dynamic path according to claim 1, wherein: In the human-computer interaction adaptation module, the ray detection method is specifically: the system emits a virtual ray along the specified direction from the starting point of the camera, and detects the intersection point with the collision body on the ray path. If the ray intersects the collision body, the detected value is returned; the ground height dynamic calculation technology is used to calculate the height of the mobile device from the ground. Through the ground height calculation algorithm in the system, the parameter value obtained by ray detection is combined with the three-dimensional space coordinate, and finally the relative height of the mobile device from the ground is calculated.

4. The indoor AR navigation system for fusing sparse spatial maps and dynamic paths according to claim 1, wherein: In the landmark marking management module, feature point matching is performed through ORB combined with FAST key point detection and BRIEF descriptor. FAST is used to detect corner points in the image, and the corner points are determined by detecting the pixel brightness change around a circle of pixels. ORB estimates the direction of the key points detected by FAST; BRIEF generates a descriptor by comparing the intensities of pixel pairs in the image; ORB improves BRIEF, and the generated descriptor is a binary vector; in AR navigation, feature point matching is performed by matching the ORB feature points in the real map recognized by the camera with the pre-stored virtual scene feature points.

5. The indoor AR navigation system for fusing a sparse spatial map and a dynamic path according to claim 1, characterized in that: In the dynamic path fusion module, the specific method for the topology association mechanism to fuse the user-defined path and the algorithm-planned path is as follows: By topologically associating the nodes in the map with the nodes of the user-defined path to form a topology graph, and finding the optimal path from the starting point to the ending point in the topology graph through a search algorithm.

6. The indoor AR navigation system for fusing a sparse spatial map and a dynamic path according to claim 1, wherein: In the AR navigation execution module, during AR navigation, after the map location detection unit finishes map matching, the system compares the feature points of the real map with the feature points of the saved sparse spatial map in the spatial coordinate system, and calculates the relative direction and position of the locations existing in the real map according to the relative positions.

7. An indoor AR navigation method for fusing sparse spatial maps and dynamic paths, which is applied to the indoor AR navigation system for fusing sparse spatial maps and dynamic paths according to any one of claims 1-6, and is characterized in that: It includes the following steps: S1. Create a map, synchronously collect environmental data through multiple sensors, use the SLAM algorithm to construct a three-dimensional sparse map of the indoor scene, and store the feature points and semantic labels in the cloud database; S2. Based on the ray detection method, solve the ground height in real time. When the user interaction behavior triggers the coordinate system correction, update the spatial coordinate axis offset and reprojection of the virtual object; S3. Match the current pose of the mobile device in the cloud map, generate an AR location information marker that supports dynamic binding and modification of data, so as to set the locations required for navigation in the map; S4. Between the set navigation locations, fuse the user-defined path and the algorithm-planned path, and generate a multi-objective optimized path through the dynamic path fusion and path comparison and optimization algorithm; According to the starting point and ending point selected by the user, combine the predefined path data and the path data obtained from algorithm analysis, and dynamically plan an optimal path; S5. Execute AR indoor navigation. When the user selects a target location in the current map, the system will generate a visual multi-objective optimized path to guide the user to the target location.

8. The indoor AR navigation method for fusing sparse spatial map and dynamic path according to claim 7, characterized in that: In the step S1, capture the feature points in the environment through the camera. The feature points include corner points and edges, and combine the visual inertial odometer or pure visual odometer to estimate the pose of the mobile device in real time, that is, the position and direction, and at the same time construct a sparse three-dimensional space map; The motion model of the mobile device is used to describe the change of the pose over time, expressed as: p t = f(p t-1 , u t ) + w t Among them, f(p t-1 , u t ) is a motion function that describes the transformation of the mobile device from p t-1 to p t ; u t is the control input, including speed and angular velocity; w t is the motion noise; The observation model of the mobile device is used to describe the observation of the feature points in the environment by the mobile device, expressed as: z t =h(p t ,m)+v t where \(h(p t , m)\) is the observation function that describes the process from the pose \(p t \) of the mobile device to the observed feature point \(m\); \(z t \) is the observation value, i.e., the coordinates of the feature point; \(v t \) is the observation noise.

9. The indoor AR navigation method for fusing sparse spatial maps and dynamic paths according to claim 7, characterized in that: In the step S3, compare the feature points of the real map obtained by the camera with the feature points of the cloud map to achieve pose matching; Achieve feature point matching by minimizing the reprojection error: Among them, z ti is the observation value of the feature point m i with respect to time t, p t is the pose of the mobile device, and h(p t , m i ) is the predicted value of projecting the feature point m i onto the image plane.

10. The indoor AR navigation method for fusing sparse spatial map and dynamic path according to claim 7, characterized in that: In the step S3, based on the matched sparse map, set the target locations required for navigation in the space. The system uses the information visualization placement algorithm to visualize the location information, generate an AR location information marker that supports dynamic binding and modification of attribute data in the map scene, and set its position and name.

Images