A dynamic slam method based on static semantic anchor points

Abstract

The application relates to the technical field of SLAM, and provides a dynamic SLAM method and system based on static semantic anchor points, which comprises the following steps: static structure semantic anchor point generation, anchor point-based robust tracking and multi-sensor tight coupling fusion, back-end optimization and static semantic map construction. By actively selecting high-stability static structure features, combining lightweight semantic segmentation and space-time consistency verification, the problems of poor dynamic elimination effect and large calculation cost in the prior art are solved. The method improves the positioning robustness and precision in a dynamic scene, reduces the calculation cost, generates a static map rich in semantics, and has strong universality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of SLAM, in particular to a dynamic SLAM method based on static semantic anchor points. BACKGROUND

[0002] SLAM technology is the core support for realizing autonomous perception and navigation in the fields of robots, autonomous driving, augmented reality, etc. Traditional SLAM technology is generally based on the strong assumption that the observed environment is static, but in real application scenarios, the existence of dynamic objects is very common, and this assumption is difficult to hold. In existing dynamic processing schemes, the dynamic rejection scheme based on geometry and motion consistency is the mainstream choice, which identifies and rejects abnormal feature points through geometric constraints or motion models. However, when dynamic objects are stationary or moving slowly for a long time, the motion of feature points of these objects is very similar to that of static background points, and geometric methods cannot effectively distinguish them, which may misjudge them as static points and introduce them into the calculation, resulting in map pollution and positioning drift. When dynamic objects account for a large proportion in the field of view, a large number of dynamic feature points may dominate the statistics, and static background points may be misjudged as outliers and rejected, causing tracking failure. Another type of pixel-level rejection scheme based on deep learning semantic segmentation requires full-pixel-level processing of each frame of image to mask the dynamic object area. However, this scheme has huge computational overhead, and high-precision real-time semantic segmentation models require a large amount of computing resources, which is difficult to run in real time on embedded platforms, and there is a problem of misdeleting static features and wasting available static features, which severely limits its application range.

[0003] Based on the above problems, there is an urgent need for a SLAM technology scheme that can accurately distinguish static and dynamic features in dynamic scenes while considering computational efficiency. SUMMARY

[0004] The purpose of the present application is to provide a dynamic SLAM method based on static semantic anchor points, which includes the steps of sensor data acquisition, positioning estimation and map construction, and further includes the steps of static structure semantic anchor point generation, filtering high-reliability feature points from original sensor data; robust tracking based on anchor points and multi-sensor tight coupling fusion, using the generated anchor points to perform accurate pose estimation; backend optimization and static semantic map construction, maintaining a global consistent and rich semantic information long-term map; the static structure semantic anchor point generation step, the robust tracking based on anchor points and the multi-sensor tight coupling fusion step, and the backend optimization and static semantic map construction step are executed in sequence to form a complete SLAM operation process.

[0005] Preferably, the static structure semantic anchor generation step includes lightweight semantic segmentation and static region identification operation, inputting single-frame RGB or RGB-D images, using a semantic segmentation network with an improved MaskR-CNN structure taking MobileNetV2 as the backbone network for real-time inference, and the semantic segmentation network is pre-trained to identify static structures such as walls, columns, fixed shelves, door frames, and ground categories, and outputs a pixel-level semantic mask that identifies static structure regions.

[0006] Preferably, the static structure semantic anchor generation step further includes anchor feature extraction and preliminary screening operation, only in the static structure region identified by the semantic mask, local feature points are extracted using ORB algorithm, SIFT algorithm or lightweight convolutional network, and geometric stability filtering is performed on the extracted feature points, low-curvature points with flat surfaces are selected in laser SLAM, and corner points that can be stably tracked in the image pyramid are selected in visual SLAM.

[0007] Preferably, the static structure semantic anchor generation step further includes spatio-temporal consistency verification operation, the newly extracted candidate anchor points are associated with the static and successfully tracked anchor points in the past several frames, the consistency of the motion vector of the new anchor point with the camera motion estimation and the average motion pattern of the surrounding stable anchor points is verified, the spatial relationship of the three-dimensional position of the new anchor point with the adjacent structure in the existing static map is verified, and a set of static structure semantic anchor points with a credibility score is output.

[0008] Preferably, the anchor-based robust tracking and multi-sensor tight coupling fusion step includes a priority tracking strategy, static structure semantic anchors are preferentially used for inter-frame matching and pose calculation in visual odometry or front-end tracking, and the anchor points with the highest credibility are preferentially used to calculate the initial pose through PnP algorithm or ICP algorithm in stages, and other ordinary feature points are matched and abnormal values are eliminated under the guidance and verification of the initial pose.

[0009] Preferably, the anchor-based robust tracking and multi-sensor tight coupling fusion step further includes multi-sensor tight coupling fusion operation, when the system is equipped with an IMU sensor, the visual pose calculated based on the anchor points is tightly coupled with the pre-integration result of the IMU, and in the optimization framework, the information matrix weight of the observation value from the anchor points is higher than that of the IMU instantaneous measurement value.

[0010] Preferably, the back-end optimization and static semantic map construction step includes anchor-enhanced back-end graph optimization operation, in the g2o or GTSAM-based graph optimization back-end, anchor pose constraint edges and ordinary feature point constraint edges are introduced, the anchor pose constraint edges are generated by static structure semantic anchors and are assigned higher weights, and the ordinary feature point constraint edges are generated by ordinary feature points extracted from other static regions and have lower weights.

[0011] Preferably, the back-end optimization and static semantic map construction step further comprises a long-term static semantic map management operation, the constructed map contains three-dimensional point cloud or geometric information of feature points, each point is associated with a corresponding semantic label to form a static semantic map, the semantic label includes wall points and column points.

[0012] Preferably, the back-end optimization and static semantic map construction step further comprises a loop closure detection operation, in the loop closure detection, key frames with the same semantic anchor point type are preferentially matched, the semantic anchor point type includes a wall corner type.

[0013] Preferably, the back-end optimization and static semantic map construction step further comprises a map lifelong maintenance operation, the stability of anchor points in the map is periodically evaluated, the credibility of an anchor point continuously observed is continuously improved, and a map point with a significant change in position is reduced in credibility or marked as changed.

[0014] Compared with the prior art, the present application has the following beneficial effects:

[0015] The present application creatively adopts the technical idea of actively selecting static structure semantic anchor points, solves the core problems of poor dynamic rejection effect and large computational overhead of the prior art through the collaborative design of lightweight semantic segmentation, spatiotemporal consistency verification, priority tracking and multi-sensor fusion. The accurate screening of static semantic anchor points ensures the reliability of SLAM calculation data, the lightweight semantic segmentation reduces the computational burden, the multi-module collaboration improves the positioning accuracy and stability in dynamic scenes, and the generated static map rich in semantics lays a foundation for high-level task decision-making, and is strong in universality. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0017] Figure 1 The flow chart of the dynamic SLAM method based on static semantic anchor points of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] Traditional SLAM techniques in dynamic scenes either rely on geometric or motion consistency to passively remove dynamic features, which cannot effectively distinguish static and dynamic features when dynamic objects are long-term stationary or slow-moving, leading to map pollution and positioning drift, or use full-pixel-level semantic segmentation to remove dynamic regions, which has huge computational overhead and is prone to mistakenly deleting static features, while wasting static features within dynamic objects, making it difficult to run in real time on embedded platforms.

[0020] Based on this, please refer to Figure 1 The embodiment provides a dynamic SLAM method based on static semantic anchor points, including the steps of sensor data acquisition, positioning estimation and map construction, and further comprising: a static structural semantic anchor point generation step for screening high-reliability feature points from original sensor data; a robust tracking and multi-sensor tight coupling fusion step based on anchor points for performing accurate pose estimation using the generated anchor points; a back-end optimization and static semantic map construction step for maintaining a globally consistent and semantically rich long-term map; the static structural semantic anchor point generation step, the robust tracking and multi-sensor tight coupling fusion step based on anchor points, and the back-end optimization and static semantic map construction step are executed in sequence to form a complete SLAM operation process.

[0021] The embodiment first screens high-reliability feature points from original sensor data through the static structural semantic anchor point generation step, which is different from the traditional passive dynamic feature removal approach, and instead actively selects feature points on inherent high-stability static structural elements in the environment to ensure that the data basis for subsequent calculations is reliable; then enters the robust tracking and multi-sensor tight coupling fusion step based on anchor points to perform pose estimation using the high-credibility anchor points selected in the early stage, avoiding the interference of dynamic features on positioning accuracy; finally, the back-end optimization and static semantic map construction step is used to maintain a globally consistent map while giving the map semantic information to improve the practicality of the map. The three steps are closely linked, covering the entire process from data input to map output, building a hierarchical decision and fusion framework to ensure that SLAM core calculations are always based on the most reliable data basis, fundamentally immunizing against the interference of dynamic objects. The technical effect is to maintain high positioning accuracy and stability in extremely high dynamic scenes, reduce computational overhead, generate a semantically rich long-term map, and adapt to multiple sensor types.

[0022] Existing SLAM solutions based on semantic segmentation mostly use high-precision but computationally intensive semantic segmentation models, requiring pixel-level processing of the entire image, which is difficult to run in real time on embedded platforms with limited computing resources, and do not optimize for the identification of key static structures in the SLAM scene, resulting in low efficiency in identifying static regions.

[0023] Based on this, the static structure semantic anchor generation step includes lightweight semantic segmentation and static region identification operation, inputting single-frame RGB or RGB-D image, using the improved MaskR-CNN structure semantic segmentation network with MobileNetV2 as the backbone network for real-time inference, the semantic segmentation network is pre-trained to identify static structures such as walls, columns, fixed shelves, door frames, and ground categories, and outputs a pixel-level semantic mask that identifies static structure regions.

[0024] This operation first determines that the input data is a single-frame RGB image or an RGB-D image, which adapts to the commonly used sensor data type of visual SLAM; in the selection of the semantic segmentation network, an improved MaskR-CNN structure with MobileNetV2 as the backbone network is used, the depth separable convolution design of MobileNetV2 can significantly reduce the network computation, meeting the real-time requirements of embedded platforms, and the improved MaskR-CNN structure can guarantee the accuracy of pixel-level segmentation, realizing accurate division of static structure regions; the network training stage is specially trained for high-stability static structure categories in the SLAM scene, including walls, columns, fixed shelves, door frames, and ground, which are the most common and immobile static elements in the environment, and can provide stable feature support for SLAM; the final output of the pixel-level semantic mask clearly identifies the regions in the image that belong to the above static structures, laying the foundation for subsequent feature extraction only in static regions and avoiding invalid feature extraction operations in dynamic regions. Technical effect is to greatly reduce the computational overhead of semantic segmentation while ensuring the accuracy of static region identification, meet the real-time operation requirements of embedded platforms, and accurately lock high-value static regions.

[0025] The existing SLAM feature extraction operation does not constrain the extraction range, often extracts a large number of invalid features in dynamic regions, increases the subsequent computational burden, and lacks geometric stability filtering of extracted feature points, resulting in some feature points being unable to be stably tracked due to their poor geometric properties, affecting the SLAM positioning accuracy.

[0026] Based on this, the static structure semantic anchor generation step also includes anchor feature extraction and preliminary screening operation, only in the static structure region identified by the semantic mask, local feature points are extracted using ORB algorithm, SIFT algorithm or lightweight convolution network, and the extracted feature points are filtered for geometric stability, low-curvature points with flat surfaces are selected in laser SLAM, and corner points that can be stably tracked in the image pyramid are selected in visual SLAM.

[0027] The operation firstly strictly limits the feature extraction range, only carries out local feature point extraction in the area identified by the static structure semantic mask of the lightweight semantic segmentation output, excludes feature points of dynamic areas from the source, and reduces the interference of invalid features on subsequent calculation; in the selection of feature extraction algorithm, three optional schemes of ORB algorithm, SIFT algorithm or lightweight convolution network are provided, the ORB algorithm has the characteristics of strong real-time, scale and rotation robustness, the SIFT algorithm is excellent in scale invariance and feature descriptor distinction, and the lightweight convolution network can extract more discriminative features through deep learning, and is suitable for different precision and real-time demand SLAM scene; after the feature extraction is completed, geometric stability filtering is carried out for different types of SLAM systems, in laser SLAM, low-curvature points with flat surfaces are selected, such points usually come from large-area static structures such as walls and floors, and have stable geometric properties and are not easy to change position due to object movement, in visual SLAM, corner points that can be stably tracked in the image pyramid are selected, such corner points are consistent at different scales, and can guarantee the stability of inter-frame matching. Technical effects are that the extracted feature points come from static areas and have good geometric stability, which provides high-quality candidate anchor points for subsequent SLAM calculation and reduces the calculation redundancy caused by invalid features.

[0028] The candidate anchor points obtained only through semantic segmentation and geometric stability screening lack spatio-temporal dimension correlation verification, and may exist false anchor points caused by single-frame image noise or instantaneous occlusion. If such anchor points are directly used for SLAM calculation, the positioning and mapping accuracy will be affected.

[0029] Therefore, the static structure semantic anchor point generation step further includes a spatio-temporal consistency verification operation, which associates the newly extracted candidate anchor points with anchor points that have been confirmed to be static and successfully tracked in the past several frames, verifies the consistency of the motion vector of the new anchor point with the camera motion estimation and the average motion mode of the surrounding stable anchor points, and verifies the rationality of the spatial relationship of the three-dimensional position of the new anchor point with the adjacent structure in the existing static map, and outputs a set of static structure semantic anchor points with a credibility score.

[0030] The operation first establishes a historical anchor database to store anchors in past frames that have been confirmed to be static and successfully implement inter-frame tracking, associates a newly extracted candidate anchor with historical anchors, and determines whether the candidate anchor has spatial continuity with historical static anchors. Then, motion consistency verification is performed, the optical flow or motion vector of the new anchor is calculated, and it is compared with the motion estimation result of the camera itself and the average motion pattern of surrounding anchors that have been confirmed to be stable. If the motion vector of the new anchor conforms to the motion law of static features, that is, the deviation from the camera motion estimation is within a predetermined range and consistent with the motion pattern of surrounding stable anchors, the motion consistency verification is passed. Subsequently, spatial consistency verification is performed. For a SLAM system equipped with an RGB-D or binocular camera, the three-dimensional position of the new anchor is calculated, and a spatial relationship judgment is performed with the adjacent static structure (such as the intersection of the wall and the ground, or the connection between the column and the ground) in the existing static map. If the three-dimensional position of the new anchor conforms to the spatial constraints of the adjacent static structure, for example, it is within the range of the wall and at a reasonable distance from the ground, the spatial consistency verification is passed. Finally, the results of the motion consistency and spatial consistency verifications are combined to assign a credibility score to each verified candidate anchor. The higher the score, the more reliable the static properties of the anchor. The final output is a set of static structure semantic anchors with credibility scores. The technical effect is to further improve the reliability of anchors and effectively exclude false anchors caused by noise and occlusion, providing high-credibility anchor input for subsequent SLAM core calculations.

[0031] The front-end tracking step of existing SLAM does not distinguish the reliability of feature points, and uses dynamic feature points, low-credibility static feature points, and high-credibility static feature points together for pose calculation, which is easily disturbed by dynamic feature points, leading to large deviations in initial pose estimation and affecting the accuracy of subsequent inter-frame matching, and even causing tracking failure.

[0032] Therefore, the anchor-based robust tracking and multi-sensor tight coupling fusion step includes a priority tracking strategy, which prioritizes the use of static structure semantic anchors for inter-frame matching and pose calculation in visual odometry or front-end tracking. The highest-credibility anchors are used to calculate the initial pose through PnP or ICP algorithms in stages, and other ordinary feature points are matched and outliers are removed based on the initial pose as a guide.

[0033] The strategy first establishes the priority of static structure semantic anchor points, and in the visual odometry or front-end tracking process, the static structure semantic anchor points are preferentially selected for inter-frame feature matching. Since such anchor points have been verified for spatio-temporal consistency, they have high reliability and stability, which can guarantee the accuracy of inter-frame matching. Then, a multi-stage optimization pose solving method is adopted. In the first stage, only the anchor points with the highest reliability are used for initial pose estimation. In visual SLAM, the PnP algorithm is used to solve the camera pose by matching multiple 3D points and their corresponding 2D image points. Since the input is high-reliability anchor points, the initial pose obtained by solving has high accuracy and strong robustness. In laser SLAM, the ICP algorithm is used to solve the pose of the laser sensor by matching the corresponding relationship of two sets of point clouds. Similarly, a reliable initial pose can be obtained based on high-reliability anchor points. In the second stage, the initial pose obtained in the first stage is used as prior information to guide the inter-frame matching of other ordinary feature points (i.e. feature points from static regions that are not screened by static structure semantic anchor points). The position of the ordinary feature points in the current frame is predicted based on the initial pose, the matching search range is reduced, and the matching efficiency is improved. At the same time, the matching results of the ordinary feature points are verified based on the initial pose, the reprojection error or point cloud distance error of the matching feature points is calculated, and the matching pairs with error exceeding the preset threshold are determined as outliers and removed. The technical effect is to greatly improve the robustness and pose estimation accuracy of the front-end tracking, avoid the interference of dynamic feature points and low-reliability feature points on the tracking process, and reduce the probability of tracking failure.

[0034] The SLAM system of a single sensor has obvious defects. Visual SLAM is prone to tracking loss when moving quickly or the image is blurred. Although the IMU sensor can provide motion information when moving quickly, it has the problem of cumulative drift. The existing multi-sensor fusion scheme does not fully utilize the constraint effect of high-reliability anchor points, and the accuracy and stability of the fusion result are insufficient.

[0035] Therefore, the anchor-based robust tracking and multi-sensor tightly coupled fusion step further includes a multi-sensor tightly coupled fusion operation. When the system is equipped with an IMU sensor, the visual pose calculated based on the anchor points and the pre-integration result of the IMU are tightly coupled and optimized. In the optimization framework, the information matrix weight of the observation value from the anchor points is higher than that of the instantaneous measurement value of the IMU.

[0036] The operation is first designed for a multi-sensor SLAM system equipped with an IMU sensor, after obtaining the visual pose calculated based on the static structure semantic anchor point, the pre-integration result of the IMU is calculated at the same time, the IMU pre-integration can convert the IMU measurement value of continuous multiple frames into the relative pose increment between adjacent frames, avoid re-integrating the IMU data every time, and improve the calculation efficiency; then a tightly coupled optimization framework is constructed, the visual pose and the IMU pre-integration result are taken as the observation values of the optimization variables, and are integrated into a unified optimization model, instead of simply weighted averaging the two kinds of pose results; in the optimization framework, the information matrix weight is set according to the reliability of the observation value, since the visual pose calculated based on the anchor point comes from high reliability static features, its reliability is much higher than that of the instantaneous measurement value of the IMU, therefore, the visual pose observation value from the anchor point is given higher information matrix weight, so that the optimization result is more biased towards the reliable visual pose; through the tightly coupled optimization, not only the motion constraint provided by the IMU data when the visual tracking is temporarily invalid can be used to avoid tracking loss, but also the high reliability visual pose can be used to suppress the cumulative drift of the IMU, and the accuracy of long-term pose estimation is improved. Technical effect is to combine the high reliability of visual anchor point and the fast response advantage of IMU, improve the robustness of SLAM system in fast motion or image blur scene, suppress the drift of IMU, and ensure the accuracy of long-term pose estimation.

[0037] The existing SLAM back-end graph optimization step treats all feature point generated pose constraint edges equally, without considering the reliability difference of different feature points, if there are a small number of dynamic feature points not completely removed by the front end, the constraint edges generated by them will have a negative impact on the global trajectory and map accuracy, leading to a decrease in global consistency.

[0038] Therefore, the back-end optimization and static semantic map construction step includes an anchor point reinforced back-end graph optimization operation, in the g2o or GTSAM based graph optimization back-end, anchor point pose constraint edges and ordinary feature point constraint edges are introduced, the anchor point pose constraint edges are generated by static structure semantic anchor points and are given higher weight, and the ordinary feature point constraint edges are generated by ordinary feature points extracted from other static regions and have lower weight.

[0039] The operation first selects a mature graph optimization framework as the basis, adopts g2o or GTSAM framework, which is widely used in SLAM backend optimization, has efficient sparse matrix solving ability, and can process large-scale pose and constraint data; in the graph optimization model, two different types of constraint edges are introduced, the first type is anchor point pose constraint edge, which is generated in the inter-frame matching process of static structure semantic anchor points, the feature points corresponding to this type of constraint edge are subjected to semantic segmentation, geometric screening and spatio-temporal consistency verification, and have very high reliability, so they are given higher information matrix weight, and have greater influence on the final pose and map results in the optimization process; the second type is ordinary feature point constraint edge, which is generated by ordinary feature points extracted from other static regions but not subjected to the complete static semantic anchor point screening process. The reliability of this type of constraint edge is lower than that of the anchor point pose constraint edge, so it is given a lower information matrix weight, and its influence on the optimization result is weakened. In the optimization solving process, the graph optimization framework will consider the weights of the two types of constraint edges, preferentially satisfy the anchor point pose constraint edge with high weight, and also consider the contribution of the ordinary feature point constraint edge. Even if there are a small number of error constraint edges generated by dynamic feature points not removed by the front end, due to their low weight, the influence on the global optimization result will also be minimized. Technical effects include guaranteeing the global trajectory accuracy and consistency of the SLAM system, effectively resisting the interference of a small number of residual dynamic feature points, and improving the robustness of the system. Figure 1

[0040] The existing SLAM system constructs a map that is a pure geometric map, only containing three-dimensional coordinate information of point clouds or feature points, lacks semantic attributes, and cannot provide support for high-level task decision-making of the robot, for example, the robot cannot identify static structures such as walls and columns through the pure geometric map, and it is difficult to realize semantic-based path planning or target navigation.

[0041] Therefore, the backend optimization and static semantic map construction step further includes a long-term static semantic map management operation. The constructed map contains geometric information of three-dimensional point clouds or feature points, and each point is associated with a corresponding semantic label to form a static semantic map. The semantic label includes wall points and column points.

[0042] ​The operation firstly synchronously records the geometric information and semantic information of the three-dimensional point cloud or feature points in the map construction process during the map construction process, the geometric information includes the three-dimensional coordinates of the points, the normal vector and the like, and ensures the spatial accuracy of the map; the semantic information is obtained by associating the results of the front-end semantic segmentation, in the front-end lightweight semantic segmentation step, the static structure categories to which each pixel in the image belongs are determined, the three-dimensional point cloud or feature points are corresponded with the image pixels, and each three-dimensional point is assigned with a corresponding semantic label; the specific semantic label includes wall point, column point, and can be extended to fixed shelf point, door frame point, ground point and the like according to the actual application scene, these semantic labels directly correspond to the static structures in the environment, and can clearly reflect the semantic properties of the map; the finally constructed static semantic map not only retains the spatial accuracy of the traditional geometric map, but also gives the map semantic information through the semantic label, and the robot can quickly locate the static structures such as "wall surface" and "column" by identifying the semantic label, and provide data support for high-level task decision. The technical effect is to generate a static map rich in semantic information, break through the functional limitation of the pure geometric map, provide support for the semantic navigation, path planning and other high-level tasks of the robot, and improve the practical value of the map.

[0043] The existing SLAM loop closure detection step mainly depends on the matching of the appearance features of the image, when the scene changes in light, season or local object occlusion, the appearance features will change significantly, resulting in loop closure detection failure, unable to realize the global consistency optimization of the map, and affecting the long-term mapping accuracy.

[0044] Therefore, the back-end optimization and static semantic map construction step further includes a loop closure detection operation, in which the key frames with the same semantic anchor point type are preferentially matched, and the semantic anchor point type includes a wall corner type.

[0045] The operation first changes the idea of relying on appearance features in traditional closed-loop detection, taking the static structure semantic anchor and its type as the core retrieval feature, and the type of the static structure semantic anchor is determined by its belonging to the static structure category and spatial form, for example, the anchor point of the wall corner type comes from the intersection part of the wall and the wall or the wall and the ground, and the semantic attribute of such anchor point is stable and is not affected by appearance changes such as light and season; in the closed-loop detection process, the system first retrieves the type of the static structure semantic anchor point contained in the key frame, and preferentially matches the key frame with the same semantic anchor point type, for example, the wall corner type anchor point contained in the current key frame is compared with the wall corner type anchor point in the historical key frame, if there are multiple semantic anchor points of the same type in the two key frames and their spatial distribution relationship is consistent, it is determined that there may be a closed loop; on the basis of matching the semantic anchor point type, the appearance feature is further verified to further improve the accuracy of closed-loop detection; since the semantic anchor point type has stability, even if the scene appearance changes, the semantic anchor points of the same type can remain consistent, which greatly improves the robustness of closed-loop detection. The technical effect is to improve the robustness of closed-loop detection under the scenes of light changes, seasonal changes and the like, to ensure the smooth progress of the optimization of the global consistency of the map, and to ensure the long-term mapping accuracy.

[0046] The existing SLAM system constructs a map, which is in a fixed state after completing the initial mapping, and cannot adapt to long-term changes in the environment, for example, fixed shelves in the environment are moved, walls are newly decorated, etc. These changes will cause the original map to be inconsistent with the actual environment, and if the original map is continued to be used for positioning and navigation, problems such as positioning drift and incorrect path planning will occur.

[0047] Therefore, the back-end optimization and static semantic map construction step further includes a map lifetime maintenance operation, which periodically evaluates the stability of the anchor points in the map, the credibility of the anchor points that are continuously observed is continuously improved, and the map points whose positions have changed significantly are reduced in credibility or marked as changed.

[0048] The operation first establishes a stability evaluation mechanism of the map anchor point, and the system will periodically re-observe and verify the anchor points in the map during subsequent operation, record the observation frequency and position change of the anchor points; for the anchor points that are continuously observed for a long time and have no significant position change, it is determined that the stability is high, the credibility score is continuously improved, and higher weight is given in subsequent SLAM calculation, and the constraint effect on positioning and mapping is strengthened; for the map points whose positions change significantly in multiple visits, even if they originally belong to semantic static points, it is determined that the stability is low, the credibility score is reduced, and if the position change exceeds the preset threshold, it is marked as changed, and its influence on positioning and mapping is reduced or excluded in subsequent calculation; at the same time, the system will supplement new semantic anchor points to the map according to the newly observed stable static structure, realize the update and expansion of the map; through this dynamic maintenance mechanism, it is ensured that the map is always consistent with the actual environment, and the map failure caused by environmental changes is avoided.

[0049] The technical effect is to realize adaptive update and lifelong maintenance of the map, ensure long-term effectiveness of the map, and enable the SLAM system to continuously adapt to environmental changes and maintain stable positioning and navigation performance.

[0050] In this paper, specific examples are applied to explain the principles and implementation modes of the application, and the above examples are only used to help understand the method and its core idea. The above description is only the preferred embodiment of the application. It should be pointed out that due to the limitation of language expression, there are infinite specific structures, and for ordinary technical personnel in the technical field, some improvements, decorations or changes can be made without departing from the principles of the application, and the above technical features can be combined in an appropriate way; these improvements, decorations, changes or combinations, or the application of the inventive concept and technical scheme to other occasions without improvement, shall be regarded as the protection scope of the application.

Claims

1. A dynamic SLAM method based on static semantic anchors, comprising the steps of sensor data acquisition, localization estimation, and map construction, characterized in that, Also includes: The SLAM computation process consists of three steps: a static structural semantic anchor point generation step, which selects high-reliability feature points from the raw sensor data; a robust tracking and multi-sensor tightly coupled fusion step based on anchor points, which performs accurate pose estimation using the generated anchor points; and a backend optimization and static semantic map construction step, which maintains a globally consistent and semantically rich long-term map. These steps are executed sequentially to form a complete SLAM computation flow. The static structural semantic anchor point generation step includes lightweight semantic segmentation and static region recognition operations, inputting a single-frame RGB or RGB-D image and using an improved MaskR-2 architecture with MobileNetV2 as the backbone network. A CNN-structured semantic segmentation network performs real-time inference. In visual odometry or front-end tracking, static structural semantic anchors are prioritized for inter-frame matching and pose calculation. The optimization is phased, first using the anchor with the highest confidence to calculate the initial pose using the PnP or ICP algorithm. Then, using this initial pose as a guide, other ordinary feature point matching is guided and verified, and outliers are removed. The back-end optimization and static semantic map construction steps include anchor-strengthened back-end graph optimization operations. In the graph optimization back-end based on g2o or GTSAM, anchor pose constraint edges and ordinary feature point constraint edges are introduced. The anchor pose constraint edges are generated by static structural semantic anchors and given higher weights, while the ordinary feature point constraint edges are generated by ordinary feature points extracted from other static regions and have lower weights. The semantic segmentation network is pre-trained to identify static structures such as walls, pillars, fixed shelves, door frames, and floors, and outputs pixel-level semantic masks that identify static structure regions. The static structure semantic anchor point generation step also includes anchor point feature extraction and preliminary screening operations. Local feature points are extracted only within the static structure area identified by the semantic mask using the ORB algorithm, SIFT algorithm, or lightweight convolutional network. Geometric stability filtering is performed on the extracted feature points. In laser SLAM, low curvature points with flat surfaces are selected, and in visual SLAM, corner points that can be stably tracked in the image pyramid are selected. The static structural semantic anchor point generation step also includes a spatiotemporal consistency verification operation, which associates the newly extracted candidate anchor points with the confirmed static and successfully tracked anchor points in the past few frames, verifies the consistency between the motion vector of the new anchor point and the camera's own motion estimation and the average motion mode of the surrounding stable anchor points, verifies the rationality of the spatial relationship between the three-dimensional position of the new anchor point and the adjacent structures in the existing static map, and outputs a set of static structural semantic anchor points with a confidence score.

2. The dynamic SLAM method based on static semantic anchors according to claim 1, characterized in that, The anchor-based robust tracking and multi-sensor tight-coupled fusion step also includes a multi-sensor tight-coupled fusion operation. When the system is equipped with an IMU sensor, the visual pose calculated based on the anchor point is tightly coupled and optimized with the pre-integration result of the IMU. In the optimization framework, the observations from the anchor point are assigned information matrix weights that are higher than the instantaneous measurements of the IMU.

3. The dynamic SLAM method based on static semantic anchors according to claim 1, characterized in that, The backend optimization and static semantic map construction steps also include long-term static semantic map management operations. The constructed map contains geometric information of three-dimensional point clouds or feature points. Each point is associated with a corresponding semantic label to form a static semantic map. The semantic labels include wall points and columnar points.

4. The dynamic SLAM method based on static semantic anchors according to claim 1, characterized in that, The backend optimization and static semantic map construction steps also include a loop closure detection operation, in which key frames with the same semantic anchor type are matched first, and the semantic anchor type includes corner type.

5. The dynamic SLAM method based on static semantic anchors according to claim 4, characterized in that, The backend optimization and static semantic map construction steps also include lifelong map maintenance operations, periodically evaluating the stability of anchor points in the map, continuously increasing the credibility of continuously observed anchor points, and reducing the credibility of map points whose positions have changed significantly or marking them as changed.

Images