Positioning method and device based on SLAM (Simultaneous Localization and Mapping) and electronic equipment
By employing the Z-score dynamic threshold displacement constraint method to filter keyframes in the SLAM system, the problem of poor keyframe selection effect was solved, and efficient localization and mapping in dynamic environments were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-03-31
AI Technical Summary
In existing technologies, keyframe selection is not effective and cannot balance accuracy and real-time performance, especially in static or slowly changing scenes and dynamic scenes.
A Z-score-based dynamic threshold displacement constraint method is adopted, which combines the dynamic mean of the cumulative pixel error rate between frames and the dynamic mean of the cumulative pixel error rate between reference key frames to screen key frames. In addition, local mapping and loop closure detection are combined to remove abnormal error frames caused by sudden changes in illumination.
It improves the positioning accuracy and real-time performance of the SLAM system in dynamic environments, reduces positioning drift, and enhances the system's robustness and mapping accuracy.
Smart Images
Figure CN121761863A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of spatial positioning technology, and in particular to a positioning method, apparatus and electronic device based on SLAM. Background Technology
[0002] Simultaneous Localization and Mapping (SLAM), a key capability for robots to autonomously locate and navigate in unknown environments, has received considerable attention. SLAM acquires data through sensors such as cameras, LiDAR, and IMUs, enabling mobile devices to simultaneously build a map of the environment and determine their own position within that map. It is now widely used in various fields such as autonomous driving, intelligent robots, and drones.
[0003] SLAM utilizes sensors to collect raw environmental data, providing raw materials for subsequent processing. Visual odometry estimates the relative motion trajectory of the camera in real time through continuous image frames, providing a short-term, locally consistent pose sequence. Back-end optimization globally optimizes the drifting initial trajectory output by visual odometry, eliminating accumulated errors and improving the global consistency of the trajectory. Loop closure detection checks whether the camera returns to previously visited areas, constructing loop closure constraints to eliminate long-term accumulated drift errors and ensure global trajectory consistency. These five parts work together, supporting each other and forming a fault-tolerant buffer, allowing traditional visual SLAM to maintain a certain level of reliability in static, ideal environments, completing basic simultaneous localization and mapping. The quality of keyframes is closely related to the mapping accuracy.
[0004] In existing technologies, motion pattern-based keyframe selection algorithms are suitable for both dynamic and static scenes, but perform poorly in static or slowly changing scenes. Appearance-based keyframe selection methods work well for static or low-motion videos, highlighting visual changes, but they ignore motion information, resulting in poor keyframe selection in dynamic scenes. Data association-based methods analyze the similarity and information content changes between frames, calculate inter-frame similarity, identify the correlation between frames, and select frames with low correlation to preceding and following frames but high information content as keyframes. These methods are applicable to different types of videos and can adaptively select keyframes, but they can affect the real-time performance of SLAM to some extent. Summary of the Invention
[0005] This invention provides a SLAM-based localization method, apparatus, and electronic device to address the problem of poor keyframe selection and inability to balance accuracy and real-time performance in existing technologies.
[0006] In a first aspect, embodiments of the present invention provide a SLAM-based localization method, comprising: Get the reference keyframe of the current frame and the previous frame of the current frame; Based on the current frame and the previous frame, determine the dynamic mean of the inter-frame cumulative pixel error rate of the current frame; Based on the current frame and the reference keyframe of the current frame, determine the dynamic mean of the cumulative pixel error rate between the reference keyframes of the current frame; Based on the dynamic mean of the inter-frame cumulative pixel error rate of the current frame and the dynamic mean of the inter-frame cumulative pixel error rate of the reference key frame of the current frame, the Z-score dynamic threshold displacement constraint method is used to determine whether the current frame is a key frame. If so, the current frame is added to the keyframe row and column, and local mapping and loop closure detection are performed based on the keyframe row and column.
[0007] Secondly, embodiments of the present invention provide a SLAM-based positioning device, comprising: The parameter acquisition module is used to obtain the reference keyframe of the current frame and the previous frame of the current frame; The first dynamic mean calculation module is used to determine the dynamic mean of the inter-frame cumulative pixel error rate of the current frame based on the current frame and the previous frame. The second dynamic mean calculation module is used to determine the dynamic mean of the cumulative pixel error rate between the reference keyframes of the current frame based on the current frame and the reference keyframes of the current frame. The keyframe determination module is used to determine whether the current frame is a keyframe based on the dynamic mean of the inter-frame cumulative pixel error rate of the current frame and the dynamic mean of the inter-frame cumulative pixel error rate of the reference keyframes of the current frame, combined with the Z-score dynamic threshold displacement constraint method. The keyframe application module is used to add the current frame to the keyframe row and column if the condition is met, and to perform local mapping and loop closure detection based on the keyframe row and column.
[0008] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the SLAM-based localization method as described in the first aspect or any possible implementation of the first aspect.
[0009] This invention provides a SLAM-based localization method, apparatus, and electronic device. The SLAM-based localization method includes: acquiring a reference keyframe of the current frame and the previous frame; determining the dynamic mean of the inter-frame cumulative pixel error rate of the current frame based on the current frame and the previous frame; determining the dynamic mean of the inter-reference keyframe cumulative pixel error rate of the current frame based on the current frame and the reference keyframe; determining whether the current frame is a keyframe based on the dynamic mean of the inter-frame cumulative pixel error rate and the dynamic mean of the inter-reference keyframe cumulative pixel error rate, combined with the Z-score dynamic threshold displacement constraint method; if so, adding the current frame to the keyframe row and performing local mapping and loop closure detection based on the keyframe row. This invention can comprehensively capture pixel-level changes between frames and between frames and keyframes, avoiding the bias caused by single error judgments, and can promptly detect errors caused by minute camera displacements or attitude changes. Meanwhile, by combining the Z-score dynamic threshold displacement constraint method to screen key frames, compared with a fixed threshold, it can adapt to the differences in error distribution under different scenarios, effectively remove abnormal error frames caused by sudden changes in illumination, temporary occlusion, etc., avoid such frames from interfering with map construction as key frames, thereby reducing positioning drift and improving the effectiveness and accuracy of subsequent data processing. Attached Figure Description
[0010] Figure 1 This is a schematic diagram of some SLAM technology applications; Figure 2 This is a flowchart illustrating the implementation of a SLAM-based localization method provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of inter-frame tracking and key inter-frame tracking provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the LK optical flow method in the prior art; Figure 5 This is a schematic diagram of the pyramid-LK optical flow method in the prior art; Figure 6 This is a schematic diagram of keyframe selection provided in an embodiment of the present invention; Figure 7 This is the overall architecture of existing SLAM systems; Figure 8 This is an overall architecture of a SLAM system provided in an embodiment of the present invention; Figure 9 This is a structural diagram of the GCBlock network in the existing technology; Figure 10 This is a structural diagram of a PGCBlock network provided in an embodiment of the present invention; Figure 11 A structural diagram of the ShuffleNetV2 network in the prior art; Figure 12 This is a PGC-Shu provided in the embodiments of the present invention. Structure diagram of the eNetV2 network; Figure 13 This is a structural diagram of the YOLO-FastestV2 network in the prior art; Figure 14 This is a structural diagram of a PGC-Yolo-Fastestv2 model provided in an embodiment of the present invention; Figure 15 This is a schematic diagram of a detection frame motion method provided in an embodiment of the present invention; Figure 16 This is a comparison chart of the mapping accuracy between ORB-SLAM3 provided in the embodiments of the present invention and the method of this application; Figure 17 This is a schematic diagram of the SLAM-based positioning device provided in an embodiment of the present invention; Figure 18 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation
[0011] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0012] SLAM is now widely used in many fields such as autonomous driving, intelligent robots, and drones. For example... Figure 1 The demonstration showcased some applications of SLAM technology: cars using SLAM for autonomous driving, home service robots using SLAM for path planning and obstacle avoidance, and drones using SLAM for forest exploration.
[0013] In SLAM systems, keyframes are used for camera pose estimation and optimization, and their quality directly affects positioning accuracy. Appropriate selection strategies and optimization methods can improve the overall system performance, making it more stable and efficient in complex environments. However, while adding criteria for keyframe selection can improve mapping accuracy, it often overlooks the impact of redundant keyframes on the real-time performance of the SLAM system. Balancing real-time performance with accuracy has become a bottleneck in improving the performance of SLAM systems in dynamic environments.
[0014] See Figure 2 The diagram illustrates a flowchart of a SLAM-based localization method provided by an embodiment of the present invention, detailed below: The above-mentioned SLAM-based localization method includes: refer to Figure 3Inter-frame tracking refers to matching each frame with the previous frame to find similar regions between the two frames, and then calculating the positional change of each pixel in the current frame. This reduces errors caused by lighting variations and improves the stability of optical flow. Keyframe tracking, on the other hand, selects only important keyframes for matching, ignoring less important intermediate frames. While this method can further reduce errors, it may also lead to information loss or omission, hindering the understanding and analysis of the entire video sequence. This application comprehensively considers inter-frame tracking and keyframe tracking, calculating the dynamic mean of the cumulative pixel error rate as a criterion for selecting keyframes. Inter-frame tracking P and keyframe tracking Q are, as shown... Figure 3 The diagram shows inter-frame tracking and key inter-frame tracking.
[0015] S101: Obtain the reference keyframe of the current frame and the previous frame of the current frame; First, a reference keyframe is determined. There may be multiple keyframes preceding the current frame. This application calculates the correlation between the current frame and any keyframe preceding it, and selects the keyframe with the highest correlation as the reference keyframe for the current frame.
[0016] S102: Determine the dynamic mean of the inter-frame cumulative pixel error rate of the current frame based on the current frame and the previous frame. In one possible implementation, S102 may include: S1021: The average pixel error between the current frame and the previous frame is calculated using the pyramid LK optical flow method, and is used as the inter-frame average pixel error of the current frame. Optical flow is a core technique in computer vision used to describe the motion patterns of pixels in an image sequence. Essentially, it analyzes the grayscale changes of pixels in consecutive frames to calculate the pixel's motion vector in a spatial coordinate system—the optical flow vector—thereby inferring the motion state of a target or scene. It is widely used in target tracking, action recognition, video stabilization, and autonomous driving. A schematic diagram of the LK optical flow method is shown below. Figure 4 As shown.
[0017] Traditional LK optical flow methods assume small motion amplitudes between two frames. When the object's displacement is large between frames, the algorithm exhibits significant errors. To track larger motion amplitudes, this application employs a pyramid-LK optical flow method. This method decomposes the original image layer by layer using a pyramid structure, tracking everything from the lowest to the highest detail, resulting in more robust performance. An overview diagram of the pyramid-LK optical flow method is shown below. Figure 5 As shown.
[0018] Set the current frame for The previous frame was The average pixel error between frames in the current frame is calculated using the pyramid LK optical flow method.
[0019] In one possible implementation, S1021 may include: 1. The pyramid LK optical flow method is used, combined with the first formula, to determine the average pixel error between frames in the current frame; The first formula may include:
[0020] in, This represents the average pixel error between frames in the current frame. For the current frame, The previous frame of the current frame. and Each pixel is in and Speed of motion on the axis This represents the total number of pixels in the current frame.
[0021] The inter-frame average pixel error between the current frame and the previous frame is calculated based on the first formula and used to determine the inter-frame global average pixel error of the current frame.
[0022] S1022: Determine the global average pixel error between frames in the current frame based on the average pixel error between frames in the current frame. In one possible implementation, S1022 may include: 1. Determine the global average pixel error between frames of the current frame based on the average pixel error between frames of the current frame and the second formula. The second formula may include:
[0023] in, This represents the inter-frame global average pixel error for the current frame. This represents the total number of frames including the current frame. For the first The average pixel error between frames. The global average pixel error between frames in the current frame is the average of the sum of the average pixel errors between frames in each frame.
[0024] S1023: Use the global average pixel error between frames of the current frame as the error threshold to determine the cumulative pixel error rate between frames of the current frame. In one possible implementation, S1023 may include: 1. Use the global average pixel error between frames of the current frame as the error threshold, and combine it with the third formula to determine the cumulative pixel error rate between frames of the current frame; The third formula may include:
[0025] in, The cumulative pixel error rate between frames for the current frame. This is an indicator function that takes the value 1 when the condition is true and 0 otherwise.
[0026] Will As an error threshold, the error exceeds... The pixels are the error pixels. Substituting them into the above formula, we can calculate the inter-frame cumulative pixel error rate of the current frame. .
[0027] Inter-frame cumulative pixel error rate of the current frame As a preliminary screening criterion, it can intuitively reflect the intensity and range of inter-frame changes. In order to adapt to the dynamic environment and reduce noise interference, a dynamic mean enhancement algorithm is adopted to reduce the response to changes in illumination. The dynamic mean of the cumulative pixel error rate between frames in the current frame is determined as the screening criterion to improve the robustness of the algorithm.
[0028] S1024: Determine the dynamic mean of the inter-frame cumulative pixel error rate of the current frame based on the inter-frame cumulative pixel error rate and the inter-frame average pixel error of the current frame.
[0029] In one possible implementation, S1024 may include: 1. Based on the cumulative inter-frame pixel error rate and the average inter-frame pixel error of the current frame, and combined with the fourth formula, determine the dynamic mean of the cumulative inter-frame pixel error rate of the current frame. The fourth formula may include:
[0030] in, This is the dynamic mean of the inter-frame cumulative pixel error rate for the current frame. This is the dynamic average of the cumulative pixel error rate between the current frame and the previous frame.
[0031] This application uses the average pixel error between frames as a smoothing coefficient. The more frequently the lighting changes in the environment, the larger the smoothing coefficient, and the more sensitive the mean is to new changes, making it easier to identify frames with strong lighting changes.
[0032] The dynamic mean of the inter-frame cumulative pixel error rate of the current frame is related to the dynamic mean of the inter-frame cumulative pixel error rate of the previous frame. Therefore, during system initialization, the current frame may not have a previous frame or a reference keyframe. Let the initial value of the previous 10 frames be... ,in , S103: Determine the dynamic average of the cumulative pixel error rate between the reference keyframes of the current frame based on the current frame and the reference keyframe of the current frame; Based on the above, the same method is used to determine the dynamic mean of the cumulative pixel error rate between reference keyframes for the current frame.
[0033] In one possible implementation, S103 may include: S1031: Determine the average pixel error between the current frame and the reference keyframe of the current frame, and use it as the average pixel error between the reference keyframes of the current frame. Specifically, let the current frame be... for The reference keyframe is The formula for calculating the average pixel error between reference keyframes of the current frame is as follows:
[0034] in, The average pixel error between reference keyframes for the current frame. For the current frame, This serves as the reference keyframe for the current frame. and Each pixel is in and Speed of motion on the axis This represents the total number of pixels in the current frame.
[0035] S1032: Determine the global average pixel error of the reference keyframes of the current frame based on the average pixel error between reference keyframes of the current frame. The global average pixel error of the reference keyframe for the current frame is determined using the following formula:
[0036] in, The global average pixel error of the reference keyframe for the current frame. This represents the total number of frames including the current frame. For the first The average pixel error between keyframes of a frame. The global average pixel error of the reference keyframe for the current frame is the average of the average pixel errors between keyframes of each frame.
[0037] S1033: Use the global average pixel error of the reference keyframe of the current frame as the error threshold to determine the cumulative pixel error rate between the reference keyframes of the current frame. The formula for calculating the cumulative pixel error rate between reference keyframes of the current frame is as follows:
[0038] in, The cumulative pixel error rate between reference keyframes for the current frame. This is an indicator function that takes the value 1 when the condition is true and 0 otherwise.
[0039] Will As an error threshold, the error exceeds... The pixels are the error pixels. Substituting them into the above formula, we can calculate the cumulative pixel error rate between the reference keyframes of the current frame. .
[0040] S1034: Determine the dynamic mean of the cumulative pixel error rate between reference keyframes of the current frame based on the cumulative pixel error rate between reference keyframes of the current frame and the average pixel error between reference keyframes of the current frame.
[0041] Similarly, As an adjustment for the smoothing coefficient, the calculation formula is as follows:
[0042] in, The dynamic mean of cumulative pixel error rate between reference keyframes.
[0043] S104: Based on the dynamic mean of the inter-frame cumulative pixel error rate of the current frame and the dynamic mean of the inter-frame cumulative pixel error rate of the reference key frame of the current frame, combine the Z-score dynamic threshold displacement constraint method to determine whether the current frame is a key frame. In one possible implementation, S104 may include: S1041: Based on the dynamic mean of the inter-frame cumulative pixel error rate of the current frame and the dynamic mean of the inter-frame cumulative pixel error rate of the reference key frame of the current frame, and in conjunction with the fifth formula, determine whether the current frame is a candidate frame. S1042: If so, the Z-score dynamic threshold displacement constraint method is used, combined with the sixth formula, to determine whether the current frame is a key frame; The fifth formula may include:
[0044] in, This is the dynamic mean of the inter-frame cumulative pixel error rate for the current frame. This represents the cumulative pixel error rate between frames for the current frame. This is the dynamic mean of the cumulative pixel error rate between reference keyframes for the current frame. The cumulative pixel error rate between reference keyframes for the current frame; This indicates that the current frame is a candidate frame. This indicates that the current frame is not a candidate frame; The sixth formula may include:
[0045]
[0046] in, This represents the Lie algebraic distance between the current frame and its reference keyframe. This represents the number of all keyframes preceding the current frame. The standard deviation of the historical frame shift. This is the sensitivity coefficient. It is the minimum value; This indicates that the current frame is a keyframe. This indicates that the current frame is not a keyframe. Reflects the degree of displacement fluctuation; Adjust dynamically based on the scenario; the more stable the scenario, the better. The smaller it is, the larger it is.
[0047] If the sum of the cumulative pixel error rate between frames in the current frame and the cumulative pixel error rate between reference keyframes in the current frame is greater than the sum of the dynamic mean of the cumulative pixel error rate between frames in the current frame and the dynamic mean of the cumulative pixel error rate between reference keyframes, then the frame is selected as a candidate frame. To avoid excessive redundant keyframes due to continuous illumination changes when using the optical flow method for keyframe selection, displacement constraints are introduced to limit the keyframe selection conditions of the optical flow method. Fixed threshold methods perform poorly under dynamic environments or noise interference; therefore, this application uses the Z-score threshold method from statistics to design dynamic displacement constraints, optimize keyframe selection, and improve mapping accuracy.
[0048] If the Lie algebraic distance between the current frame and the reference keyframe exceeds a set threshold, a candidate frame is selected as the keyframe. This displacement constraint method prevents the LK optical flow method from selecting consecutive and closely spaced non-keyframes as keyframes. A selection diagram is shown below. Figure 6 As shown, this application can significantly reduce redundant keyframes, reduce computational load, and improve the real-time performance of the system.
[0049] S105: If so, add the current frame to the keyframe row and column, and perform local mapping and loop closure detection based on the keyframe row and column.
[0050] The above method is repeated for each frame of the image to obtain keyframe rows and columns for subsequent data processing by the user.
[0051] Furthermore, this method can coexist with the original keyframe selection procedure of the SLAM system. For each frame removed by the original procedure, the methods S101 to S105 are executed respectively to re-add the selected keyframes for localization, providing more pose information and improving the mapping accuracy of the system.
[0052] Figure 7The overall architecture of the SLAM system is shown. Traditional visual SLAM algorithms are built upon assumptions about a static environment, which are correct in certain specific scenarios. However, in environments with moving objects, traditional SLAM becomes very vulnerable. A large number of dynamic points lead to incorrect data associations, resulting in significant trajectory errors and even loss of tracking status, thus requiring dynamic target removal. While existing object detection models can guarantee the real-time performance of the SLAM system, their object detection accuracy is low. Detection boxes typically contain static feature points, and the erroneous deletion of many static feature points within the detection boxes affects the system's robustness.
[0053] Based on this, refer to Figure 8 This application provides an improved SLAM system that forms a PGC-Yolo-Fastestv2 network for target detection.
[0054] Specifically: 1. Establish the PGCBlock network.
[0055] GCBlock (Global Context Block) is a lightweight global context module that enhances the model's ability to capture global contextual information through global average pooling and channel attention mechanisms. It has low computational cost and parameter count, making it easy to integrate into existing networks, and it can significantly improve the model's detection accuracy while maintaining high inference speed. This aligns well with the real-time requirements of SLAM systems. Its network structure diagram is shown below. Figure 9 .
[0056] The PGCBlock network extends the channel-dimensional attention mechanism of GCBlock to each feature point. In Cntext Modeling, after Softmax, it transforms the channels into 1×H×W, with each (C, H, W) position having independent attention weights, resulting in an output attention matrix of the same size as the input feature map. Being lightweight and designed for Computer Vision (CV) tasks, it improves upon LayerNorm in Transform, suitable for Natural Language Processing, by replacing it with BatchNorm (BN), which provides smoother cross-sample statistics and accelerates convergence more effectively in CV. It calculates the mean and variance for each channel across all spatial locations of all samples within a batch before normalization. The PGCBlock (Pixel Global Context Block) network structure is as follows: Figure 10 As shown.
[0057] 2. A lightweight global context module, PGCBlock, is inserted into the backbone network of ShuffleNetV2 to propose the PGC-ShuffleNetV2 model, which enhances the basic feature representation and extraction capabilities, while helping the network to better combine global information to focus on key regions.
[0058] ShuffleNetV2, as the backbone network of the real-time object detection model YOLO-FastestV2, is responsible for image feature extraction. The basic unit of eNetV2 does not change the feature map size, only performing channel shuffling and feature transformation. Its feature input is split into two branches. The main branch uses 1×1 convolutions to reduce the dimensionality, decreasing the number of channels and reducing computation for 3×3 depthwise convolutions. It then uses another 1×1 convolution to increase the dimensionality back to the original number of channels, and concatenates this with the identity mapping of the other branch for channel shuffling. The ShuffleNetV2 unit, used for spatial downsampling, achieves downsampling through 3×3 convolutions with a stride of 2. Combined with increased network depth, this expands the receptive field, allowing higher-level features to cover a larger area of the input image. However, this compresses the spatial dimension, halving the feature map size and causing some loss of detail. Because more channels can encode richer semantic features, doubling the number of channels compensates for the information loss caused by spatial compression, maintaining the network's feature representation ability. Both branches contain 3×3 depthwise convolutions, and the two branches are concatenated for channel shuffling, achieving the downsampling goal of doubling the number of channels. The ShuffleNetV2 network structure is as follows: Figure 11 As shown.
[0059] PGC-Shu After the eNetV2 basic unit completes feature transformation in the main branch, a PGCBlock attention module is added before the output. By modeling global context relationships, it assigns dynamic weights to features in different channels, allowing the network to focus more on features more important to the task, enhancing key features and weakening irrelevant information. In the PGC-ShuffleNetV2 unit used for spatial downsampling, both branches of the downsampling unit are responsible for feature transformation. However, to avoid redundancy caused by repeated enhancement of the same feature due to dual attention, or even loss of useful information due to over-focusing on certain channels, the PGCBlock attention module is added only to the main branch. This more effectively enhances key features, while the projection branch is mainly used to retain the downsampled version of the original features, ensuring feature continuity. eNetV2 network structure as follows Figure 12 As shown.
[0060] 3. Replace ShuffleNetV2 in the Backbone part of the YOLO-FastestV2 model with PGC-ShuffleNetV2, and add the GCBlock attention module to light-FPN to improve the accuracy of YOLO-FastestV2 in detecting targets.
[0061] The YOLO-FastestV2 network's backbone abandons the DarkNet used in traditional YOLO, adopting an improved ShuffleNetV2 as the main network. Its core advantages are computational efficiency and memory-friendly operation. The YOLO-FastestV2 network's light-FPN (Feature Pyramid Network) is a lightweight feature fusion module that addresses the difficulty of detecting small objects in object detection through multi-scale feature fusion. It is designed to overcome the computational limitations of edge devices, significantly reducing computation and parameter size while maintaining multi-scale feature representation capabilities, providing high-quality features for rapid head detection. The head consists of three components: classification, regression, and detection. These three components work on the same feature map, avoiding redundant feature computation, and use independent 1×1 convolutional outputs to ensure task specificity. The YOLO-FastestV2 network is as follows: Figure 13 As shown.
[0062] The ShuffleNetV2 model in the Yolo-FastestV2 model is replaced with PGC-ShuffleNetV2, and an attention module GCBlock 1 is added after dimensionality reduction of the generated semantic features to avoid losing some key semantics after dimensionality reduction. The multi-scale feature concatenation after light-FPN is a crucial node for the fusion of details and semantics; the addition of the GCBlock 2 attention module enhances cross-scale complementary channels without destroying details, solving the problem of channel redundancy (i.e., duplicated information in some channels) caused by the sudden increase in the number of channels. This provides higher-quality input features for subsequent shallow detection heads. The structure of the PGC-Yolo-Fastestv2 model is as follows: Figure 14 As shown.
[0063] Based on the above, the PGC-Yolo-Fastestv2 model was obtained. This model has a 2% higher detection accuracy than Yolo-Fastestv2, but the number of parameters only increases by 0.065M.
[0064] Furthermore, based on Figure 8 By combining the PGC-Yolo-Fastestv2 system with the ORB-SLAM3 system, dynamic target detection is performed on images that have already undergone ORB feature point extraction.
[0065] A separate sub-thread is created for GC-Yolo-Fastestv2 to avoid the detection time affecting the real-time performance of the SLAM main thread. While SLAM receives image frames for feature extraction, GC-Yolo-Fastestv2 performs dynamic object detection on the original image frames, generating initial dynamic object detection boxes. Then, combined with the image from which ORB feature points have already been extracted, the positions of the detection boxes are marked on the image containing the feature points, completing the dynamic feature point initialization marking.
[0066] In one possible implementation, this application employs a detection box motion method, which compares the displacement of the dynamic detection box in the current frame with that in the previous frame to determine whether the detection box region is actually moving, and removes points located within the dynamic region that have obvious regional movement.
[0067] When the camera moves, both static and dynamic feature points in consecutive image frames have inter-frame displacement distances relative to the previous frame. This is their common base displacement. If the displacement of a dynamic feature point is greater than the base displacement, it indicates that the dynamic feature point is moving (because some dynamic targets may not actually be moving; for example, a person sitting still in a chair is still a detected dynamic target). A schematic diagram of the detection box motion method is shown below. Figure 15 As shown, a1 and a2 represent the center points of detection boxes A1 and A2, respectively, and λ is the Euclidean distance between the center points of the two detection boxes. When λ is greater than the average Lie algebra distance of the historical frames, it means that the inter-frame displacement + target region displacement > inter-frame displacement, that is, the target region displacement > 0, indicating that the detection box region is in motion. At this time, the feature points in the corresponding detection box A2 of the current frame are removed.
[0068] Finally, the static feature point set after removing dynamic feature points is used for initialization and relocation, which improves the robustness of the SLAM system in dynamic environments and the accuracy of subsequent mapping.
[0069] The core logic of SLAM initialization and relocalization is to calculate spatial relationships through the geometric constraints of feature points across different frames. Static feature points follow rigid body motion laws across different frames, meaning their position changes are caused only by camera motion. Their geometric constraints are stable and reliable, providing accurate spatial references for initialization and relocalization.
[0070] The positional changes of dynamic feature points include not only camera motion but also their own motion, which can disrupt the consistency of geometric constraints (for example, the displacement of a dynamic point between two frames may be much greater than the displacement caused by camera motion). This leads to errors in the estimation of the fundamental and essential matrices during initialization, and incorrect geometric relationships can cause drift in the initial camera pose estimation or even initialization failure. By removing dynamic points and calculating the initial parameters using only the constraints of static points, the accuracy of initialization can be guaranteed.
[0071] Relocalization relies on matching features in the current frame with features in the map. Dynamic points may not exist in the map or their positions may have changed, leading to incorrect matches and causing relocalization failures or pose jumps. A large number of relocalization failures can lead to map construction failures. By removing dynamic points, matching only occurs between static points, significantly reducing the false match rate and improving the robustness of relocalization.
[0072] SLAM map building relies on camera pose and feature point triangulation, and the accuracy of the camera pose directly affects the map accuracy. If initialization and relocalization result in pose errors due to dynamic point interference, all subsequent map point triangulation based on that pose will accumulate errors, leading to map distortion (such as ghosting or inconsistent scale in point clouds of static objects). Removing dynamic points results in more accurate poses during initialization and relocalization. Maps built on this basis can realistically reflect the structure of the static environment, avoiding the "contamination" of the map by dynamic objects (such as avoiding false point clouds containing moving pedestrians). This improves system robustness while maintaining real-time performance, enabling SLAM systems to complete tracking and mapping in highly dynamic environments. (See the map accuracy comparison chart for reference.) Figure 16 .
[0073] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0074] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.
[0075] Figure 17 A schematic diagram of the SLAM-based positioning device provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below: like Figure 17 As shown, the SLAM-based positioning device includes: The parameter acquisition module 21 is used to acquire the reference keyframe of the current frame and the previous frame of the current frame; The first dynamic mean calculation module 22 is used to determine the dynamic mean of the inter-frame cumulative pixel error rate of the current frame based on the current frame and the previous frame of the current frame. The second dynamic mean calculation module 23 is used to determine the dynamic mean of the cumulative pixel error rate between the reference keyframes of the current frame based on the current frame and the reference keyframes of the current frame. The keyframe determination module 24 is used to determine whether the current frame is a keyframe based on the dynamic mean of the inter-frame cumulative pixel error rate of the current frame and the dynamic mean of the inter-frame cumulative pixel error rate of the reference keyframe of the current frame, combined with the Z-score dynamic threshold displacement constraint method. The keyframe application module 25 is used to add the current frame to the keyframe row and column if the condition is met, and to perform local mapping and loop closure detection based on the keyframe row and column.
[0076] In one possible implementation, the first dynamic mean calculation module 22 may include: The first calculation unit is used to calculate and determine the average pixel error between the current frame and the previous frame using the pyramid LK optical flow method, which is used as the inter-frame average pixel error of the current frame. The second calculation unit is used to determine the global average pixel error between frames of the current frame based on the average pixel error between frames of the current frame. The third calculation unit is used to determine the inter-frame cumulative pixel error rate of the current frame by using the inter-frame global average pixel error of the current frame as the error threshold. The fourth calculation unit is used to determine the dynamic mean of the inter-frame cumulative pixel error rate of the current frame based on the inter-frame cumulative pixel error rate and the inter-frame average pixel error of the current frame.
[0077] In one possible implementation, the first computing unit may specifically be used for: 1. The pyramid LK optical flow method is used, combined with the first formula, to determine the average pixel error between frames in the current frame; The first formula includes:
[0078] in, This represents the average pixel error between frames in the current frame. For the current frame, The previous frame of the current frame. and Each pixel is in and Speed of motion on the axis This represents the total number of pixels in the current frame.
[0079] In one possible implementation, the second computing unit may specifically be used for: 1. Determine the global average pixel error between frames of the current frame based on the average pixel error between frames of the current frame and the second formula. The second formula may include:
[0080] in, This represents the inter-frame global average pixel error for the current frame. This represents the total number of frames including the current frame. For the first Average pixel error between frames.
[0081] In one possible implementation, the third computing unit can be specifically used for: 1. Use the global average pixel error between frames of the current frame as the error threshold, and combine it with the third formula to determine the cumulative pixel error rate between frames of the current frame; The third formula may include:
[0082] in, The cumulative pixel error rate between frames for the current frame. This is an indicator function that takes the value 1 when the condition is true and 0 otherwise.
[0083] In one possible implementation, the fourth computing unit can be specifically used for: 1. Based on the cumulative inter-frame pixel error rate and the average inter-frame pixel error of the current frame, and combined with the fourth formula, determine the dynamic mean of the cumulative inter-frame pixel error rate of the current frame. The fourth formula may include:
[0084] in, This is the dynamic mean of the inter-frame cumulative pixel error rate for the current frame. This is the dynamic average of the cumulative pixel error rate between the current frame and the previous frame.
[0085] In one possible implementation, the second dynamic mean calculation module 23 may include: The fifth calculation unit is used to determine the average pixel error between the current frame and the reference keyframe of the current frame, which is used as the average pixel error between the reference keyframes of the current frame. The sixth calculation unit is used to determine the global average pixel error of the reference keyframe of the current frame based on the average pixel error between the reference keyframes of the current frame. The seventh calculation unit is used to determine the cumulative pixel error rate between reference keyframes of the current frame by using the global average pixel error of the reference keyframe of the current frame as the error threshold. The eighth calculation unit is used to determine the dynamic average value of the cumulative pixel error rate between reference keyframes of the current frame based on the cumulative pixel error rate between reference keyframes of the current frame and the average pixel error between reference keyframes of the current frame.
[0086] In one possible implementation, the keyframe determination module 24 may include: The candidate frame determination unit is used to determine whether the current frame is a candidate frame based on the dynamic average of the inter-frame cumulative pixel error rate of the current frame and the dynamic average of the inter-frame cumulative pixel error rate of the reference key frame of the current frame, combined with the fifth formula. The keyframe determination unit is used to determine whether the current frame is a keyframe if the frame is a keyframe by using the Z-score dynamic threshold displacement constraint method in combination with the sixth formula. The fifth formula may include:
[0087] in, This is the dynamic mean of the inter-frame cumulative pixel error rate for the current frame. This represents the cumulative pixel error rate between frames for the current frame. This is the dynamic mean of the cumulative pixel error rate between reference keyframes for the current frame. The cumulative pixel error rate between reference keyframes for the current frame; This indicates that the current frame is a candidate frame. This indicates that the current frame is not a candidate frame; The sixth formula may include:
[0088]
[0089] in, This represents the Lie algebraic distance between the current frame and its reference keyframe. This represents the number of all keyframes preceding the current frame. The standard deviation of the historical frame shift. This is the sensitivity coefficient. It is the minimum value; This indicates that the current frame is a keyframe. This indicates that the current frame is not a keyframe.
[0090] Figure 18 This is a schematic diagram of the electronic device 3 provided in an embodiment of the present invention. Figure 18 As shown, the electronic device 3 of this embodiment includes a processor 30 and a memory 31. The memory 31 stores a computer program 32. When the processor 30 executes the computer program 32, it implements the steps in the various method embodiments described above. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each module / unit in the various device embodiments described above.
[0091] For example, computer program 32 may be divided into one or more modules / units, which are stored in memory 31 and executed by processor 30 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 32 in electronic device 3.
[0092] Electronic device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 18 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 3 may also include input / output devices, network access devices, buses, etc.
[0093] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.
[0094] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.
[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A SLAM-based positioning method, characterized by, The method comprises the following steps: obtaining a reference key frame of a current frame and a previous frame of the current frame; determining an inter-frame accumulated pixel error rate dynamic mean value of the current frame according to the current frame and the previous frame of the current frame; determining a reference inter-key frame accumulated pixel error rate dynamic mean value of the current frame according to the current frame and the reference key frame of the current frame; determining whether the current frame is a key frame according to the inter-frame accumulated pixel error rate dynamic mean value of the current frame and the reference inter-key frame accumulated pixel error rate dynamic mean value of the current frame and a Z-score dynamic threshold displacement constraint method; if yes, adding the current frame to a key frame row and performing local mapping and loop detection according to the key frame row.
2. The SLAM-based positioning method of claim 1, wherein, The method for determining the inter-frame accumulated pixel error rate dynamic mean value of the current frame according to the current frame and the previous frame of the current frame comprises the following steps: determining an average pixel error between the current frame and the previous frame of the current frame as an inter-frame average pixel error of the current frame by using a pyramid LK optical flow method; determining an inter-frame global average pixel error of the current frame according to the inter-frame average pixel error of the current frame; determining an inter-frame accumulated pixel error rate of the current frame by taking the inter-frame global average pixel error of the current frame as an error threshold value; determining the inter-frame accumulated pixel error rate dynamic mean value of the current frame according to the inter-frame accumulated pixel error rate of the current frame and the inter-frame average pixel error of the current frame.
3. The SLAM-based positioning method of claim 2, wherein, The method for determining the inter-frame average pixel error of the current frame by using the pyramid LK optical flow method comprises the following steps: determining the inter-frame average pixel error of the current frame by using the pyramid LK optical flow method and a first formula. The first formula comprises: wherein, is the inter-frame average pixel error for the current frame, is the current frame, is the previous frame of the current frame, and are the motion velocities of the pixel in the x and y axes, respectively, and are the motion velocities of the pixel in the x and y axes, respectively, is the total number of pixels of the current frame.
4. The SLAM-based positioning method of claim 3, wherein, The method for determining the inter-frame global average pixel error of the current frame according to the inter-frame average pixel error of the current frame comprises the following steps: determining the inter-frame global average pixel error of the current frame according to the inter-frame average pixel error of the current frame and a second formula. The second formula comprises: wherein, is the inter-frame global average pixel error for the current frame, is the total number of frames including the current frame before the current frame, is the inter-frame average pixel error for the first frame.
5. The SLAM-based positioning method of claim 4, wherein, The method for determining the inter-frame accumulated pixel error rate of the current frame by taking the inter-frame global average pixel error of the current frame as the error threshold value comprises the following steps: determining the inter-frame accumulated pixel error rate of the current frame by taking the inter-frame global average pixel error of the current frame as the error threshold value and a third formula. The third formula comprises: wherein is the inter-frame accumulated pixel error rate for the current frame, is an indicator function that takes the value 1 when the condition is true and 0 otherwise.
6. The SLAM-based positioning method of claim 5, wherein, The method for determining the inter-frame accumulated pixel error rate dynamic mean value of the current frame according to the inter-frame accumulated pixel error rate of the current frame and the inter-frame average pixel error of the current frame comprises the following steps: determining the inter-frame accumulated pixel error rate dynamic mean value of the current frame according to the inter-frame accumulated pixel error rate of the current frame and the inter-frame average pixel error of the current frame and a fourth formula. The fourth formula comprises: wherein, is the inter-frame accumulated pixel error rate dynamic mean of the current frame, is the inter-frame accumulated pixel error rate dynamic mean of the previous frame of the current frame.
7. The SLAM-based positioning method according to any one of claims 1 to 6, characterized in that, The method for determining the reference inter-key frame accumulated pixel error rate dynamic mean value of the current frame according to the current frame and the reference key frame of the current frame comprises the following steps: determining an average pixel error between the current frame and the reference key frame of the current frame as a reference inter-key frame average pixel error of the current frame; determining a reference key frame global average pixel error of the current frame according to the reference inter-key frame average pixel error of the current frame. The global average pixel error of the reference key frame of the current frame is taken as an error threshold to determine the inter-frame accumulated pixel error rate of the reference key frame of the current frame; The inter-frame accumulated pixel error rate of the reference key frame of the current frame is determined according to the inter-frame accumulated pixel error rate of the reference key frame of the current frame and the average pixel error of the reference key frame of the current frame.
8. The SLAM-based positioning method according to any one of claims 1 to 6, characterized in that, The Z-score dynamic threshold displacement constraint method is combined to determine whether the current frame is a key frame according to the inter-frame accumulated pixel error rate dynamic mean of the current frame and the inter-frame accumulated pixel error rate dynamic mean of the reference key frame of the current frame, and the determination comprises: The fifth formula is combined to determine whether the current frame is a candidate frame according to the inter-frame accumulated pixel error rate dynamic mean of the current frame and the inter-frame accumulated pixel error rate dynamic mean of the reference key frame of the current frame; If yes, the Z-score dynamic threshold displacement constraint method is combined to determine whether the current frame is a key frame according to the sixth formula; The fifth formula comprises: wherein, is a dynamic mean of inter-frame accumulated pixel error rate for the current frame, is an inter-frame accumulated pixel error rate for the current frame; is a dynamic mean of reference key inter-frame accumulated pixel error rate for the current frame, is a reference key inter-frame accumulated pixel error rate for the current frame; , indicates that the current frame is a candidate frame, indicates that the current frame is not a candidate frame; The sixth formula comprises: wherein, is the Lie algebra distance between the current frame and the reference key frame of the current frame, is the number of all key frames before the current frame; is the standard deviation of the historical frame displacement, is the sensitivity coefficient, is the minimum value; indicates that the current frame is a key frame, indicates that the current frame is not a key frame.
9. A SLAM-based positioning apparatus, characterized by The fifth formula comprises: The parameter acquisition module is configured to acquire the reference key frame of the current frame and the previous frame of the current frame; The first dynamic mean calculation module is configured to determine the inter-frame accumulated pixel error rate dynamic mean of the current frame according to the current frame and the previous frame of the current frame; The second dynamic mean calculation module is configured to determine the inter-frame accumulated pixel error rate dynamic mean of the reference key frame of the current frame according to the current frame and the reference key frame of the current frame; The key frame determination module is configured to determine whether the current frame is a key frame according to the inter-frame accumulated pixel error rate dynamic mean of the current frame and the inter-frame accumulated pixel error rate dynamic mean of the reference key frame of the current frame by combining the Z-score dynamic threshold displacement constraint method. The key frame application module is configured to add the current frame to the key frame row if yes, and perform local mapping and loop detection according to the key frame row.
10. An electronic device, comprising: The memory stores a computer program, and the processor implements the SLAM-based positioning method according to any one of claims 1 to 8 when executing the computer program.