Indoor and outdoor dynamic target intelligent positioning system and method based on SLAM

Abstract

The invention discloses an indoor and outdoor dynamic target intelligent positioning system and method based on SLAM. The positioning system comprises an image acquisition module, a self-adaptive target detection module, a key frame verification module, a pose optimization and closed-loop detection module and a three-dimensional model reconstruction module. The image acquisition module is used for acquiring an original image frame; the adaptive target detection module is used for acquiring static feature points according to the dynamic feature points of the original image frame; the key frame verification module is used for verifying feature points of key frames according to the static feature points; the optimization and detection module is used for optimizing and detecting the key frame according to the verification result; and the three-dimensional model reconstruction module is used for obtaining a point cloud map according to the verification result in combination with the detection result. The system not only can meet accurate positioning in a dynamic environment, but also can ensure point cloud mapping of the environment.

Description

Technical Field

[0001] The invention belongs to the field of mobile robot positioning and unmanned driving, and in particular relates to a SLAM-based indoor and outdoor dynamic target intelligent positioning system and method. Background Art

[0002] Traditional visual SLAM systems mostly operate under the assumption of static scenes, but in most real-world indoor and outdoor scenes, there are inevitably dynamic objects, such as cars and people. These unstable moving objects will easily cause SLAM to produce incorrect data associations when matching feature points between frames, which in turn leads to incorrect pose estimation and map construction. Therefore, SLAM systems based on static scene assumptions have great limitations in practical applications. At present, mainstream technologies mainly include deep learning-based and geometry-based methods. Although they effectively filter out dynamic feature points and enhance the positioning accuracy of the system to a certain extent, there are still many shortcomings. For example, the network models of some semantic segmentation-based methods are large in size, and the system needs to rely on high-performance GPUs and lacks the ability to run in real time; other geometric methods based on depth information are limited by depth images and cannot run in outdoor dynamic environments; and some distance threshold-based methods only filter dynamic and static points through predefined thresholds, which are prone to over-extraction or under-extraction problems. At the same time, for non-rigid, unstable dynamic objects, such as the human body, how to effectively eliminate the problem of continuous frame occlusion and maintain the long-term association of multi-frame data is also a major challenge facing current dynamic SLAM systems. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a SLAM-based indoor and outdoor dynamic target intelligent positioning system and method, which can solve the problem of real-time and accurate positioning and mapping in dynamic scenes.

[0004] To achieve the above-mentioned object, the present invention provides an indoor and outdoor dynamic target intelligent positioning system based on SLAM, comprising: an image acquisition module, an adaptive target detection module, a key frame verification module, a posture optimization and closed-loop detection module and a three-dimensional model reconstruction module;

[0005] The image acquisition module is used to acquire original image frames;

[0006] The adaptive target detection module is used to obtain static feature points based on the dynamic feature points of the original image frame;

[0007] The key frame verification module is used to verify the feature points of the key frame according to the static feature points;

[0008] The optimization and detection module is used to optimize and detect the key frame according to the verification result;

[0009] The three-dimensional model reconstruction module is used to obtain a point cloud map based on the verification result and the detection result.

[0010] Optionally, the adaptive target detection module includes: a feature point extraction unit and a feature point detection frame unit;

[0011] The feature point extraction unit is used to extract feature points from the original image frame to obtain feature points;

[0012] The feature point detection frame unit is used to detect the feature points and obtain static feature points.

[0013] Optionally, the feature points include: static feature points and dynamic feature points.

[0014] Optionally, the key frame verification module includes: a bidirectional epipolar constraint distance calculation unit and a Gaussian distribution probability conversion unit;

[0015] The bidirectional epipolar constraint distance calculation unit is used to obtain the distance of the dynamic feature point relative to the epipolar line according to the static feature point and the key frame;

[0016] The Gaussian distribution probability conversion unit is used to obtain the movement probability of the key point according to the distance, and verify the key frame according to the movement probability.

[0017] Optionally, the optimization and detection module includes: a posture optimization unit and a closed-loop detection unit;

[0018] The posture optimization unit is used to optimize the key frame according to the verification result to obtain the next key frame;

[0019] The closed-loop detection unit is used to detect the similarity between the key frame and the next key frame and obtain a detection result.

[0020] The present invention also provides a SLAM-based indoor and outdoor dynamic target intelligent positioning method, comprising:

[0021] Collect original images;

[0022] Performing feature point extraction and target detection on the original image to obtain static feature points;

[0023] Based on the static feature points, combined with the key frames, checking the motion state of the feature points;

[0024] Based on the motion state, optimizing the key frame to obtain a next key frame;

[0025] Detecting the key frame and the next key frame to obtain a detection result;

[0026] Based on the detection results and the motion state of the feature points, three-dimensional reconstruction is performed to obtain a point cloud image.

[0027] Optionally, based on the static feature points and in combination with key frames, a method for checking the motion state of the feature points is:

[0028]

[0029] Among them, P is the movement probability of the key point, δ is the standard deviation of the distribution, and d is the distance of the dynamic feature point relative to the pole line.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] The present invention not only overcomes the limitation of the existing technology that the neural network model is too large and heavily dependent on the GPU, but also significantly improves the accuracy and quality of positioning and mapping without relying on depth information, so that the device can perform well in a variety of dynamic scenes indoors and outdoors, such as environmental mapping, robot positioning and navigation, unmanned driving research, etc. In addition, by introducing a coarse-to-fine dynamic feature point selection strategy, the system has higher flexibility and applicability, which has important practical significance and potential commercial value for promoting the progress of the field of robot synchronous positioning and mapping technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0033] Figure 1 The present invention is a flowchart of a method for intelligent positioning of indoor and outdoor dynamic targets based on SLAM according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] The present invention proposes an indoor and outdoor dynamic target intelligent positioning system based on SLAM, which specifically includes the following steps: including: an image acquisition module, an adaptive target detection module, a key frame verification module, a posture optimization and closed-loop detection module and a three-dimensional model reconstruction module;

[0037] An image acquisition module, used for acquiring original image frames;

[0038] An adaptive target detection module is used to obtain static feature points based on dynamic feature points of the original image frame;

[0039] A key frame verification module is used to verify the feature points of the key frame based on the static feature points;

[0040] An optimization and detection module, used to optimize and detect key frames according to the verification results;

[0041] The 3D model reconstruction module is used to obtain a point cloud map based on the verification results and the detection results.

[0042] Further, the adaptive target detection module includes: a feature point extraction unit, a feature point detection frame unit;

[0043] A feature point extraction unit is used to extract feature points from the original image frame to obtain feature points;

[0044] The feature point detection frame unit is used to detect feature points and obtain static feature points.

[0045] Specifically, the system includes an adaptive target detection module: This module includes a lightweight dynamic target detection algorithm and an adaptive dynamic feature point filtering algorithm based on distance probability. Using a multi-threaded programming architecture, the YOLOv5 lightweight target detection network is integrated into the SLAM system through LibTorch to perform target detection on the input image and obtain the prior frame information of the dynamic object. The feature points that are located in both the static and dynamic detection frames ensure that while filtering out unstable dynamic feature points, high-quality, stable static feature points are retained.

[0046] Furthermore, the feature points include: static feature points and dynamic feature points.

[0047] Further, the keyframe verification module includes: a bidirectional epipolar constraint distance calculation unit and a Gaussian distribution probability conversion unit;

[0048] A bidirectional epipolar constraint distance calculation unit, used for obtaining the distance of the dynamic feature point relative to the epipolar line based on the static feature point and the key frame;

[0049] The Gaussian distribution probability conversion unit is used to obtain the movement probability of the key point according to the distance, and verify the key frame according to the movement probability.

[0050] Specifically, the bidirectional epipolar constraint distance calculation unit: First, the key point matching pairs from the previous frame to the current frame are obtained by calculating the optical flow in the current frame, and a reverse optical flow tracking is performed. If the pixel-by-pixel coordinate difference between the forward and backward tracked feature points is less than 0.5 pixels, a successful match is determined. After that, we use RANSAC to remove outliers, and obtain the basic matrix based on the matching points, and use polarity constraints to measure the distance of the dynamic feature points relative to the epipolar lines.

[0051] Gaussian distribution probability conversion unit: The longer the distance obtained in the bipolar line constraint distance calculation unit, the greater the probability of movement. Assuming that the distance from the matching key point to its corresponding polar line satisfies the Gaussian distribution, that is, the normal probability density function is used to calculate the movement probability P of the key point, when P>0.5, it is considered to be a dynamic feature point and will not participate in the subsequent pose estimation. The combination of the bipolar line constraint distance calculation unit and the Gaussian distribution probability conversion unit effectively verifies the actual motion state of the potential dynamic feature points.

[0052] Further, the optimization and detection module includes: a posture optimization unit and a closed-loop detection unit;

[0053] A posture optimization unit, used to optimize the key frame according to the verification result and obtain the next key frame;

[0054] The closed loop detection unit is used to detect the similarity between the key frame and the next key frame and obtain the detection result.

[0055] Specifically, the pose optimization module uses the idea of ​​nonlinear optimization to construct a larger-scale graph optimization problem to consider the optimal trajectory and map over a long period of time, and performs least squares optimization and update on the pose and map points in key frames to remove the inevitable error accumulation caused by the long-term positioning of the system.

[0056] Closed-loop detection module: The received key frames and historical key frames are subjected to norm calculation to determine the similarity between the two frames, so as to obtain the degree of association between all historical frame data and the current frame data, and obtain a globally consistent pose estimation and map, which greatly improves the accuracy and robustness of the system.

[0057] 3D model reconstruction module: It applies efficient 3D point cloud filtering, registration and modeling algorithms to create point cloud data and filter optimization processing for the received key frames, and finally constructs a high-quality static 3D model that can reflect the global morphological features and remove dynamic objects.

[0058] In actual operation, the system provided by the present invention first obtains the original image frame according to the sensor, then obtains the detection frame prior information of the detection object in real time through the target detection algorithm, synchronously extracts features from the image frame, then detects and filters the feature points in the detection frame accordingly, and uses stable static feature points to perform pose estimation. After inserting the key frame, motion consistency detection is performed, and the pose and map points are optimized and updated at the back end, and the results are fed back to the next frame. At the same time, the closed-loop detection module performs a similarity check on the current key frame and the historical key frame to ensure the global consistency of the pose estimation and the map. Finally, the three-dimensional model reconstruction module creates and optimizes the point cloud data to construct a high-quality dense point cloud map. During the entire process, the system runs in real time using only the CPU.

[0059] The present invention also provides a SLAM-based indoor and outdoor dynamic target intelligent positioning method, comprising:

[0060] Collect original images;

[0061] Perform feature point extraction and target detection on the original image to obtain static feature points;

[0062] Based on static feature points and key frames, the motion state of feature points is checked;

[0063] Based on the motion state, the key frame is optimized to obtain the next key frame;

[0064] Detect the key frame and the next key frame to obtain the detection result;

[0065] Based on the detection results and the motion status of the feature points, three-dimensional reconstruction is performed to obtain a point cloud image.

[0066] Specifically, in the image acquisition input stage: First, the operator defines the resolution and acquisition frequency of the acquired image (for example, this image requires a resolution of 1241×376 and an acquisition frequency of 10FPS). Then, the camera with calibrated parameters is started, and each frame of the image will be sent to the adaptive target detection module.

[0067] Adaptive dynamic feature point screening process: The adaptive target detection module performs target detection based on the input image and outputs the detection frame information and status of the detected object. At the same time, feature points are extracted from the input image. When the feature point is in the static detection frame, it is retained; when the feature point is in the dynamic detection frame, it is marked as a potential dynamic feature point. When the feature point is in both the dynamic and static detection frames, the high-quality stable static feature point is retained.

[0068] Pose optimization and closed-loop detection process: After screening the feature points, use stable static feature points to match the two frames, and verify the feature points of the detected object after inserting the keyframe. After that, the pose and map points are optimized and updated in the pose optimization module, and the unstable dynamic feature points are removed and the results are fed back to the next frame. At the same time, the closed-loop detection module performs norm calculation on the current keyframe and the historical keyframe to determine the similarity between the two frames, ensuring the global consistency of the pose estimation and the map.

[0069] 3D model reconstruction: Create point cloud data and filter optimization processing for the received key frames, including but not limited to denoising, smoothing, surface fitting and other steps, and finally reconstruct a high-precision static 3D model without dynamic objects. The model not only shows the overall outline of the environment, but also removes the dynamic objects contained in it.

[0070] Furthermore, based on the static feature points and in combination with the key frames, the method for checking the motion state of the feature points is as follows:

[0071]

[0072] Among them, P is the movement probability of the key point, δ is the standard deviation of the distribution, and d is the distance of the dynamic feature point relative to the pole line.

[0073] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A SLAM-based indoor and outdoor dynamic target intelligent positioning system, characterized in that: include: Image acquisition module, adaptive target detection module, key frame verification module, pose optimization and closed-loop detection module and 3D model reconstruction module; The image acquisition module is used to acquire original image frames; The adaptive target detection module is used to obtain static feature points based on the dynamic feature points of the original image frame; The key frame verification module is used to verify the feature points of the key frame according to the static feature points; The optimization and detection module is used to optimize and detect the key frame according to the verification result; The three-dimensional model reconstruction module is used to obtain a point cloud map based on the verification result and the detection result.

2. The SLAM-based indoor and outdoor dynamic target intelligent positioning system according to claim 1, characterized in that: The adaptive target detection module includes: a feature point extraction unit and a feature point detection frame unit; The feature point extraction unit is used to extract feature points from the original image frame to obtain feature points; The feature point detection frame unit is used to detect the feature points and obtain static feature points.

3. The SLAM-based indoor and outdoor dynamic target intelligent positioning system according to claim 2, characterized in that: The feature points include: static feature points and dynamic feature points.

4. The SLAM-based indoor and outdoor dynamic target intelligent positioning system according to claim 1, characterized in that: The key frame verification module includes: a bidirectional polar line constraint distance calculation unit and a Gaussian distribution probability conversion unit; The bidirectional epipolar constraint distance calculation unit is used to obtain the distance of the dynamic feature point relative to the epipolar line according to the static feature point and the key frame; The Gaussian distribution probability conversion unit is used to obtain the movement probability of the key point according to the distance, and verify the key frame according to the movement probability.

5. The SLAM-based indoor and outdoor dynamic target intelligent positioning system according to claim 1, characterized in that: The optimization and detection module includes: a posture optimization unit and a closed-loop detection unit; The posture optimization unit is used to optimize the key frame according to the verification result to obtain the next key frame; The closed-loop detection unit is used to detect the similarity between the key frame and the next key frame and obtain a detection result.

6. A SLAM-based intelligent positioning method for indoor and outdoor dynamic targets, characterized in that: include: Collect original images; Performing feature point extraction and target detection on the original image to obtain static feature points; Based on the static feature points, combined with the key frames, checking the motion state of the feature points; Based on the motion state, optimizing the key frame to obtain a next key frame; Detecting the key frame and the next key frame to obtain a detection result; Based on the detection results and the motion state of the feature points, three-dimensional reconstruction is performed to obtain a point cloud image.

7. The method for intelligent positioning of indoor and outdoor dynamic targets based on SLAM according to claim 6, characterized in that: Based on the static feature points and in combination with key frames, the method for checking the motion state of the feature points is: Among them, P is the movement probability of the key point, δ is the standard deviation of the distribution, and d is the distance of the dynamic feature point relative to the pole line.

Images