SLAM position identification method and system with low memory occupation

Abstract

The invention discloses an SLAM (Simultaneous Localization and Mapping) position identification method with low memory occupancy. The method comprises the following steps: S1, a robot uses a monocular camera and an IMU (Inertial Measurement Unit) to carry out first mapping on an environmental space by adopting a VINS-MONO algorithm; during mapping, only key frames are stored, and loopback detection is not carried out. S2, after the robot completes first mapping and key frame storage, the key frame is not stored any more, the key frame stored before is loaded, the robot only carries out loopback detection when moving to the corner, and the pose of the current frame is obtained after loopback detection. S3, if the pose of the current frame calculated by the robot during loopback detection is smaller than or equal to a preset threshold value, it is regarded that position recognition succeeds, and the robot closes loopback detection and restores motion; otherwise, position identification fails, and the robot closes loopback detection and recovers motion; and when the robot continues to advance to the next corner, the step S2 is repeated. The method and the device are suitable for being used by an embedded platform with small memory resources and low cost.

Description

Technical Field

[0001] The present application relates to a SLAM (simultaneous localization and mapping) technology, and in particular to a VI-SLAM (visual-inertial SLAM) technology. Background Art

[0002] SLAM is a technology that achieves simultaneous positioning and map construction in unknown environments. It is widely used in fields such as autonomous driving and autonomous mobile robots. According to the type of sensor, SLAM technology can be roughly divided into laser SLAM (Lidar SLAM), visual SLAM (VSLAM) using cameras as sensors, inertial navigation SLAM, etc.

[0003] VI-SLAM uses a method that combines cameras and IMUs (inertial measurement units). VINS-MONO is a representative algorithm in VI-SLAM. This algorithm occupies too much memory in embedded systems and cannot meet the deployment conditions for embedded systems with limited memory resources. Summary of the invention

[0004] The technical problem to be solved by this application is: how to improve the VINS-MONO position recognition method and reduce its memory usage.

[0005] In order to solve the above technical problems, the present application proposes a SLAM position recognition method with low memory usage, including the following steps. Step S1: The robot uses a monocular camera and IMU to adopt the VINS-MONO algorithm to map the environment space for the first time; only key frames are saved during mapping, and loop detection is not performed. Step S2: After the robot completes the first mapping and saving of key frames, it no longer saves key frames and loads the previously saved key frames. The robot only performs loop detection when moving to a corner, and obtains the position and posture of the current frame after loop detection. Step S3: If the position and posture of the current frame calculated by the robot during loop detection is less than or equal to the preset threshold, the position recognition is deemed to be successful, and the robot turns off the loop detection and resumes movement; otherwise, the position recognition is deemed to have failed, and the robot turns off the loop detection and resumes movement; when the robot continues to move to the next corner, step S2 is repeated.

[0006] Furthermore, in step S1, when building the map, the current frame is saved as a key frame only when the robot drives to a corner.

[0007] Furthermore, in the step S1, when a key frame is saved, only the image, posture and feature points of the frame are saved, and the .dat description subfile of the frame is not saved.

[0008] Furthermore, in step S2, the robot remains stationary when performing loop detection.

[0009] Furthermore, in step S2, when the robot performs loop detection, it uses the ORB algorithm and the RANSAC algorithm to search for loop frames.

[0010] Furthermore, in step S2, when searching for the loop frame, the robot first uses the ORB algorithm to detect and match feature points of the current frame and the key frame - ORB feature points of the current frame are matched with the key frame sequence one by one, and then the RANSAC algorithm is used to delete the mismatched points, and a similarity query is performed between the current frame and the key frame to find the loop frame.

[0011] Furthermore, in step S2, when the robot performs loop detection, the brief descriptor is used to calculate the pose of the current frame.

[0012] Furthermore, in step S2, when calculating the pose of the current frame, the robot first uses the VINS-MONO algorithm to extract optical flow feature points between the current frame and the loop frame, and then calculates the brief descriptor of each feature point, and matches the brief descriptor of the feature point of the current frame with the brief descriptor of the feature point of the best key frame using BFMatcher, and then uses the RANSAC matching algorithm to delete the mismatched points to obtain 2D feature points and corresponding 3D spatial feature points; the obtained 2D feature points and 3D spatial feature points are used to perform PnP calculation according to the VINS-MONO algorithm to obtain the pose of the current frame.

[0013] The present application also proposes a SLAM position recognition system with low memory usage, including a mapping unit, a position recognition unit, and a judgment unit. The mapping unit is used to use a monocular camera and an IMU to adopt the VINS-MONO algorithm to map the environment space for the first time; only key frames are saved during mapping, and loop detection is not performed. The position recognition unit is used to no longer save key frames after completing the first mapping and saving of key frames, load the key frames saved previously, and perform loop detection only when moving to a corner, and obtain the posture of the current frame after loop detection. The judgment unit is used to determine that the position recognition is successful when the posture of the current frame calculated during loop detection is less than or equal to a preset threshold, turn off loop detection and resume movement; when the posture of the current frame calculated during loop detection is greater than the preset threshold, it is determined that the position recognition fails, turn off loop detection and resume movement.

[0014] The technical effect achieved by the present application is as follows: by utilizing a low-cost monocular camera and an IMU, autonomous mapping can be successfully completed in an indoor environment without using a GPU (graphics processing unit), and the robustness is high for areas with less texture (such as white walls, etc.); at the same time, the robot can perform position recognition according to the previous map when it is turned on, and find its current position, thereby meeting the functional requirements of actual production applications; and the memory occupied is less than 200MB, which is suitable for use in embedded platforms with small memory resources and low costs, and is of great significance to actual production and life. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of the low memory usage SLAM position recognition method proposed in this application.

[0016] Figure 2 It is a structural diagram of the low memory usage SLAM position recognition system proposed in this application.

[0017] Description of the reference numerals in the figure: mapping unit 1, location identification unit 2, judgment unit 3. DETAILED DESCRIPTION

[0018] See also Figure 1 ,The SLAM position recognition method with low memory usage proposed in this application includes the following steps.

[0019] Step S1: The embedded platform robot (hereinafter referred to as the robot) uses a monocular camera and IMU to adopt the VINS-MONO algorithm to map the environment space for the first time. When building the map, only key frames are saved, and loop detection is not performed. After the map is built, the map is saved. When building the map, the existing VINS-MONO algorithm saves key frames and performs loop detection at the same time. This application cancels the loop detection, which effectively avoids the loop detection process from occupying memory.

[0020] In addition, when building a map, the current frame is saved as a key frame only when the robot drives to a corner. The existing VINS-MONO algorithm calculates the position distance to determine whether the current frame is a key frame. This application improves the key frame selection logic to reduce the number of key frames that need to be saved during operation, effectively reducing memory usage.

[0021] In addition, when saving a key frame, only the image, posture and feature points of the frame are saved, and the .dat description subfile of the frame is not saved. The existing VINS-MONO algorithm saves the .dat description subfile of the frame and the image, posture and feature points of the frame at the same time when saving a key frame. This application discards the .dat file that occupies a large amount of memory, effectively reducing memory usage.

[0022] Step S2: After the robot completes the first mapping and keyframe saving, it no longer saves the keyframes and loads the previously saved keyframes. The robot performs loop detection only when it moves to a corner. After loop detection, the position and posture of the current frame are obtained, and the robot remains stationary during loop detection. The existing SLAM system turns on the loop detection function throughout the positioning process. This application changes it to only perform loop detection under specified conditions to save memory.

[0023] In addition, when the robot performs loop detection, it uses the ORB (Oriented FAST and Rotated BRIEF) algorithm and the RANSAC (random sample consensus) algorithm to search for loop frames. Specifically, this application uses the ORB algorithm to detect and match feature points of the current frame and key frames - ORB feature points are matched one by one between the current frame and the key frame sequence, and the RANSAC algorithm is used to delete mismatched points, and similarity queries are performed between the current frame and the key frame to find loop frames.

[0024] The existing VINS-MONO algorithm uses the Bag-of-words model to compare the similarity between the current frame and the key frame during loop detection, which has good real-time performance, but the Bag-of-words model occupies a large amount of memory; and as time goes by, the required memory will gradually increase. After abandoning the Bag-of-words model that occupies a large amount of memory, this application integrates the ORB algorithm and the RANSAC algorithm to replace the Bag-of-words model to complete loop detection, and the matching time required for a single loop detection increases; but this application limits loop detection to only when the robot moves to a corner; after the two factors are superimposed, it does not affect the real-time performance of the entire SLAM system. Experiments show that this application greatly reduces the memory usage without a significant increase in the overall matching time, and the matching effect is relatively accurate, and it has strong robustness to changes in illumination, angle, etc. in indoor conditions.

[0025] In addition, when the robot performs loop detection, it uses the brief descriptor to calculate the pose of the current frame. Specifically, the application first uses the VINS-MONO algorithm to extract the optical flow feature points between the current frame and the best key frame (i.e., the loop frame), and then calculates the brief descriptor of each feature point. The brief descriptor of the feature point of the current frame and the brief descriptor of the feature point of the best key frame are matched using BFMatcher (Brute-Force Matcher), and then the RANSAC matching algorithm is used to delete the mismatched points to obtain more accurate 2D feature points and corresponding 3D space feature points; finally, the obtained 2D feature points and 3D space feature points are used to perform PnP (Perspective-n-Point) calculation according to the VINS-MONO algorithm to obtain the current frame pose.

[0026] The existing VINS-MONO algorithm uses the bag-of-words model to calculate the descriptor between the current frame and the best keyframe during loop detection, and then performs subsequent matching calculations. The present application uses the brief descriptor to replace the bag-of-words model that occupies a large amount of memory to perform feature matching between the feature points of the current frame and the loop frame, thereby reducing memory usage. Experiments show that the matching and pose calculation results of the present application are relatively accurate. For mapping and position recognition of a room of about 100 square meters, the pose_graph process of the existing VINS-MONO algorithm occupies about 2GB to 4GB of memory, and the improved method of the present application reduces the memory usage to less than 200MB. This makes it possible to deploy on low-memory embedded platforms with less computing resources, and to implement map reuse position recognition functions and subsequent navigation functions.

[0027] Step S3: If the current frame pose calculated by the robot during loop detection is less than or equal to the preset threshold, the position recognition is deemed to be successful, and the robot turns off the loop detection and resumes movement. If the current frame pose calculated by the robot during loop detection is greater than the preset threshold, the position recognition is deemed to have failed, and the robot turns off the loop detection and resumes movement. When the robot continues to move forward to the next corner, step S2 is repeated.

[0028] See also Figure 2 The low memory occupancy SLAM position recognition system proposed in this application includes a mapping unit 1, a position recognition unit 2, and a judgment unit 3. Figure 2 The system shown corresponds to Figure 1 The method shown.

[0029] The mapping unit 1 is used to use a monocular camera and an IMU to perform the first mapping of the environment space using the VINS-MONO algorithm. When mapping, only key frames are saved, and loop detection is not performed.

[0030] The position recognition unit 2 is used to stop saving key frames after completing the first mapping and key frame saving, load the previously saved key frames, and perform loop detection only when moving to the corner, and obtain the position and posture of the current frame after loop detection.

[0031] The judgment unit 3 is used to determine that the position recognition is successful when the posture of the current frame calculated during loop detection is less than or equal to a preset threshold, turn off the loop detection and resume the movement; when the posture of the current frame calculated during loop detection is greater than the preset threshold, determine that the position recognition fails, turn off the loop detection and resume the movement.

[0032] Compared with the prior art, the present application has the following beneficial effects.

[0033] First, the existing SLAM algorithms cannot meet the three requirements of low cost, low memory usage and stability at the same time. This application improves the VINS-MONO algorithm while ensuring low cost, abandons the bag-of-words model, and integrates the ORB feature matching algorithm to greatly reduce memory and computing power usage. This application can ensure that the memory consumption is below 200MB, realize deployment on small memory embedded platforms, and realize autonomous mapping and location recognition functions at startup.

[0034] Second, in the existing SLAM algorithm, such as ORB-SLAM and its derivative algorithm also use the ORB algorithm to extract feature points to complete the positioning and mapping function. However, this type of method also needs to use a bag-of-words model with large memory usage to save computing time. At the same time, this type of method is heavily dependent on the texture complexity of the picture in the field of view, and is prone to positioning loss in areas with fewer texture features (such as white walls). Therefore, the stability of this type of method is lacking and cannot be put into practical use. The present application uses a monocular camera and IMU as sensors at the same time when building a map for the first time, wherein IMU has the effect of auxiliary positioning, and although there is less visual information (fewer feature points) in weak texture areas, it can also be located with the help of IMU, so it avoids the strong dependence of pure visual SLAM such as ORB-SLAM on texture information, that is, the present application improves the robustness of weak texture areas.

[0035] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should all be included in the protection scope of the present application.

Claims

1. A low memory usage SLAM position recognition method, characterized in that: The method comprises the following steps: Step S1: The robot uses a monocular camera and IMU to build a first map of the environment space using the VINS-MONO algorithm. When building the map, only key frames are saved, and loop detection is not performed. Step S2: After the robot completes the first mapping and keyframe saving, it no longer saves the keyframes and loads the previously saved keyframes. The robot performs loop detection only when it moves to a corner, and obtains the pose of the current frame after loop detection. Step S3: If the position of the current frame calculated by the robot during loop detection is less than or equal to the preset threshold, the position recognition is deemed to be successful, and the robot turns off the loop detection and resumes movement; otherwise, the position recognition is deemed to have failed, and the robot turns off the loop detection and resumes movement; when the robot continues to move forward to the next corner, step S2 is repeated.

2. The SLAM position recognition method with low memory usage according to claim 1, characterized in that: In step S1, when building a map, the current frame is saved as a key frame only when the robot drives to a corner.

3. The SLAM position recognition method with low memory usage according to claim 1, characterized in that: In the step S1, when a key frame is saved, only the image, posture and feature points of the frame are saved, and the .dat description subfile of the frame is not saved.

4. The SLAM position recognition method with low memory usage according to claim 1, characterized in that: In step S2, the robot remains stationary while performing loop detection.

5. The SLAM position recognition method with low memory usage according to claim 1, characterized in that: In step S2, when performing loop detection, the robot uses the ORB algorithm and the RANSAC algorithm to search for loop frames.

6. The low memory usage SLAM position recognition method according to claim 5, characterized in that: In step S2, when searching for the loop frame, the robot first uses the ORB algorithm to detect and match feature points of the current frame and the key frame - ORB feature points of the current frame are matched with the key frame sequence one by one, and then the RANSAC algorithm is used to delete the mismatched points, and a similarity query is performed between the current frame and the key frame to find the loop frame.

7. The SLAM position recognition method with low memory usage according to claim 1, characterized in that: In step S2, when the robot performs loop detection, the brief descriptor is used to calculate the pose of the current frame.

8. The low memory usage SLAM position recognition method according to claim 7, characterized in that: In the step S2, when calculating the pose of the current frame, the robot first uses the VINS-MONO algorithm to extract the optical flow feature points between the current frame and the loop frame, then calculates the brief descriptor of each feature point, matches the brief descriptor of the feature point of the current frame with the brief descriptor of the feature point of the best key frame using BFMatcher, and then uses the RANSAC matching algorithm to delete the mismatched points to obtain 2D feature points and corresponding 3D spatial feature points; and uses the obtained 2D feature points and 3D spatial feature points to perform PnP calculation according to the VINS-MONO algorithm to obtain the pose of the current frame.

9. A low memory footprint SLAM position recognition system, characterized in that: It includes a mapping unit, a location recognition unit, and a judgment unit; The mapping unit is used to use a monocular camera and an IMU to perform a first mapping of the environment space using a VINS-MONO algorithm; When building a map, only key frames are saved and loop detection is not performed; The position recognition unit is used to stop saving key frames after completing the first mapping and key frame saving, load the previously saved key frames, perform loop detection only when moving to the corner, and obtain the position and posture of the current frame after loop detection; The judgment unit is used to determine that the position recognition is successful when the posture of the current frame calculated during loop detection is less than or equal to a preset threshold, turn off the loop detection and resume movement; when the posture of the current frame calculated during loop detection is greater than the preset threshold, determine that the position recognition fails, turn off the loop detection and resume movement.

Images