Closed-loop optimization detection method and device based on VSLAM system
By identifying and eliminating error closed loops in the VSLAM system and adjusting the position and attitude of keyframes, the problem of introducing error closed loops in the V-SLAM system in complex environments is solved, and the positioning accuracy and stability of the system are improved.
Patent Information
- Application Number
- CN202510050164.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-13
- Publication Date
- 2025-05-23
AI Technical Summary
Existing V-SLAM systems are prone to introduce error closed loops in complex or highly dynamic environments, resulting in map distortion and positioning errors, affecting the accuracy and stability of the system.
By obtaining the keyframes in the same map in the global map of the VSLAM system, calculating the relative displacement, identifying the wrong closed loop and its abnormal distortion ratio, eliminating the wrong closed loop, adjusting the position and attitude of the effective keyframes, and updating the global map in real time.
Effectively identify and eliminate error closed loops, improve the system's positioning accuracy, robustness and stability in complex environments and high dynamic environments, reduce dependence on training data, and is suitable for devices with resource limitations.
Smart Images
Figure CN120027826A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of spatial mapping and positioning technology, and in particular to a closed-loop optimization detection method and device based on a VSLAM system. Background Art
[0002] In the prior art, V-SLAM systems for visual simultaneous localization and map construction are widely used in robotics, autonomous driving, virtual reality (VR) and other fields. The closed-loop detection process is used to eliminate drift errors and improve positioning accuracy. However, in complex or highly dynamic environments, due to factors such as noise in the perception data and key frame matching errors, closed-loop detection may introduce erroneous closed-loops, resulting in serious map distortion or positioning errors during the optimization process, seriously affecting the accuracy and stability of the system.
[0003] Currently, loop closure matching is usually performed through loop closure detectors, and the similarity of visual features is used to determine whether a loop exists. However, such methods are easily affected by factors such as environmental changes and occlusions. Especially in high-dynamic scenes, the probability of false detection of loop closures is high, which in turn affects the performance of the entire system. Therefore, a method that can accurately detect loop closure errors is needed to improve the reliability of the system.
[0004] With the continuous development of robotics and autonomous driving technology, the real-time and robustness requirements of V-SLAM systems are becoming increasingly higher. Most of the existing closed-loop detection methods rely on the matching of visual features, the estimation of relative pose, and the auxiliary fusion of inertial measurement unit (IMU) data. However, in practical applications, the accuracy of visual feature matching is affected by many factors, such as lighting changes, scene occlusions, dynamic objects, etc., resulting in erroneous closed-loop detection and inaccurate map optimization; in highly dynamic environments, especially when the robot moves quickly, the existing closed-loop detection algorithm is difficult to effectively deal with the complex spatial relationship between key frames, and it is easy to introduce erroneous closed loops, resulting in a decrease in the positioning accuracy of the system.
[0005] In addition, in recent years, some deep learning methods have been applied to the closed-loop detection of V-SLAM systems, such as extracting global image features through convolutional neural networks (CNNs) to assist closed-loop detection; however, these methods usually require a large amount of training data, have high computational complexity, and are difficult to achieve real-time requirements on resource-constrained devices. Summary of the invention
[0006] One purpose of the present application is to provide a closed-loop optimization detection method and device based on a VSLAM system, which can effectively identify and eliminate erroneous closed loops introduced by visual feature matching errors, thereby improving the positioning accuracy and stability of the system.
[0007] According to one aspect of the present application, a closed-loop optimization detection method based on a VSLAM system is provided, wherein the method includes:
[0008] During the closed-loop detection in the global map of the VSLAM system, obtain all the key frames in the same map between the loop closure key frame and the current key frame;
[0009] Calculate the relative displacement between each pair of adjacent key frames respectively;
[0010] Based on the relative displacement between each pair of the adjacent key frames, determine at least one false closed loop formed between the loop closure key frame and the current key frame, and its corresponding abnormal distortion ratio and key frame information;
[0011] Based on the key frame information, eliminate the at least one false closed loop between the loop closure key frame and the current key frame;
[0012] Based on the abnormal distortion ratio, adjust the positions and poses of the remaining valid key frames between the loop closure key frame and the current key frame;
[0013] Update the positions and poses of the adjusted valid key frames to the global map of the VSLAM system in real time.
[0014] Further, in the above method, during the closed-loop detection in the global map of the VSLAM system, obtaining all the key frames in the same map between the loop closure key frame and the current key frame includes:
[0015] During the closed-loop detection in the global map of the VSLAM system, obtain the pose data of the current key frame and the pose data of the corresponding loop closure key frame;
[0016] Based on the map identifier carried in the pose data, determine whether the current key frame and the loop closure key frame belong to the same map;
[0017] If the map identifier carried in the pose data of the current key frame is the same as the map identifier carried in the pose data of the loop closure key frame, determine that the current key frame and the loop closure key frame belong to the same map, and obtain all the key frames between the loop closure key frame and the current key frame.
[0018] Further, in the above method, after determining whether the current key frame and the loop closure key frame belong to the same map based on the map identifier carried in the pose data, the method further includes:
[0019] If the map identifier carried by the pose data of the current key frame is different from the map identifier carried by the pose data of the loop key frame, it is determined that the current key frame and the loop key frame belong to different maps.
[0020] Furthermore, in the above method, the step of determining at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information based on the relative displacement between each pair of adjacent key frames includes:
[0021] Projecting the relative displacement between each pair of adjacent key frames onto the loop direction, and calculating the cumulative projection distance of all key frames from the loop key frame to the current key frame;
[0022] Determine whether the accumulated projection distance is greater than or equal to a preset distance threshold,
[0023] If not, then analyzing the relative displacement between each pair of adjacent key frames frame by frame, and calculating the distortion ratio of each pair of adjacent key frames;
[0024] Based on the distortion ratio of each pair of adjacent key frames, at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information are determined.
[0025] Furthermore, in the above method, the step of determining at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information based on the distortion ratio of each pair of adjacent key frames includes:
[0026] Determine whether the distortion ratio of each pair of adjacent key frames exceeds a preset burden ratio.
[0027] If so, the distortion ratio corresponding to the preset burden ratio is determined as an abnormal distortion ratio, and the closed loop formed between adjacent key frames corresponding to the abnormal distortion ratio is determined as an erroneous closed loop, and the abnormal distortion ratio and key frame information are recorded, wherein the key frame information is the information of the next key frame in the adjacent key frames corresponding to the abnormal distortion ratio;
[0028] If not, the next key frame among the adjacent key frames corresponding to the distortion ratio that does not exceed the preset burden ratio is determined as a valid key frame.
[0029] Furthermore, in the above method, after determining whether the accumulated projection distance is greater than or equal to a preset distance threshold, the method further includes:
[0030] If so, it is used to indicate that the entire closed loop formed from the loop key frame to the current key frame is invalid.
[0031] Furthermore, in the above method, adjusting the position and posture of each remaining valid key frame from the loop key frame to the current key frame based on the abnormal distortion ratio includes:
[0032] Based on the abnormal distortion ratio, recalculate the relative posture relationship between the loop key frame and the current key frame and between the remaining valid key frames;
[0033] The position and posture of each of the valid key frames in the global map are adjusted based on the relative position and posture relationship.
[0034] According to another aspect of the present application, a non-volatile storage medium is also provided, on which computer-readable instructions are stored. When the computer-readable instructions can be executed by a processor, the processor implements the closed-loop optimization detection method based on the VSLAM system as described above.
[0035] According to another aspect of the present application, a closed-loop optimization detection device based on a VSLAM system is also provided, wherein the device comprises:
[0036] one or more processors;
[0037] A computer readable medium for storing one or more computer readable instructions,
[0038] When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the closed-loop optimization detection method based on the VSLAM system as described above.
[0039] Compared with the prior art, the present application obtains all key frames from the loop key frame to the current key frame in the same map during the loop detection process in the global map of the VSLAM system; calculates the relative displacement between each pair of adjacent key frames respectively; determines at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information based on the relative displacement between each pair of adjacent key frames; based on the key frame information, eliminates the at least one erroneous closed loop from the loop key frame to the current key frame; based on the abnormal distortion ratio, adjusts the position and posture of each remaining valid key frame from the loop key frame to the current key frame; updates the adjusted position and posture of each valid key frame in real time to the global map of the VSLAM system, realizes the recognition and detection of the erroneous closed loop corresponding to the abnormal distortion ratio, and can effectively eliminate the erroneous closed loop, which can not only improve the probability of effectively finding the erroneous closed loop in complex environment and / or high dynamic environment, but also improve the positioning accuracy, robustness and stability of the VSLAM system.
[0040] In addition, unlike traditional deep learning-based methods, this application does not require a large amount of training data, reduces dependence on training data, can achieve higher real-time and reliability on resource-constrained devices, and is suitable for a variety of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0042] Figure 1 A schematic flow chart of a closed-loop optimization detection method based on a VSLAM system according to one aspect of the present application is shown.
[0043] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION
[0044] The present application is described in further detail below in conjunction with the accompanying drawings.
[0045] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (CPU), input / output interfaces, network interfaces and memories.
[0046] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0047] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.
[0048] Existing closed-loop detection methods usually rely on visual feature matching, which is easily affected by lighting changes, scene occlusions, and dynamic objects, resulting in inaccurate closed-loop detection and affecting the overall performance of the system. In order to improve the accuracy and robustness of closed-loop detection, a lightweight and efficient method is still needed that can identify and remove incorrect closed-loop connections during the closed-loop optimization process to improve the robustness and accuracy of the VSLAM system. Figure 1 As shown, Figure 1 A flow chart of a closed-loop optimization detection method based on a VSLAM system proposed for one aspect of the present application, the method mainly relates to visual synchronous positioning and mapping (VSLAM) technology, especially for closed-loop optimization problems in the fields of robots, virtual reality (VR), etc., and is intended to solve the problem of false closed loops in existing closed-loop detection technology, especially for application scenarios in complex environments and / or high dynamic environments, and can effectively deal with the complex spatial relationship problems between key frames caused by rapid motion or drastic changes, and can improve the accuracy of closed-loop detection in complex environments and / or high dynamic environments through distortion detection algorithms, and effectively reduce the impact of false closed loops on the system. Whether the closed loop is correct is identified by detecting the spatial distortion between each adjacent key frame, and abnormal distortion between key frames is detected and the false closed loop corresponding to the abnormal distortion is eliminated, to improve the accuracy of closed-loop detection, and ensure the positioning accuracy, robustness and stability of the system in complex environments and / or high dynamic environments. Wherein, the method includes steps S11, S12, S13, S14, S15 and S16, and specifically includes the following steps:
[0049] Step S11, in the process of closed-loop detection in the global map of the VSLAM system, all key frames between the loop key frame and the current key frame in the same map are obtained; it should be noted that the loop key frame is the key frame that is most similar to the history of the current key frame among all the historical data collected.
[0050] Step S12, respectively calculating the relative displacement between each pair of adjacent key frames;
[0051] Step S13, based on the relative displacement between each pair of adjacent key frames, determining at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information;
[0052] Step S14, based on the key frame information, eliminating the at least one erroneous closed loop from the loop key frame to the current key frame;
[0053] Step S15, adjusting the positions and postures of the remaining valid key frames between the loop key frame and the current key frame based on the abnormal distortion ratio;
[0054] Step S16, updating the adjusted position and posture of each of the valid key frames in real time to the global map of the VSLAM system.
[0055] Through the above steps S11 to S16, the identification and detection of the erroneous closed loops corresponding to the abnormal distortion ratios are realized, and the erroneous closed loops can be effectively eliminated, which not only increases the probability of effectively eliminating erroneous closed loops in complex environments and / or high dynamic environments, but also improves the positioning accuracy, robustness and stability of the VSLAM system.
[0056] Following the above embodiment of the present application, the step S11 obtains all key frames from the loop key frame to the current key frame in the same map during the loop detection process in the global map of the VSLAM system, specifically including:
[0057] In a closed-loop detection process in a global map of the VSLAM system, obtaining the pose data of the current key frame and the pose data of the corresponding loop-back key frame;
[0058] Based on the map identifier carried in the pose data, determining whether the current keyframe and the loop keyframe belong to the same map;
[0059] If the map identifier carried by the pose data of the current key frame is the same as the map identifier carried by the pose data of the loop key frame, it is determined that the current key frame and the loop key frame belong to the same map, and all key frames from the loop key frame to the current key frame are obtained.
[0060] For example, in the closed-loop detection process in the global map of the VSLAM system, due to the inter-frame constraints in the map, errors and precision loss may occur. By obtaining the pose data of the current key frame and the pose data of the loopback key frame associated with the current key frame, the purpose of optimizing the key frames of the related links is achieved to avoid misconnection. Before performing data analysis of the current key frame and the loopback key frame, it is necessary to ensure that the current key frame and the loopback key frame belong to the same map. Since the pose data of the current key frame and the loopback key frame both carry their own map identifiers, it is determined whether the current key frame and the loopback key frame belong to the same map. When the map identifier carried by the pose data of the current key frame and the map identifier carried by the pose data of the loopback key frame are the same or consistent, it is determined that the current key frame and the loopback key frame belong to the same map, and all key frames from the loopback key frame to the current key frame are obtained, thereby realizing data collection of the current key frame and the loopback key frame and selection of all key frames from the loopback key frame to the current key frame in the same map.
[0061] In this embodiment, after determining whether the current key frame and the loop key frame belong to the same map based on the map identifier carried in the pose data, the method further includes:
[0062] If the map identifier carried by the pose data of the current keyframe is different from the map identifier carried by the pose data of the loop keyframe, it is determined that the current keyframe and the loop keyframe belong to different maps. That is, when the map identifier carried by the current keyframe is different or inconsistent with the map identifier carried by the loop keyframe, it is determined that the current keyframe and the loop keyframe do not belong to the same map, and the closed loop detection process from the loop keyframe to the current keyframe can be directly ended.
[0063] Following the above embodiment of the present application, the step S13 determines at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information based on the relative displacement between each pair of adjacent key frames, specifically including:
[0064] Projecting the relative displacement between each pair of adjacent key frames onto the loop direction, and calculating the cumulative projection distance of all key frames from the loop key frame to the current key frame;
[0065] Determine whether the cumulative projection distance is greater than or equal to a preset distance threshold.
[0066] If not, perform frame-by-frame analysis on the relative displacement between each pair of adjacent key frames, and calculate the distortion ratio of each pair of adjacent key frames.
[0067] Based on the distortion ratio of each pair of adjacent key frames, determine at least one incorrect closed loop formed between the loopback key frame and the current key frame, and its corresponding abnormal distortion ratio and key frame information.
[0068] For example, first, obtain all the key frames (N frames, where N is a positive integer greater than or equal to 1) between the loopback key frame K0 and the current key frame K1. Then, there are a total of (N + 2) key frames between the loopback key frame K0 and the current key frame K1. Calculate the relative displacement frame by frame. Among these (N + 2) key frames, for each pair of adjacent key frames, calculate the relative displacement between each pair of adjacent key frames, and (N + 1) relative displacements can be obtained. The relative displacement is used to reflect the spatial transformation relationship between each pair of adjacent key frames. Then, project the relative displacement between each pair of adjacent key frames onto the loopback direction, calculate the cumulative projection distance of the (N + 2) key frames from the loopback key frame to the current key frame, and analyze the cumulative projection distance to determine whether there is an abnormality in the closed loop from the loopback key frame to the current key frame through the cumulative projection distance. After that, determine whether the cumulative projection distance is greater than or equal to the preset distance threshold. If not, when the cumulative projection distance is greater than or equal to the preset distance threshold, it indicates that there is abnormal spatial distortion, so it is determined that the entire closed loop formed between the loopback key frame and the current key frame is an invalid closed loop or an incorrect closed loop, that is, it is indicated that the entire closed loop formed between the loopback key frame and the current key frame is invalid, and the closed loop detection process from the loopback key frame to the current key frame can be directly ended.
[0069] Next, in this embodiment, after determining whether the cumulative projection distance is greater than or equal to the preset distance threshold, if not, that is, the cumulative projection distance is less than the preset distance threshold, indicating that the closed loop from the loop key frame to the current key frame is valid, the subsequent steps can be performed: the relative displacement between each pair of adjacent key frames is analyzed frame by frame, the distortion ratio of each pair of adjacent key frames is calculated, and based on the distortion ratio of each pair of adjacent key frames, at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information are determined, so as to detect the abnormal distortion from the loop key frame to the current key frame, and determine the erroneous closed loop and its corresponding abnormal distortion ratio and key frame information, so as to detect and eliminate the erroneous closed loop in real time during the subsequent optimization process, thereby ensuring the stability of the system.
[0070] In this embodiment, during the closed-loop optimization detection process, by analyzing the relative displacement and cumulative projection distance of each pair of adjacent key frames from the loop key frame to the current key frame, it is determined whether there is abnormal distortion, thereby identifying the connection of erroneous closed loops formed by adjacent key frames corresponding to the abnormal distortion ratio, which can significantly reduce the generation of erroneous closed loops and avoid the problem of mis-optimization caused by feature matching errors in traditional methods.
[0071] Here, by detecting the spatial distortion from the loop keyframe to the current interference witness, it is possible to identify whether the closed loop between adjacent keyframes is wrong or correct, especially in complex and / or highly dynamic environments, effectively reducing the probability of incorrect closed loops, thereby improving the positioning accuracy and stability of the system.
[0072] Following the above embodiment of the present application, the step S13 of determining at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information based on the distortion ratio of each pair of adjacent key frames specifically includes:
[0073] Determine whether the distortion ratio of each pair of adjacent key frames exceeds a preset burden ratio.
[0074] If so, the distortion ratio corresponding to the preset burden ratio is determined as an abnormal distortion ratio, and the closed loop formed between adjacent key frames corresponding to the abnormal distortion ratio is determined as an erroneous closed loop, and the abnormal distortion ratio and key frame information are recorded, wherein the key frame information is the information of the next key frame in the adjacent key frames corresponding to the abnormal distortion ratio;
[0075] If not, the next key frame among the adjacent key frames corresponding to the distortion ratio that does not exceed the preset burden ratio is determined as a valid key frame.
[0076] For example, after calculating the distortion ratio of each pair of adjacent key frames, it is necessary to determine whether the distortion ratio of each pair of adjacent key frames exceeds the preset burden ratio. If there is a distortion ratio that exceeds the preset burden ratio, the distortion ratio corresponding to the distortion ratio exceeding the preset burden ratio is determined as an abnormal distortion ratio, and the closed loop formed between the adjacent key frames corresponding to the abnormal distortion ratio is determined as an erroneous closed loop, and the abnormal distortion ratio and key frame information are recorded, wherein the key frame information is the information of the next key frame in the adjacent key frames corresponding to the abnormal distortion ratio, so that when abnormal distortion is detected, the corresponding abnormal distortion ratio and key frame information are recorded for reference in the subsequent closed-loop optimization process; on the other hand, if the determined distortion ratio does not exceed the preset burden ratio, the next key frame in the adjacent key frames corresponding to the distortion ratio that does not exceed the preset burden ratio is determined as a valid key frame, so that the valid key frames from the loop key frame to the current key frame are retained.
[0077] Following the above-mentioned embodiment of the present application, in step S13, based on the relative displacement between each pair of adjacent key frames, at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information are determined, and then step S14 eliminates the at least one erroneous closed loop from the loop key frame to the current key frame based on the key frame information, thereby realizing timely elimination of the connection of each erroneous closed loop for the detected erroneous closed loop, preventing the erroneous closed loop from having an adverse effect on the subsequent map optimization process, and achieving the purpose of eliminating the erroneous closed loop; after eliminating the erroneous closed loop, step S15 adjusts the position and posture of each remaining valid key frame from the loop key frame to the current key frame based on the abnormal distortion ratio; and in step S16, the adjusted position and posture of each valid key frame are updated in real time to the global map of the VSLAM system, thereby realizing the optimized recalculation of the closed loop formed from the loop key frame to the current key frame and the adjustment and real-time update of the global map.
[0078] Following the above embodiment of the present application, the step S15 adjusts the position and posture of each remaining valid key frame from the loop key frame to the current key frame based on the abnormal distortion ratio, specifically including:
[0079] Based on the abnormal distortion ratio, recalculate the relative posture relationship between the loop key frame and the current key frame and the remaining valid key frames;
[0080] The position and posture of each of the valid key frames in the global map are adjusted based on the relative position and posture relationship.
[0081] For example, after eliminating the erroneous closed loop formed from the loop keyframe to the current keyframe, when optimizing the remaining correct closed loops, first, based on the recorded abnormal distortion ratio, recalculate the relative pose relationship from the loop keyframe to the current keyframe and between the remaining valid keyframes to ensure the global consistency of the map; then, based on the recalculated relative pose relationship between the valid keyframes, adjust the position and posture of each valid keyframe in the global map to optimize the remaining correct closed loops and adjust the global map; finally, in step S16, the position and posture of each valid keyframe obtained after optimization and adjustment are updated in real time to the global map of the VSLAM system, which not only realizes the real-time update of the global map of the VSLAM system, but also ensures the positioning accuracy and robustness of the VSLAM system in complex environments.
[0082] In an embodiment of the present application, by analyzing the spatial distortion between key frames, calculating the cumulative projection distance and the distortion ratio of each pair of adjacent key frames calculated frame by frame, it is possible to effectively identify and eliminate the connections of erroneous closed loops, significantly reducing the map distortion and positioning deviation caused by erroneous closed loops, and improving the accuracy of closed loop detection in the VSLAM system and the overall map quality.
[0083] Since traditional closed-loop detection in a dynamic environment is easily affected by factors such as lighting changes, dynamic objects and occlusions, thereby introducing erroneous closed-loop connections, in an embodiment of the present application, by analyzing the relative displacement between each pair of adjacent key frames frame by frame, it is possible to identify and eliminate erroneous closed loops in real time in a highly dynamic environment, thereby enhancing the robustness of the system in dynamic and complex environments.
[0084] In the embodiments of the present application, the accuracy and real-time performance of the closed-loop optimization detection process are ensured by real-time detection and elimination of erroneous closed loops, so that the system can quickly identify erroneous closed loops during map construction and positioning, and make corresponding optimization adjustments, thereby improving the overall real-time and stability of the system.
[0085] Compared with existing closed-loop detection methods based on deep learning, this application does not rely on a large amount of labeled data and reduces dependence on external training data, which enables it to be more lightweight and applied on resource-constrained hardware platforms, and is suitable for real-time closed-loop detection needs in various scenarios.
[0086] In an embodiment of the present application, after eliminating the erroneous closed loop, the relative relationship between the postures of the key frames is recalculated, and the global map is optimized and adjusted, which not only makes the map construction of the entire VSLAM system more consistent and accurate, but also significantly reduces the system positioning error and map inaccuracy caused by erroneous closed loops. Through innovative closed-loop error identification methods, the positioning accuracy, real-time and robustness of the VSLAM system are also significantly improved, which is particularly suitable for robot positioning and map construction tasks in highly dynamic and complex environments.
[0087] According to another aspect of the present application, a non-volatile storage medium is also provided, on which computer-readable instructions are stored. When the computer-readable instructions can be executed by a processor, the processor implements the closed-loop optimization detection method based on the VSLAM system as described above.
[0088] According to another aspect of the present application, a closed-loop optimization detection device based on a VSLAM system is also provided, wherein the device comprises:
[0089] one or more processors;
[0090] A computer readable medium for storing one or more computer readable instructions,
[0091] When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the closed-loop optimization detection method based on the VSLAM system as described above.
[0092] Here, for the detailed contents of each embodiment of the closed-loop optimization detection device based on the VSLAM system, please refer to the corresponding part of the embodiment of the closed-loop optimization detection method based on the VSLAM system mentioned above, which will not be repeated here.
[0093] In summary, the present application obtains all key frames from the loop key frame to the current key frame in the same map during the loop detection process in the global map of the VSLAM system; calculates the relative displacement between each pair of adjacent key frames respectively; determines at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information based on the relative displacement between each pair of adjacent key frames; based on the key frame information, eliminates the at least one erroneous closed loop from the loop key frame to the current key frame; based on the abnormal distortion ratio, adjusts the position and posture of each remaining valid key frame from the loop key frame to the current key frame; updates the adjusted position and posture of each valid key frame in real time to the global map of the VSLAM system, realizes the identification and detection of the erroneous closed loop corresponding to the abnormal distortion ratio, and can effectively eliminate the erroneous closed loop, which can not only improve the probability of effectively finding the erroneous closed loop in complex environment and / or high dynamic environment, but also improve the positioning accuracy, robustness and stability of the VSLAM system.
[0094] It should be noted that the present application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.
[0095] In addition, a part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. The program instruction for calling the method of the present application may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in a working memory of a computer device that runs according to the program instruction. Here, according to an embodiment of the present application, a device is included, the device including a memory for storing computer program instructions and a processor for executing program instructions, wherein, when the computer program instruction is executed by the processor, the device is triggered to run the method and / or technical solution based on the aforementioned multiple embodiments according to the present application.
[0096] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is limited by the attached claims rather than the above description, so it is intended to include all changes that fall within the meaning and scope of the equivalent elements of the claims in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.
Claims
1. A closed-loop optimization detection method based on a VSLAM system, wherein: The method comprises: In the process of loop closure detection in the global map of the VSLAM system, all key frames between the loop closure key frame and the current key frame in the same map are obtained; Calculate the relative displacement between each pair of adjacent key frames respectively; Based on the relative displacement between each pair of adjacent key frames, determining at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information; Based on the key frame information, eliminating the at least one erroneous closed loop from the loop key frame to the current key frame; Based on the abnormal distortion ratio, adjusting the positions and postures of the remaining valid key frames from the loop key frame to the current key frame; The adjusted positions and postures of each of the valid key frames are updated in real time to the global map of the VSLAM system.
2. The method according to claim 1, wherein: In the loop closure detection process in the global map of the VSLAM system, all key frames between the loop closure key frame and the current key frame in the same map are obtained, including: In a closed-loop detection process in a global map of the VSLAM system, obtaining the pose data of the current key frame and the pose data of the corresponding loop-back key frame; Based on the map identifier carried in the pose data, determining whether the current keyframe and the loop keyframe belong to the same map; If the map identifier carried by the pose data of the current key frame is the same as the map identifier carried by the pose data of the loop key frame, it is determined that the current key frame and the loop key frame belong to the same map, and all key frames from the loop key frame to the current key frame are obtained.
3. The method according to claim 2, wherein: After determining whether the current key frame and the loop key frame belong to the same map based on the map identifier carried in the pose data, the method further includes: If the map identifier carried by the pose data of the current key frame is different from the map identifier carried by the pose data of the loop key frame, it is determined that the current key frame and the loop key frame belong to different maps.
4. The method according to claim 3, wherein: The determining, based on the relative displacement between each pair of adjacent key frames, at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information comprises: Projecting the relative displacement between each pair of adjacent key frames onto the loop direction, and calculating the cumulative projection distance of all key frames from the loop key frame to the current key frame; Determine whether the accumulated projection distance is greater than or equal to a preset distance threshold, If not, then analyzing the relative displacement between each pair of adjacent key frames frame by frame, and calculating the distortion ratio of each pair of adjacent key frames; Based on the distortion ratio of each pair of adjacent key frames, at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information are determined.
5. The method according to claim 4, wherein: The determining, based on the distortion ratio of each pair of adjacent key frames, at least one erroneous closed loop formed from the loop key frame to the current key frame and its corresponding abnormal distortion ratio and key frame information includes: Determine whether the distortion ratio of each pair of adjacent key frames exceeds a preset burden ratio. If so, the distortion ratio corresponding to the preset burden ratio is determined as an abnormal distortion ratio, and the closed loop formed between adjacent key frames corresponding to the abnormal distortion ratio is determined as an erroneous closed loop, and the abnormal distortion ratio and key frame information are recorded, wherein the key frame information is the information of the next key frame in the adjacent key frames corresponding to the abnormal distortion ratio; If not, the next key frame among the adjacent key frames corresponding to the distortion ratio that does not exceed the preset burden ratio is determined as a valid key frame.
6. The method according to claim 4, wherein: After determining whether the accumulated projection distance is greater than or equal to a preset distance threshold, the method further includes: If so, it is used to indicate that the entire closed loop formed from the loop key frame to the current key frame is invalid.
7. The method according to any one of claims 1 to 6, wherein: The adjusting, based on the abnormal distortion ratio, the positions and postures of the remaining valid key frames between the loop key frame and the current key frame includes: Based on the abnormal distortion ratio, recalculate the relative posture relationship between the loop key frame and the current key frame and between the remaining valid key frames; The position and posture of each of the valid key frames in the global map are adjusted based on the relative position and posture relationship.
8. A non-volatile storage medium having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executable by a processor, the processor is enabled to implement the method according to any one of claims 1 to 7.
9. A closed-loop optimization detection device based on a VSLAM system, wherein: The equipment includes: one or more processors; A computer readable medium for storing one or more computer readable instructions, When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.