Loop detection method, device and storage medium

Abstract

The present application relates to the field of robotics, and discloses a loop detection method, device, and storage medium, the method comprising: obtaining the image features, point cloud geometric features, and key frame graph including the current key frame of the current key frame; determining the global topological structure features of the current key frame according to the key frame graph; determining the global consistency descriptor of the current key frame according to the image features, point cloud geometric features, and global topological structure features; performing loop detection according to the first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame. In the above manner, the present application improves the accuracy of loop detection.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of robotics technology, and in particular to a loop detection method, device, and storage medium. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) technology can create maps and estimate the position of sensors in real time in unknown environments. This technology is widely used in robotics, unmanned driving, drones, augmented reality (AR) and other fields. The application of SLAM technology is particularly important in complex scenes and multi-floor environments. In the SLAM system, loop detection is used to identify whether the device has revisited the same location before, and to correct the previous accumulated errors accordingly to ensure the consistency and accuracy of the map and positioning information.

[0003] However, most existing loop detection technologies rely on local feature matching algorithms, which have high accuracy in simple environments. However, in complex multi-story buildings or similar environments, local feature matching algorithms are difficult to accurately distinguish feature points at different locations, resulting in reduced accuracy of loop detection. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application provide a loop detection method, device and storage medium, which improve the accuracy of loop detection.

[0005] According to one aspect of an embodiment of the present application, a loop detection method is provided, which includes: obtaining image features, point cloud geometric features and a key frame graph including the current key frame; determining the global topological structure features of the current key frame based on the key frame graph; determining the global consistency descriptor of the current key frame based on the image features, point cloud geometric features and global topological structure features; and performing loop detection based on a first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame.

[0006] In an optional manner, a key frame graph includes a current key frame and multiple neighboring key frames of the current key frame, and the key frame graph includes connecting edges between the current key frame and multiple neighboring key frames; determining the global topological structure features of the current key frame based on the key frame graph further includes: obtaining features of each neighboring key frame, the features being image features, point cloud geometric features, or fusion features of image features and point cloud geometric features; calculating the weight of each neighboring key frame to the current key frame, and obtaining the weight corresponding to each neighboring key frame; determining the global topological structure features of the current key frame based on the features of each neighboring key frame and the weight corresponding to each neighboring key frame.

[0007] In an optional manner, the global topological structure characteristics of the current key frame are: ,in, Represents the global topological structure characteristics of the current keyframe, represents the set of neighborhood keyframes, Represents neighborhood keyframes For the current keyframe The weight of Represents a neighborhood keyframe characteristics.

[0008] In an optional manner, the global consistency descriptor of the current key frame is: ,in, Represents the global consistency descriptor of the current keyframe, Represents the image features of the current key frame, Represents the point cloud geometric features of the current keyframe, Represents the global topological structure characteristics of the current keyframe, , , Respectively , , The weight of .

[0009] In an optional manner, loop detection is performed based on a first distance between a global consistency descriptor of a current key frame and a global consistency descriptor of a historical key frame, further comprising: calculating a first distance between a global consistency descriptor of a current key frame and a global consistency descriptor of a historical key frame; determining whether the first distance is less than a first preset distance threshold; if the first distance is less than the first preset distance threshold, determining that a loop is detected; if the first distance is greater than or equal to the first preset distance threshold, determining that no loop is detected.

[0010] In an optional manner, after determining that a loop is detected, the method also includes: determining a historical key frame whose first distance is less than a first preset distance threshold as a target historical key frame; determining a common neighborhood key frame of the current key frame and the target historical key frame; determining a first topological structure feature between the current key frame and the common neighborhood key frame; determining a second topological structure feature between the target historical key frame and the common neighborhood key frame; and performing loop closure detection between the current key frame and the target historical key frame based on a second distance between the first topological structure feature and the second topological structure feature.

[0011] In an optional manner, performing loop closure detection between the current key frame and the target historical key frame based on the second distance between each first topological structure feature and each second topological structure feature further includes: calculating the second distance between each first topological structure feature and each second topological structure feature to obtain multiple second distances; determining whether the sum of all second distances is less than a second preset distance threshold; if the sum of the second distances is less than the second preset distance threshold, determining that the current key frame and the target historical key frame are loop closed; if the sum of the second distances is greater than or equal to the second preset distance threshold, determining that the current key frame and the target historical key frame are not loop closed.

[0012] In an optional manner, the first topological structure is characterized by: ,in, Indicates the current key frame. represents the common neighborhood keyframe, represents the first topological structure feature, Represents common neighborhood keyframes The weight of the current keyframe, Represents common neighborhood keyframes Features, common neighborhood keyframes The feature is an image feature, a point cloud geometric feature, or a fusion feature of the image feature and the point cloud geometric feature; the second topological structure feature is: ,in, represents the target historical keyframe, represents the second topological structure feature, Represents common neighborhood keyframes The weight of the target history keyframe.

[0013] According to another aspect of an embodiment of the present application, an electronic device is provided, including: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the loop detection method provided in any of the above embodiments.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the loop detection method provided by any of the above embodiments is implemented.

[0015] The embodiment of the present application determines the global topological structure features of the current key frame through a key frame graph including the current key frame, and then obtains the global consistency descriptor of the current key frame by fusing the image features, point cloud geometric features and global topological structure features of the current key frame, so that the global consistency descriptor can integrate the local features and global topological features of the current key frame. Finally, loop detection is performed through the first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame. The local features and global topological features of the current key frame and the historical key frames can be comprehensively compared to implement global consistency constraints on the key frames, ensuring that the loop detection not only matches the local features but also maintains consistency in the overall structure, reducing false detections due to the similarity of local features, thereby improving the accuracy of loop detection.

[0016] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0018] Figure 1 A schematic diagram of a loop detection method according to an embodiment of the present invention is shown;

[0019] Figure 2 A schematic diagram of a loop closure detection method according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of the structure of a loop detection device provided in an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0023] Laser-based SLAM technology can provide high-precision distance measurement, and therefore has high accuracy in mapping and positioning. Vision-based SLAM technology can provide rich environmental information, including color, texture, shape, etc., which helps to identify and classify objects in the environment. However, both laser SLAM and visual SLAM may encounter the problem of cumulative errors in long-term operation and complex environments. In order to solve the problem of cumulative errors, a loop detection mechanism is introduced into the SLAM system. The loop detection mechanism is used to identify whether the device has revisited the same location before, and to correct the previous accumulated errors accordingly to ensure the consistency and accuracy of the map and positioning information.

[0024] Most existing loop detection technologies rely on feature matching, especially local feature matching. In visual SLAM, commonly used local feature matching methods involve visual descriptors such as ORB (Oriented FAST and Rotated BRIEF) and SIFT (Scale-Invariant Feature Transform). These descriptors extract key feature points in the image and match them to determine whether two scenes are similar. In laser SLAM, commonly used point cloud registration algorithms include ICP (Iterative Closest Point) and NDT (Normal Distributions Transform). These algorithms process the point cloud data obtained by laser scanning to achieve precise alignment of point clouds at different scanning positions.

[0025] The above local feature matching methods usually only focus on the similarity between the current keyframe and the historical keyframe, and the accuracy of loop detection in simple environments is relatively high. However, in complex multi-story buildings or similar environments, the environmental structures are often highly similar, and there are a large number of repeated textures and shapes. This similarity makes it difficult for the local feature matching algorithm to accurately distinguish feature points at different locations, increasing the risk of false detection. For example, in two different floors, there may be similar corridor structures or decorative patterns. These similar feature points may be mistakenly matched together, reducing the accuracy of loop detection.

[0026] Based on this, the present application provides a loop detection method, which not only focuses on the local features of the current key frame and the historical key frame, but also introduces the global topological information of the key frame, and performs multimodal feature fusion of the global topological information of the key frame, the visual features of the key frame and the laser point cloud features before performing loop detection, thereby realizing the global consistency constraint of the key frame, ensuring that the loop detection not only matches the local features but also remains consistent in the overall structure, reducing false detections caused by the similarity of local features, thereby improving the accuracy of loop detection.

[0027] Figure 1 The flowchart of the loop detection method provided in the embodiment of the present application is shown. The method is executed by electronic devices such as robots, servers, smart phones, PCs, tablet computers, portable computers, etc. The embodiment of the present application is described by taking robots as an example, which does not constitute a limitation. Figure 1 As shown, the method comprises the following steps:

[0028] Step 100: Obtain image features, point cloud geometric features, and a key frame graph including the current key frame.

[0029] A keyframe is a representative frame selected during the SLAM process. The current keyframe is the image frame that the SLAM system is processing at the current moment, that is, the frame that was recently marked as a keyframe. It contains enough information, such as corners, edges, etc., to estimate the robot's camera position and build an environment map.

[0030] Among them, the image feature of the current key frame is an n-dimensional real vector extracted from the image, and its extraction formula is:

[0031] f_v=FeatureExtract_v(I),

[0032] Among them, f_v represents the image features of the current key frame, I represents the image, and FeatureExtract_v(I) means extracting features from image I.

[0033] The point cloud geometric feature of the current key frame is an m-dimensional real vector extracted from the point cloud, and its extraction formula is:

[0034] f_l=FeatureExtract_l(P),

[0035] Among them, f_l represents the point cloud geometric features of the current key frame, P represents the point cloud, and FeatureExtract_l(P) means extracting features from the point cloud P.

[0036] Keyframe Graph is a data structure used to represent the relationship between keyframes in SLAM system. The keyframe graph including the current keyframe can be expressed as G=(V,E), where V represents the current keyframe and the set of multiple neighboring keyframes of the current keyframe, and E represents the connecting edges between the current keyframe and its multiple neighboring keyframes. Neighboring keyframes refer to keyframes that are adjacent to the current keyframe in time and space. Connecting edges represent the relative pose relationship between two keyframes, which can be calculated by visual odometer or optimization algorithm. The current keyframe and its neighboring keyframes are directly connected through connecting edges.

[0037] Step 200: Determine the global topological structure features of the current key frame according to the key frame graph.

[0038] The global topological structure feature of the current keyframe is a description of the position and connection relationship of the current keyframe in the entire map, which helps to identify whether the current keyframe represents the same position as a previous keyframe.

[0039] Specifically, step 200 includes the following steps:

[0040] Step 210: Acquire features of each neighborhood keyframe, where the features are image features, point cloud geometric features, or fusion features of image features and point cloud geometric features.

[0041] Among them, the image features of the neighborhood key frame are extracted from its image, and the point cloud geometric features are extracted from its point cloud. The fusion features of the neighborhood key frame can be obtained by weighted summation of the image features and point cloud geometric features of the neighborhood key frame.

[0042] Step 220: Calculate the weight of each neighborhood key frame to the current key frame to obtain the weight corresponding to each neighborhood key frame.

[0043] Specifically, a feature matching algorithm can be used to match the feature points of the current key frame with the feature points of the neighboring key frames, and then the number of matching feature point pairs between each neighboring key frame and the current key frame is calculated, and finally the weight corresponding to each neighboring key frame is determined according to the number of matches. The more the number of matches between the neighboring key frame and the current key frame, the higher the weight corresponding to the neighboring key frame.

[0044] Step 230: Determine the global topological structure features of the current key frame according to the features of each neighborhood key frame and the weight corresponding to each neighborhood key frame.

[0045] The calculation formula of the global topological structure characteristics of the current key frame is as follows:

[0046] ,

[0047] in, Represents the global topological structure characteristics of the current keyframe, represents the set of neighborhood keyframes, Represents a neighborhood keyframe For the current keyframe The weight of Represents a neighborhood keyframe characteristics.

[0048] The weight of each neighborhood keyframe to the current keyframe is multiplied by the features of the neighborhood keyframe to obtain the weighted features of each neighborhood keyframe, and then the weighted features of each neighborhood keyframe are added together to obtain the global topological structure features of the current keyframe. The global topological structure features of the current keyframe integrate the features of all its neighborhood keyframes and their relative importance to the current keyframe, and can more accurately describe the position and role of the current keyframe in the global map.

[0049] In the embodiment of the present application, loop detection is performed through the image features, point cloud geometric features and global topological structure features of the key frame, which greatly reduces the amount of data to be processed. In addition, in the process of calculating the global consistency descriptor of the current key frame, only the neighboring key frames of the current key frame need to be selected for calculation, without processing all key frames, which can reduce the sampling amount of key frames and the amount of calculation, improve the processing speed of the SLAM system, and meet the real-time requirements of robot navigation.

[0050] Step 300: Determine a global consistency descriptor of the current key frame according to image features, point cloud geometric features and global topological structure features.

[0051] The calculation formula of the global consistency descriptor is as follows:

[0052] ,

[0053] in, Represents the global consistency descriptor of the current keyframe, Represents the image features of the current key frame, Represents the point cloud geometric features of the current keyframe, Represents the global topological structure characteristics of the current keyframe, , , Respectively , , They determine the importance of each feature in the final globally consistent descriptor.

[0054] The global consistency descriptor combines the local features and global topological structure features of the current keyframe. It not only considers the local visual features, but also the distribution and relationship of these features in the global map, which can more effectively identify places that have been visited.

[0055] Step 400: Perform loop detection based on a first distance between a global consistency descriptor of a current key frame and a global consistency descriptor of a historical key frame.

[0056] First, obtain the global consistency descriptor of the historical key frame. During the operation of the SLAM system, the key frame will be selected according to certain standards, the global consistency descriptor of each key frame will be calculated, and the global consistency descriptor of each key frame will be stored. Among them, the calculation process of the global consistency descriptor of each key frame can refer to the calculation process of the global consistency descriptor of the current key frame, which will not be repeated here.

[0057] Then, the first distance between the global consistency descriptor of the current keyframe and the global consistency descriptor of each historical keyframe is calculated , the calculation formula is as follows:

[0058] ,

[0059] in, Represents the global consistency descriptor of the current keyframe, A globally consistent descriptor representing a historical keyframe.

[0060] For each historical key frame, the first distance corresponding to the historical key frame can be calculated.

[0061] Finally, for each historical key frame corresponding to the first distance, determine whether the first distance is less than the first preset distance threshold, if the first distance is less than the first preset distance threshold, it is determined that a loop is detected; if the first distance is greater than or equal to the first preset distance threshold, it is determined that no loop is detected. In this way, it can be determined whether the current key frame is in the same position in space as a certain historical key frame.

[0062] In this step, KD-Tree or hash index can be used to perform a fast neighbor search based on the global consistency descriptor of the current key frame to find potential matching historical key frames. After that, loop detection is performed based on the first distance between the global consistency descriptor of the current key frame and the global consistency descriptors of these historical key frames, which can reduce computational overhead.

[0063] The embodiment of the present application determines the global topological structure features of the current key frame through a key frame graph including the current key frame, and then obtains the global consistency descriptor of the current key frame by fusing the image features, point cloud geometric features and global topological structure features of the current key frame, so that the global consistency descriptor can integrate the local features and global topological features of the current key frame. Finally, loop detection is performed through the first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame. The local features and global topological features of the current key frame and the historical key frames can be comprehensively compared to implement global consistency constraints on the key frames, ensuring that the loop detection not only matches the local features but also maintains consistency in the overall structure, reducing false detections due to the similarity of local features, thereby improving the accuracy of loop detection.

[0064] In highly repetitive environments, such as large shopping malls, office buildings, or residential areas, there may be a large number of repeated elements (such as doors, windows, corridors, etc.), which dominate the global features, making the features at different locations look very similar, despite the fact that Figure 1 The method shown, that is, after combining local features and global topological structure features for loop detection, may still lead to false detection due to noise, places with similar appearance or other factors.

[0065] In order to avoid false detection, a closed loop verification is required for the detected potential loop to determine whether the loop detection is accurate, that is, to determine whether the device has actually returned to its previous position. Figure 2 , Figure 2 The flowchart of the loop closure detection method provided by the embodiment of the present application is shown. Figure 2 The steps shown are performed after the loop is detected in step 400. As shown in FIG. 2 , the loop closure detection method includes the following steps:

[0066] Step 410: Determine a historical key frame whose first distance is less than a first preset distance threshold as a target historical key frame.

[0067] The target historical keyframe is a potential loop, which can be one, two or even more.

[0068] Step 420: Determine common neighborhood key frames of the current key frame and the target historical key frame.

[0069] A common neighborhood keyframe refers to a keyframe that is both a neighborhood keyframe of the current keyframe and a neighborhood keyframe of the target historical keyframe, and there are usually multiple common neighborhood keyframes.

[0070] After determining the target historical keyframe, a keyframe graph including the target historical keyframe can be determined, and then the neighborhood keyframes of the target historical keyframe can be determined. Afterwards, by comparing the neighborhood keyframes of the current keyframe with the neighborhood keyframes of the target historical keyframe, all common neighborhood keyframes shared by the current keyframe and the target historical keyframe can be determined.

[0071] Step 430: Determine a first topological structure feature between the current key frame and a common neighborhood key frame.

[0072] In this step, the first topological structure feature between the current key frame and each common neighborhood key frame is calculated in turn, and the first topological structure feature corresponding to each common neighborhood key frame can be obtained. The calculation formula of the first topological structure feature is as follows:

[0073] ,

[0074] in, Indicates the current key frame. represents the common neighborhood keyframe, Indicates the current keyframe With common neighbor keyframes The first topological feature between Represents common neighborhood keyframes For the current keyframe The weight of Represents common neighborhood keyframes Features, common neighborhood keyframes The features are image features, point cloud geometric features, or fusion features of image features and point cloud geometric features.

[0075] Step 440: Determine a second topological structure feature between the target historical key frame and the common key frame.

[0076] Specifically, the second topological structure feature between the target historical key frame and each common neighborhood key frame is calculated in sequence, and the second topological structure feature corresponding to each common neighborhood key frame can be obtained. The calculation formula of the second topological structure feature is as follows:

[0077] ,

[0078] in, represents the target historical keyframe, Represents the target history keyframe With common neighbor keyframes The second topological structure feature between Represents common neighborhood keyframes To target history keyframe The weight of .

[0079] Step 450: Perform loop closure detection between the current key frame and the target historical key frame according to the second distance between the first topological structure feature and the second topological structure feature.

[0080] First, for each common neighborhood keyframe, the second distance between the first topological structure feature and the second topological structure feature is calculated to obtain the second distance corresponding to each common neighborhood keyframe. Then, the second distance corresponding to each common neighborhood keyframe is added to obtain the sum of the second distances. The sum of the second distances can be used to detect the topological consistency between the current keyframe and the target historical keyframe. Specifically, the calculation formula for the sum of the second distances is as follows:

[0081] ConsistencyCheck(v_t,v_j)=∑(k)||T(v_t,v_k)-T(v_j,v_k)||

[0082] Among them, ConsistencyCheck(v_t,v_j) represents the sum of the second distances, and ∑(k) represents the sum of the second distances corresponding to all common neighborhood key frames v_k.

[0083] Then, determine whether the sum of the second distances is less than the second preset distance threshold. If the sum of the second distances is less than the second preset distance threshold, it is determined that the current key frame and the target historical key frame are loop-closed, that is, the current key frame coincides with the target historical key frame; if the sum of the second distances is greater than or equal to the second preset distance threshold, it is determined that the current key frame and the target historical key frame are not loop-closed, that is, the target historical key frame is a falsely detected loop.

[0084] By comparing the relationship between the current keyframe and multiple neighboring keyframes in the target historical keyframe, the key differences between two similar environments are revealed, so that it can be determined whether the current keyframe and the target historical keyframe are loop closed, so as to verify the detected potential loops, further improving the accuracy of loop detection. In addition, by accumulating the transformation differences between multiple neighboring keyframes, the impact of a single transformation error on the result can be reduced. If the neighborhood topological structures of two keyframes are inconsistent, even if some individual transformation differences are not large, the accumulated differences will exceed the threshold, further avoiding the occurrence of false detection.

[0085] Figure 3 FIG. 2 shows a schematic diagram of the structure of a loop detection device provided in an embodiment of the present application. Figure 3As shown, the loop detection device 500 includes: an acquisition module 510, a first determination module 520, a second determination module 530 and a detection module 540. The acquisition module 510 is used to acquire the image features, point cloud geometric features and key frame graph including the current key frame of the current key frame. The first determination module 520 is used to determine the global topological structure features of the current key frame according to the key frame graph. The second determination module 530 is used to determine the global consistency descriptor of the current key frame according to the image features, point cloud geometric features and global topological structure features. The detection module 540 is used to perform loop detection according to the first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame.

[0086] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0087] like Figure 4 As shown, the electronic device may include: a processor (processor) 602 and a memory (memory) 604.

[0088] The memory 604 is used to store a computer program 606. The memory 604 may include a high-speed RAM memory, or may also include a non-volatile memory, such as at least one disk memory. The computer program 606 may include computer executable instructions.

[0089] The processor 602 is configured to execute a computer program 606 to implement the above loop detection method embodiment.

[0090] The processor 602 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the electronic device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0091] An embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above loop detection method embodiment is implemented.

[0092] An embodiment of the present application provides a computer program, which can be executed by a processor to implement the above loop detection method embodiment.

[0093] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above loop detection method embodiment is implemented.

[0094] In several embodiments provided in the present application, if any function is implemented in the form of a software function module / unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, part or all of the technical solution of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server or other electronic device) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store computer program code.

[0095] The algorithm or display provided here are not inherently related to any specific computer, virtual system or other equipment. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application.

[0096] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the claims that list several devices, several units or modules in these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

[0097] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A loop detection method, characterized in that: The method comprises: Acquire image features, point cloud geometric features, and a key frame graph including the current key frame; Determining the global topological structure features of the current key frame according to the key frame graph; the key frame graph includes the current key frame and a plurality of neighboring key frames of the current key frame, and the key frame graph includes connecting edges between the current key frame and a plurality of neighboring key frames; Determine a global consistency descriptor of the current key frame according to the image features, the point cloud geometric features and the global topological structure features; Perform loop detection according to a first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame; The step of determining the global topological structure feature of the current key frame according to the key frame graph further comprises: Acquire features of each of the neighborhood key frames, where the features are image features, point cloud geometric features, or fusion features of image features and point cloud geometric features; Calculating the weight of each of the neighborhood key frames to the current key frame to obtain the weight corresponding to each of the neighborhood key frames; Determining the global topological structure features of the current key frame according to the features of each of the neighborhood key frames and the weight corresponding to each of the neighborhood key frames; The global topological structure characteristics of the current key frame are: ， in, represents the global topological structure characteristics of the current key frame, Indicates the current keyframe The set of neighborhood key frames of Represents a key frame Belongs to the current keyframe Neighborhood keyframes, Represents a neighborhood keyframe For the current key frame The weight of Represents a neighborhood keyframe Features; The global consistency descriptor of the current key frame is: ， in, Represents the global consistency descriptor of the current keyframe, Represents the image features of the current key frame, Represents the point cloud geometric features of the current keyframe, Represents the global topological structure characteristics of the current keyframe, , , Respectively , , The weight of .

2. The method according to claim 1, characterized in that The loop detection is performed according to a first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame, further comprising: Calculating a first distance between the global consistency descriptor of the current key frame and the global consistency descriptor of the historical key frame; Determining whether the first distance is less than a first preset distance threshold; If the first distance is less than the first preset distance threshold, determining that a loop is detected; If the first distance is greater than or equal to the first preset distance threshold, it is determined that no loop is detected.

3. The method according to claim 2, characterized in that After determining that a loop is detected, the method further includes: Determine a historical key frame whose first distance is less than the first preset distance threshold as a target historical key frame; Determine a plurality of common neighboring key frames of the current key frame and the target historical key frame; Determining a first topological structure feature between the current key frame and each of the common neighborhood key frames; Determining a second topological structure feature between the target historical key frame and each of the common neighborhood key frames; A loop closure detection is performed between the current key frame and the target historical key frame according to a second distance between each of the first topological structure features and each of the second topological structure features.

4. The method according to claim 3, characterized in that The loop closure detection between the current key frame and the target historical key frame according to the second distance between each of the first topological structure features and each of the second topological structure features further includes: Calculating a second distance between each of the first topological structure features and each of the second topological structure features to obtain a plurality of second distances; Determining whether the sum of the second distances is less than a second preset distance threshold; If the sum of the second distances is less than the second preset distance threshold, determining that the current key frame and the target historical key frame are loop-closed; If the sum of the second distances is greater than or equal to the second preset distance threshold, it is determined that the current key frame and the target historical key frame are not loop closed.

5. The method according to claim 3, characterized in that: The first topological structure is characterized by: ， in, represents the current key frame, represents the common neighborhood keyframe, represents the first topological structure feature, represents the common neighborhood keyframe The weight of the current key frame, represents the common neighborhood keyframe The common neighborhood keyframe The features are image features, point cloud geometric features, or fusion features of image features and point cloud geometric features; The second topological structure is characterized by: ， in, represents the target historical key frame, represents the second topological structure feature, represents the common neighborhood keyframe The weight of the target historical keyframe.

6. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the loop detection method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the loop detection method according to any one of claims 1 to 5 is implemented.

Images