Loop detection method, device, electronic device, and computer-readable medium

Abstract

Embodiments of the present application disclose a loop detection method, apparatus, electronic device, and computer-readable medium. The method comprises: performing a Euclidean distance transform on a laser subgraph to be tested to obtain a Euclidean distance map; determining feature points in the laser subgraph to be tested based on the Euclidean distance map and obtaining descriptors for the feature points in the laser subgraph to be tested; and performing loop detection on the laser subgraph to be tested based on the descriptors of the feature points in the laser subgraph to be tested and the descriptors of the feature points in historical laser subgraphs to obtain a loop detection result. This embodiment improves the efficiency of loop detection.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and more particularly to a loop detection method, apparatus, electronic device, and computer-readable medium. Background Art

[0002] Loop detection, also known as closed-loop detection, is a detection method that uses a mobile robot to match the current scene with historical scenes to eliminate the system's accumulated errors.

[0003] Existing techniques use scan matching to match the laser data of the current frame with the global map to identify historical scenes that match the current scene. However, due to the large amount of laser data, this method is time-consuming for large maps, resulting in low loop detection efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a loop detection method, apparatus, electronic device, and computer-readable medium to address the technical problem of low efficiency of loop detection in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a loop detection method, which includes: performing a Euclidean distance transformation on a laser sub-image to be measured to obtain a Euclidean distance map; determining feature points in the laser sub-image to be measured based on the Euclidean distance map, and obtaining a descriptor of the feature points in the laser sub-image to be measured; performing loop detection on the laser sub-image to be measured based on the descriptors of the feature points in the laser sub-image to be measured and the descriptors of the feature points in historical laser sub-images to obtain a loop detection result.

[0006] In a second aspect, an embodiment of the present application provides a loop detection device, which includes: a transformation unit, configured to perform Euclidean distance transformation on a laser sub-image to be measured to obtain a Euclidean distance map; a determination unit, configured to determine the feature points in the laser sub-image to be measured based on the Euclidean distance map, and obtain a descriptor of the feature points in the laser sub-image to be measured; a detection unit, configured to perform loop detection on the laser sub-image to be measured based on the descriptors of the feature points in the laser sub-image to be measured and the descriptors of the feature points in the historical laser sub-images to obtain a loop detection result.

[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0009] The loop detection method, apparatus, electronic device, and computer-readable medium provided in the embodiments of the present application perform a Euclidean distance transformation on the laser sub-image to be tested to obtain a Euclidean distance map. The Euclidean distance map is then used to determine the feature points in the laser sub-image to be tested and obtain their descriptors. Subsequently, loop detection is performed on the laser sub-image to be tested based on the descriptors of the feature points in the laser sub-image to be tested and the descriptors of the feature points in the historical laser sub-images, thereby obtaining loop detection results. Because the number of feature points in a laser sub-image is much smaller than the amount of raw data in the laser sub-image, scene matching using the descriptors of the feature points in the laser sub-image can reduce the time required for loop detection compared to directly matching laser data, thereby improving the efficiency of loop detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0011] Figure 1 is a flowchart of an embodiment of a loop detection method according to the present application;

[0012] Figure 2 is a flowchart of another embodiment of the loop detection method according to the present application;

[0013] Figure 3 is a schematic structural diagram of an embodiment of a loop detection device according to the present application;

[0014] Figure 4 It is a structural diagram of a computer system for implementing an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0016] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0017] Please refer to Figure 1, which shows a process 100 of an embodiment of a loop detection method according to the present application. The loop detection method includes the following steps 101 to 103:

[0018] Step 101: Perform Euclidean distance transformation on the laser sub-graph to be measured to obtain a Euclidean distance graph.

[0019] In this embodiment, the execution entity of the loop detection method (such as a server, mobile robot, or other electronic device) can obtain a laser sub-map in real time and use the currently obtained laser sub-map to be subjected to loop detection as the laser sub-map to be tested. The laser sub-map can be a local map constructed from multiple frames of laser data. The above-mentioned mobile robots may include but are not limited to AGVs (Automated Guided Vehicles), AMRs (Autonomous Mobile Robots), shuttles, and other devices that can automatically locate and navigate based on a grid map.

[0020] In this embodiment, the laser sub-image can take the form of an occupied grid map. The above-mentioned execution subject can perform Euclidean distance transformation on the laser sub-image to be measured to obtain a Euclidean distance map (EDM, Euclidean Distance Mapping). In practice, Euclidean distance transformation is a process of calculating and identifying the distance between the foreground point and the background point, which can realize the transformation of a binary image into a grayscale image (where the grayscale value of each grid is equal to its distance to the nearest background point). Thus, through the Euclidean distance transformation, the distance from each unoccupied grid to the nearest occupied grid in the laser sub-image to be measured can be calculated and identified, thereby converting the laser sub-image to be measured into a Euclidean distance map. The Euclidean distance map can represent the nearest Euclidean distance from each unoccupied grid to the occupied grid. The pixel points in the Euclidean distance map correspond one-to-one to the grids in the laser sub-image to be measured.

[0021] It should be noted that the laser sub-map to be measured can also be a probability grid map. In this case, it can first be converted into an occupancy grid map, and then the Euclidean distance transform is performed on the converted occupancy grid map to obtain a Euclidean distance map. When converting the probability grid map to the occupancy grid map, for each grid, its probability value can be compared with a preset threshold to determine whether it is occupied or unoccupied.

[0022] Step 102: determining feature points in the laser sub-image to be measured based on the Euclidean distance map, and obtaining descriptors of the feature points in the laser sub-image to be measured.

[0023] In this embodiment, the execution entity may first determine feature points in the laser sub-image to be measured based on the Euclidean distance map. Feature points may be points that characterize the characteristics of the laser sub-image to be measured, such as extreme points, saddle points, and the like. Since the Euclidean distance transform is the process of calculating and identifying the distance between foreground points and background points to obtain a grayscale image, the Euclidean distance map can reflect the contour features of the laser sub-image to be measured. Since the pixels in the Euclidean distance map correspond one-to-one with the grids in the laser sub-image to be measured, feature point detection can be performed in the Euclidean distance map to obtain the feature points of the laser sub-image to be measured.

[0024] After detecting feature points, the execution entity can obtain a descriptor for each feature point. A descriptor is information used to describe the feature point. Each feature point can have a corresponding descriptor, which may include information such as the feature point type and Euclidean distance value.

[0025] Step 103 : Based on the descriptors of the feature points in the laser subgraph to be measured and the descriptors of the feature points in the historical laser subgraphs, loop closure detection is performed on the laser subgraph to be measured to obtain a loop closure detection result.

[0026] In this embodiment, the above-mentioned execution subject can obtain the descriptors of the feature points in the historical laser sub-image. The feature points and their descriptors in the historical laser sub-image can be predetermined (specifically, they can be determined after the historical laser sub-image is obtained) and stored in the above-mentioned execution subject (or stored in other devices). The above-mentioned execution subject can read from the local (or other devices). The method for obtaining the feature points in the historical laser sub-image is the same as the method for obtaining the feature points in the laser sub-image to be measured, and the method for obtaining the descriptors of the feature points in the historical laser sub-image is the same as the method for obtaining the descriptors of the feature points in the laser sub-image to be measured, which will not be repeated here.

[0027] After obtaining the descriptors of the feature points in the historical laser sub-image, the execution entity can perform loop detection on the laser sub-image to be tested based on the descriptors of the feature points in the laser sub-image to be tested and the descriptors of the feature points in the historical laser sub-image, to obtain a loop detection result. Specifically, based on the matching of the descriptors of the feature points in the laser sub-image to be tested and the descriptors of the feature points in the historical laser sub-image, the correspondence between the feature points in the laser sub-image to be tested and the feature points in the historical laser sub-image can be determined, and then based on the correspondence, whether the laser sub-image to be tested and the historical laser sub-image correspond to the same scene can be determined. If it is determined that the two correspond to the same scene, the pose transformation matrix of the feature points of the two can be obtained to eliminate the accumulated error.

[0028] The method provided in the embodiments of the present application performs a Euclidean distance transformation on the laser sub-image to be tested to obtain a Euclidean distance map. Based on the Euclidean distance map, the method then determines the feature points in the laser sub-image to be tested and obtains their descriptors. Subsequently, loop detection is performed on the laser sub-image to be tested based on the descriptors of the feature points in the laser sub-image to be tested and the descriptors of the feature points in the historical laser sub-images, thereby obtaining loop detection results. Because the number of feature points in the laser sub-image is far less than the amount of raw data in the laser sub-image, using the descriptors of the feature points in the laser sub-image for scene matching can reduce the time required for loop detection compared to directly matching the laser data, thereby improving the efficiency of loop detection.

[0029] In some optional embodiments, in step 102, the execution entity may determine the characteristic points in the laser sub-image to be measured by obtaining the determinant of the Hessian matrix. For details, see the following sub-steps 1021 to 1023:

[0030] Sub-step 1021 , determining the Hessian matrix determinant (Determinant of Hessian, determinant of the Hessian matrix) of each pixel point in the Euclidean distance map.

[0031] Here, the Hessian Matrix (also known as the Hessian Matrix, Hessian Matrix, Hessian Matrix, etc.) is a blob detection operator in computer vision. The Hessian Matrix is ​​a square matrix composed of the second-order partial derivatives of a multivariate function, which describes the local curvature of the function. For a two-dimensional image, the first-order derivative can characterize the degree of grayscale change of the image, that is, the grayscale gradient. The second-order derivative can represent the degree of change of the grayscale gradient, that is, how fast the grayscale gradient decreases. Using the Hessian Matrix, edge detection and feature point detection of the image can be achieved. For each pixel in the Euclidean distance map, its Hessian Matrix (which can be denoted as H) can be expressed as:

[0032]

[0033] Among them, H xx represents the second-order partial derivative in the x direction at the pixel point, H yy Represents the second-order partial derivative in the y direction at the pixel point, H xy 、H yx The second derivative in the xy direction at the pixel point.

[0034] The Hessian matrix determinant (which can be written as det(H)) can be expressed as:

[0035]

[0036] Sub-step 1022 , based on the Hessian matrix determinant of each pixel point, determine the saddle point, maximum point and minimum point in the Euclidean distance map.

[0037] Here, for each pixel in the Euclidean distance map, if the value of the Hessian matrix determinant of the pixel is greater than 0, then the pixel is an extreme point. Extreme points can include maximum points and minimum points. If the value of the Hessian matrix determinant of the pixel is less than 0, then the pixel is a saddle point.

[0038] After determining the extreme point, the eigenvalue of the Hessian matrix can be determined by calculating the eigenvalue of the matrix to distinguish the type of the determined extreme point (such as whether it is a maximum point or a minimum point). Here, if the eigenvalues ​​of the Hessian matrix corresponding to the extreme point are all positive, it means that the extreme point is a minimum point; if the eigenvalues ​​of the Hessian matrix corresponding to the extreme point are all negative, it means that the extreme point is a maximum point.

[0039] By distinguishing the maximum points, minimum points, and saddle points, the feature points can be classified, so that the feature points of the same category can be matched to improve the accuracy of feature point matching.

[0040] Sub-step 1023 , determining characteristic points in the laser sub-graph to be measured based on the determined saddle points, maximum points, and minimum points.

[0041] Here, the execution subject may select some or all of the points from the determined saddle points, maximum points and minimum points, and may use the corresponding points of the selected points in the laser sub-image to be measured as candidate points and feature points in the laser sub-image to be measured.

[0042] In some examples, the execution entity may perform non-maximum suppression (NMS) on the determined saddle points, maximum points, and minimum points, and determine the feature points in the laser sub-image to be measured based on the results of the non-maximum suppression process. Specifically, the execution entity may determine each of the determined saddle points, maximum points, and minimum points as a candidate point. For each candidate point, all candidate points (including the candidate point) within the neighborhood of the candidate point may be first determined, and then the local non-maximum points in all candidate points within the neighborhood of the candidate point may be filtered out through non-maximum suppression to obtain the remaining candidate points (i.e., local maximum points) within the neighborhood. After performing the above operations on the neighborhood of each candidate point, a number of remaining candidate points may be obtained. The execution entity may determine the corresponding points of each remaining candidate point in the laser sub-image to be measured as feature points in the laser sub-image to be measured. Through the non-maximum suppression process, the feature points can be made into local maximum points, thereby improving the quality of the feature points. At the same time, non-maximum suppression processing can reduce the number of feature points, thereby improving the matching efficiency of subsequent feature points.

[0043] In other examples, the aforementioned execution entity may also obtain the absolute values ​​of the Hessian matrix determinants of the identified saddle points, maxima, and minima, and determine feature points in the laser sub-image to be measured based on these absolute values. Specifically, because larger absolute values ​​represent more distinct features of the representative point, points with absolute values ​​greater than a certain threshold may be selected and their corresponding points in the laser sub-image to be measured may be used as feature points in the laser sub-image to be measured. This can further improve the quality of the feature points and the efficiency of subsequent feature point matching.

[0044] Furthermore, due to occlusion and noise at the edge of the map, the features at the edge of the map are not very stable, and the features at the edge of the map need to be excluded. In view of this, in some optional embodiments, the above-mentioned execution entity can also convert the laser sub-image to be measured into an occupied grid map, erode the unoccupied parts of the occupied grid map, and obtain a mask (MASK) corresponding to the occupied grid map. After determining the feature points in the laser sub-image to be measured, the feature points in the laser sub-image to be measured are filtered based on the mask. In this way, unstable feature points located in the edge area of ​​the map can be filtered out, thereby improving the quality of the feature points. At the same time, the number of feature points can be reduced, thereby improving the matching efficiency of subsequent feature points.

[0045] The image erosion process is similar to image convolution, in that it operates on the image using a template matrix. This template matrix is ​​called a structuring element. Similar to image convolution, the structuring element can arbitrarily specify the center point of the image, and its size and content can be customized as needed. After defining the structuring element, the center point of the structuring element is sequentially placed at each non-zero element in the image. If all image pixels covered by the structuring element are non-zero, the image pixel corresponding to the structuring element center point is retained. Otherwise, the pixel corresponding to the structuring element center point is deleted, completing the image erosion process.

[0046] In some optional embodiments, in step 102, the execution entity may obtain the descriptors of the feature points in the laser sub-image to be measured through the following sub-steps 1024 to 1026:

[0047] Sub-step 1024 , determining the binary description information and type information of each feature point in the laser sub-image to be measured.

[0048] Here, binary description information can be extracted using the BRIEF (Binary Robust Independent Elementary Features) algorithm. BRIEF is an algorithm that describes detected feature points. The resulting description information is binary, also known as a BRIEF descriptor. The type information of the feature point can indicate the category of the feature point, which can include maximum points, minimum points, and saddle points.

[0049] Sub-step 1025 , based on the gradient information of the Euclidean distance map, obtain the main gradient direction of each feature point and the average Euclidean distance value in the neighborhood.

[0050] The main direction of the gradient may refer to the direction in which the gradient changes the most. In some examples, for each feature point in the laser sub-image to be measured, the above-mentioned execution entity may determine the gradient amplitude of the corresponding point of the feature point in the Euclidean distance map in multiple gradient directions. Then, based on the determined gradient amplitude, the main gradient direction of the feature point is selected from multiple gradient directions. As an example, the gradient direction can be divided into 36 parts, that is, 0 to 360 degrees are divided into 36 parts, each corresponding to an angle range of 10 degrees. For each feature point in the laser sub-image to be measured, a histogram can be used to count the gradient amplitudes in the 36 gradient directions in its neighborhood, so that the gradient direction with the largest gradient amplitude is taken as the main gradient direction of the feature point.

[0051] The average Euclidean distance value within the neighborhood may refer to the average of the Euclidean distance values ​​within the neighborhood. In some examples, for each feature point in the laser sub-image to be measured, the execution entity may determine the average pixel value within the neighborhood of the corresponding point of the feature point in the Euclidean distance map (i.e., the average of the Euclidean distance values ​​of each pixel in the neighborhood), and use this average pixel value as the average Euclidean distance value within the neighborhood of the feature point.

[0052] Sub-step 1026 , for each feature point in the laser sub-image to be measured, the binary description information, type information, main gradient direction, and average Euclidean distance value in the neighborhood of the feature point are summarized to obtain a descriptor of the feature point.

[0053] By determining the binary description information for each feature point, the calculation speed of the distance between descriptors can be improved and the amount of data stored can be reduced, thus saving storage and computing resources. Because the descriptor of a feature point contains the binary description information of the feature point, type information, main gradient direction, and average Euclidean distance value within the neighborhood, the descriptor of the feature point is more content-rich. By matching feature points with multi-dimensional descriptor content, the robustness of the loop detection algorithm can be improved.

[0054] Further references Figure 2, which shows a process 200 of another embodiment of a loop detection method. The process 200 of the loop detection method includes the following steps 201 to 205:

[0055] Step 201: Perform Euclidean distance transformation on the laser sub-graph to be measured to obtain a Euclidean distance graph.

[0056] For step 201 in this embodiment, reference may be made to the description in the above embodiment, which will not be repeated here.

[0057] Step 202 : determining feature points in the laser sub-image to be measured based on the Euclidean distance map, and obtaining descriptors of the feature points in the laser sub-image to be measured.

[0058] For step 201 in this embodiment, reference may be made to the description in the above embodiment, which will not be repeated here.

[0059] Step 203 : determining the correspondence between the feature points in the laser submap to be measured and the feature points in the historical laser submap based on the descriptors of the feature points in the laser submap to be measured and the descriptors of the feature points in the historical laser submap.

[0060] In this embodiment, the execution subject may pre-store the descriptors of the feature points in the historical laser sub-image. In practice, each time a laser sub-image is collected, the feature points therein may be detected and the descriptors of the feature points may be extracted using the method described in the above embodiment, and the laser sub-image may be used as a historical laser sub-image, and its feature points and descriptors may be stored. The execution subject may store the laser sub-image to be measured (which may be denoted as P i , P i ={p1,p2,p3,…,p m}) in the feature point (i.e., P i p1, p2, p3, ..., p m ) descriptor and the historical laser subgraph (which can be denoted as P j , P j ={p1,p2,p3,…,p n}) in the feature point (i.e., P j p1, p2, p3, ..., p n ) descriptors to determine the correspondence between the feature points in the laser sub-image to be measured and the feature points in the historical laser sub-image based on the matching results.

[0061] As an example, a descriptor may include binary description information, type information, and the average Euclidean distance value within the neighborhood of a feature point. For each feature point in the laser submap to be measured, the execution entity may first select, from the historical laser submap, a feature point with the same type information as the feature point and a difference in the average Euclidean distance value within the neighborhood less than a target value as a candidate feature point; then, the distance between the binary description information of the feature point and each selected candidate feature point may be determined; finally, the candidate feature point with the smallest distance from the binary description information of the feature point may be determined, and a correspondence between the feature point and the determined candidate feature point may be established. The above operations may be performed for each feature point in the laser submap to be measured.

[0062] Step 204 : determining the initial pose transformation matrix of the feature points in the laser sub-image to be measured and the feature points in the historical laser sub-image based on the corresponding relationship.

[0063] In this embodiment, the execution entity may perform a geometric consistency check on the correspondence determined in step 203 to determine a pose transformation matrix between the feature points in the laser sub-image to be tested and the feature points in the historical laser sub-image, and use this matrix as the initial pose transformation matrix. The pose transformation may include rotation and translation, and the pose transformation matrix may represent the rotational and translational relationships between the feature points in the two laser sub-images. Based on the position transformation matrix, the accumulated pose errors of the two laser sub-images can be eliminated.

[0064] As an example, the RANSAC (Random Sample Consensus) algorithm can be used to determine the initial pose transformation matrix. Specifically, the following steps can be repeated: First, multiple pairs (e.g., three pairs) of feature point pairs with corresponding relationships are randomly selected, and candidate pose transformation matrices are determined based on the selected feature point pairs. For example, the ICP (Iterative Closest Point) algorithm can be used to determine the candidate pose transformation matrix. Then, based on each pair of corresponding feature point pairs, the score of the candidate pose transformation matrix is ​​determined. For example, each feature point in the laser sub-image to be measured can be pose-transformed according to the candidate pose transformation matrix to obtain the pose-transformed points. The residual sum between each feature point in the laser sub-image to be measured and the pose-transformed points is then calculated, and the residual sum is used as the score of the candidate pose transformation matrix. Repeating the above steps multiple times can obtain multiple candidate pose transformation matrices and their scores. When the target number of repetitions is reached, the execution entity can use the candidate pose transformation matrix with the highest score as the initial pose transformation matrix. The target number of times can be preset as needed, and its specific value is not limited here.

[0065] It should be noted that if the initial pose transformation matrix cannot be obtained, it can be considered that the scene corresponding to the laser sub-image to be tested is consistent with the scene corresponding to the historical laser sub-image. At this time, the laser sub-image to be tested and other historical laser sub-images can be looped detected in the same or similar way, which will not be repeated here.

[0066] Step 205 : Based on the initial pose transformation matrix, verify whether the scene corresponding to the laser sub-image to be tested is consistent with the scene corresponding to the historical laser sub-image.

[0067] In this embodiment, the execution entity can verify whether the scene corresponding to the laser sub-image to be tested is consistent with the scene corresponding to the historical laser sub-image based on the initial pose transformation matrix. For example, if the score of the initial pose transformation matrix is ​​greater than a certain threshold, the scene corresponding to the laser sub-image to be tested can be considered to be consistent with the scene corresponding to the historical laser sub-image, and the initial pose transformation matrix is ​​stored. If the score of the initial pose transformation matrix is ​​less than or equal to the threshold, the scene corresponding to the laser sub-image to be tested can be considered to be inconsistent with the scene corresponding to the historical laser sub-image. In this case, the laser sub-image to be tested and other historical laser sub-images can be looped back in the same or similar manner, which will not be repeated here.

[0068] from Figure 2 It can be seen that Figure 1 Compared to the corresponding embodiment, process 200 of the loop closure detection method in this embodiment includes a step of performing a consistency check on the scene corresponding to the laser sub-image to be tested and the scene corresponding to the historical laser sub-image based on the initial transformation matrix. This consistency check can reduce scene mismatches, thereby improving the accuracy of loop closure detection results.

[0069] In some optional embodiments, the descriptor may further include the main gradient direction of the feature point. Before determining the initial pose transformation matrix of the feature points in the laser sub-image to be measured and the feature points in the historical laser sub-image, the above-mentioned execution subject may also verify the correspondence of the feature points based on the main gradient direction. Specifically, for each pair of feature point pairs with a corresponding relationship, the above-mentioned execution subject may detect the difference in the main gradient direction of the two feature points in the feature point pair. Then, the proportion of feature point pairs whose difference is within the target range may be determined by statistics. Finally, the correspondence of feature point pairs whose proportion is less than the target value (such as 80%) may be deleted, and the remaining correspondences are valid correspondences. In this way, some mismatches (i.e., wrong correspondences) may be kicked out, thereby further improving the accuracy of loop detection.

[0070] In some optional embodiments, if it is verified based on the initial pose transformation matrix that the scene corresponding to the laser sub-image to be tested is consistent with the scene corresponding to the historical laser sub-image, the above-mentioned execution body may further perform the following sub-steps 2051 to 2052 to finely optimize the initial pose transformation matrix:

[0071] Sub-step 2051 , based on the initial pose transformation matrix, updates the correspondence between the feature points in the laser sub-image to be measured (which may be denoted as i) and the feature points in the historical laser sub-image (which may be denoted as j).

[0072] Here, the projection position of each feature point in the laser sub-image to be measured in the historical laser sub-image can be determined based on the initial pose transformation matrix, and feature points can be searched near the projection position, so as to select a feature point from the searched feature points and establish a corresponding relationship with the projected feature point in the laser sub-image to be measured.

[0073] As an example, for each feature point a in i, the position of the feature point can first be transformed based on the initial posture transformation matrix, and it can be projected into j to obtain the corresponding point a' in j. Then, the target area with the corresponding point a' as the center in the historical laser sub-map is determined, and the radius of the target area can be set in advance. Afterwards, the target feature points in the target area can be selected based on the descriptor of the feature point a and the descriptor of the feature points in the target area, so as to establish a correspondence between the feature point a and the target feature point. Among them, the method of selecting the target feature point based on the descriptor and establishing the correspondence is basically the same as the method of determining the correspondence between the feature points based on the descriptor in the above embodiment, and will not be repeated here. After performing the above operations on each feature point in i, the correspondence between the feature points in the laser sub-map to be measured and the feature points in the historical laser sub-map can be updated.

[0074] Sub-step 2052, based on the updated correspondence, updates the initial pose transformation matrix to obtain the target pose transformation matrix, and based on the target pose transformation matrix, re-checks whether the scene corresponding to the laser sub-image to be measured is consistent with the scene corresponding to the historical laser sub-image.

[0075] Here, the operation of obtaining the target pose transformation matrix based on the updated correspondence is essentially the same as the operation of determining the initial pose transformation matrix based on the correspondence in the above embodiment, and will not be repeated here. In addition, the operation of verifying whether the scene is consistent based on the target pose transformation matrix is ​​essentially the same as the operation of verifying whether the scene is consistent based on the initial pose transformation matrix in the above embodiment, and will not be repeated here.

[0076] Here, if the re-calibration result indicates that the scene corresponding to the laser sub-image to be measured is consistent with the scene corresponding to the historical laser sub-image, the above-mentioned execution entity can replace the stored initial pose transformation matrix with the target position transformation matrix.

[0077] By updating the correspondences between feature points and updating the pose transformation matrix based on the updated correspondences, the pose transformation matrix can be finely optimized, resulting in a more accurate pose transformation matrix. Furthermore, re-conducting a consistency check based on the updated target pose transformation matrix further improves the robustness of the loop detection algorithm.

[0078] Further references Figure 3 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a loop detection device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0079] like Figure 3 As shown, the loop detection device 300 of this embodiment includes: a transformation unit 301, configured to perform Euclidean distance transformation on the laser sub-image to be measured to obtain a Euclidean distance map; a determination unit 302, configured to determine the feature points in the laser sub-image to be measured based on the Euclidean distance map, and obtain a descriptor of the feature points in the laser sub-image to be measured; a detection unit 303, configured to perform loop detection on the laser sub-image to be measured based on the descriptors of the feature points in the laser sub-image to be measured and the descriptors of the feature points in the historical laser sub-images to obtain a loop detection result.

[0080] In some optional implementations of this embodiment, the above-mentioned device also includes an etching processing unit, which is configured to: convert the above-mentioned laser sub-image to be measured into an occupied grid map, convert the above-mentioned laser sub-image to be measured into an occupied grid map, and perform etching processing on the unoccupied part of the above-mentioned occupied grid map to obtain a mask corresponding to the above-mentioned occupied grid map; and a filtering unit, which is configured to filter the feature points in the above-mentioned laser sub-image to be measured based on the above-mentioned mask.

[0081] In some optional implementations of this embodiment, the above-mentioned determination unit 302 is further configured to: determine the Hessian matrix determinant of each pixel point in the above-mentioned Euclidean distance map; determine the saddle point, maximum point and minimum point in the above-mentioned Euclidean distance map based on the Hessian matrix determinant of each pixel point; and determine the characteristic points in the above-mentioned laser sub-map to be measured based on the determined saddle point, maximum point and minimum point.

[0082] In some optional implementations of this embodiment, the above-mentioned determination unit 302 is further configured to: perform non-maximum suppression processing on the determined saddle points, maximum points and minimum points, and determine the characteristic points in the above-mentioned laser sub-image to be measured based on the processing results of the above-mentioned non-maximum suppression processing; and / or obtain the absolute value of the value of the Hessian matrix determinant of the determined saddle points, maximum points and minimum points, and determine the characteristic points in the above-mentioned laser sub-image to be measured based on the obtained absolute value.

[0083] In some optional implementations of this embodiment, the above-mentioned determination unit 302 is further configured to: determine the binary description information and type information of each feature point in the above-mentioned laser sub-image to be measured; based on the gradient information of the above-mentioned Euclidean distance map, obtain the main gradient direction and the average Euclidean distance value in the neighborhood of each feature point; for each feature point in the above-mentioned laser sub-image to be measured, generate a descriptor including the binary description information, type information, main gradient direction and average Euclidean distance value in the neighborhood of the feature point.

[0084] In some optional implementations of this embodiment, the above-mentioned determination unit 302 is further configured to: for each feature point in the above-mentioned laser sub-image to be measured, determine the gradient amplitude of the corresponding point of the feature point in the above-mentioned Euclidean distance map in multiple gradient directions; based on the determined gradient amplitude, select the main gradient direction of the feature point from the above-mentioned multiple gradient directions.

[0085] In some optional implementations of this embodiment, the above-mentioned determination unit 302 is further configured to: for each feature point in the above-mentioned laser sub-image to be measured, determine the average pixel value of the feature point in the neighborhood of the corresponding point in the above-mentioned Euclidean distance map, and use the above-mentioned average pixel value as the average Euclidean distance value in the neighborhood of the feature point.

[0086] In some optional implementations of this embodiment, the detection unit 303 is further configured to: determine the correspondence between the feature points in the laser subimage to be measured and the feature points in the historical laser subimage based on the descriptors of the feature points in the laser subimage to be measured and the descriptors of the feature points in the historical laser subimage; determine the initial pose transformation matrix between the feature points in the laser subimage to be measured and the feature points in the historical laser subimage based on the correspondence; and verify whether the scene corresponding to the laser subimage to be measured is consistent with the scene corresponding to the historical laser subimage based on the initial pose transformation matrix.

[0087] In some optional implementations of this embodiment, the above-mentioned descriptor includes binary description information, type information and average Euclidean distance value in the neighborhood of the feature point; and the above-mentioned detection unit 303 is further configured to: for each feature point in the above-mentioned laser sub-image to be measured, perform the following steps: select a candidate feature point from the above-mentioned historical laser sub-image, which has the same type information as the feature point and the difference between the average Euclidean distance value in the neighborhood is less than the target value; determine the distance between the feature point and the binary description information of each selected candidate feature point; establish a correspondence between the feature point and the candidate feature point corresponding to the minimum distance value.

[0088] In some optional implementations of this embodiment, the above-mentioned detection unit 303 is further configured to: repeatedly perform the following steps: randomly select multiple pairs of feature point pairs with corresponding relationships, and determine candidate pose transformation matrices based on the selected feature point pairs; determine the scores of the above-mentioned candidate pose transformation matrices based on each pair of feature point pairs with corresponding relationships; when the number of repeated executions reaches the target number of times, use the candidate pose transformation matrix with the highest score as the initial pose transformation matrix.

[0089] In some optional implementations of this embodiment, the above-mentioned descriptor also includes the main gradient direction of the feature point; and the above-mentioned detection unit 303 is further configured to: for each pair of feature points having a corresponding relationship, detect the difference in the main gradient directions of the two feature points in the feature point pair; determine the proportion of feature point pairs whose above-mentioned difference is within the target range; and delete the correspondence of feature point pairs whose above-mentioned proportion is less than the target value.

[0090] In some optional implementations of this embodiment, the above-mentioned device also includes: a re-verification unit, configured to: update the correspondence between the feature points in the above-mentioned laser sub-image to be measured and the feature points in the above-mentioned historical laser sub-image based on the above-mentioned initial pose transformation matrix; based on the updated correspondence, update the above-mentioned initial pose transformation matrix to obtain the target pose transformation matrix, and re-verify whether the scene corresponding to the above-mentioned laser sub-image to be measured is consistent with the scene corresponding to the above-mentioned historical laser sub-image based on the above-mentioned target pose transformation matrix.

[0091] In some optional implementations of this embodiment, the above-mentioned re-calibration unit is further configured to: for each feature point in the above-mentioned laser sub-image to be measured, perform the following steps: determine the corresponding point of the feature point in the above-mentioned historical laser sub-image based on the above-mentioned initial pose transformation matrix; determine the target area in the above-mentioned historical laser sub-image with the above-mentioned corresponding point as the center; based on the descriptor of the feature point and the descriptor of the feature point in the above-mentioned target area, select the target feature point in the above-mentioned target area; establish a correspondence between the feature point and the above-mentioned target feature point.

[0092] The apparatus provided by the above-described embodiments of the present application performs a Euclidean distance transformation on the laser sub-image to be measured to obtain a Euclidean distance map. Based on the Euclidean distance map, the apparatus then determines the feature points in the laser sub-image to be measured and obtains their descriptors. Subsequently, loop detection is performed on the laser sub-image to be measured based on the descriptors of the feature points in the laser sub-image to be measured and the descriptors of the feature points in the historical laser sub-images, thereby obtaining loop detection results. Because the number of feature points in the laser sub-image is much smaller than the amount of raw data in the laser sub-image, scene matching using the descriptors of the feature points in the laser sub-image can reduce the time required for loop detection compared to directly matching laser data, thereby improving loop detection efficiency.

[0093] Reference below Figure 4 , which shows a structural schematic diagram of an electronic device for implementing some embodiments of the present application. Figure 4 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0094] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. Various programs and data required for the operation of the electronic device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0095] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic disk, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0096] In particular, according to some embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present application are performed.

[0097] It should be noted that the computer-readable medium described in some embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present application, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present application, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0098] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0099] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: performs a Euclidean distance transformation on the laser sub-image to be measured to obtain a Euclidean distance map; determines the feature points in the laser sub-image to be measured based on the Euclidean distance map, and obtains the descriptors of the feature points in the laser sub-image to be measured; performs loop detection on the laser sub-image to be measured based on the descriptors of the feature points in the laser sub-image to be measured and the descriptors of the feature points in the historical laser sub-image, and obtains loop detection results. This embodiment improves the efficiency of loop detection.

[0100] Computer program code for performing the operations of some embodiments of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++; and also conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, or can be connected to an external computer (for example, through the Internet using an Internet service provider). The above network includes a local area network (LAN) or a wide area network (WAN).

[0101] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0102] The units described in some embodiments of this application may be implemented in software or hardware. The units described may also be provided in a processor. For example, a processor may be described as comprising a first determination unit, a second determination unit, a selection unit, and a third determination unit. The names of these units do not, in some cases, limit the units themselves.

[0103] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0104] The above description is only an illustration of some preferred embodiments of the present application and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features and the technical features with similar functions disclosed in the embodiments of the present application (but not limited to) are replaced with each other to form a technical solution.

Claims

1. A loop detection method, characterized in that: The method comprises: Perform Euclidean distance transformation on the laser sub-graph to be measured to obtain a Euclidean distance graph; Determine, based on the Euclidean distance map, characteristic points in the laser sub-map to be measured, and obtain descriptors of the characteristic points in the laser sub-map to be measured, wherein the descriptors are information for describing the characteristic points in the laser sub-map to be measured; performing loop detection on the laser subgraph to be measured based on descriptors of feature points in the laser subgraph to be measured and descriptors of feature points in historical laser subgraphs to obtain a loop detection result, wherein the loop detection is performed based on a correspondence between the descriptors of feature points in the laser subgraph to be measured and the descriptors of feature points in historical laser subgraphs; The determining of the characteristic points in the laser sub-graph to be measured based on the Euclidean distance graph includes: Determining the Hessian matrix determinant of each pixel in the Euclidean distance map; Determining saddle points, maximum points, and minimum points in the Euclidean distance graph based on the Hessian matrix determinant of each pixel point; Based on the determined saddle points, maximum points and minimum points, characteristic points in the laser sub-image to be measured are determined.

2. The method according to claim 1, characterized in that The method further comprises: Converting the laser sub-image to be measured into an occupied grid map, performing etching processing on unoccupied parts of the occupied grid map to obtain a mask corresponding to the occupied grid map; and After determining the characteristic points in the laser sub-image to be measured, the method further includes: Based on the mask, the feature points in the laser sub-image to be measured are filtered.

3. The method according to claim 1, characterized in that The determining of the characteristic points in the laser sub-graph to be measured based on the determined saddle points, maximum points, and minimum points comprises: performing non-maximum suppression processing on the determined saddle points, maximum points, and minimum points, and determining characteristic points in the laser sub-graph to be measured based on the processing results of the non-maximum suppression processing; and / or The absolute values ​​of the Hessian matrix determinants of the determined saddle points, maximum points, and minimum points are obtained, and based on the obtained absolute values, characteristic points in the laser sub-image to be measured are determined.

4. The method according to claim 3, characterized in that The obtaining of the descriptor of the feature point in the laser sub-image to be measured comprises: Determine binary description information and type information of each feature point in the laser sub-image to be measured; Based on the gradient information of the Euclidean distance map, obtaining the main gradient direction and the average Euclidean distance value in the neighborhood of each feature point; For each feature point in the laser sub-image to be measured, a descriptor including binary description information, type information, main gradient direction and average Euclidean distance value in the neighborhood of the feature point is generated.

5. The method according to claim 4, characterized in that The obtaining of the main gradient direction of each feature point based on the gradient information of the Euclidean distance map includes: For each feature point in the laser sub-image to be measured, the gradient amplitude of the corresponding point of the feature point in the Euclidean distance map in multiple gradient directions is determined; and based on the determined gradient amplitude, the main gradient direction of the feature point is selected from the multiple gradient directions.

6. The method according to claim 4 or 5, characterized in that Obtaining an average Euclidean distance value within a neighborhood of each feature point based on the gradient information of the Euclidean distance map includes: For each feature point in the laser sub-image to be measured, the average pixel value in the neighborhood of the corresponding point of the feature point in the Euclidean distance map is determined, and the average pixel value is used as the average Euclidean distance value in the neighborhood of the feature point.

7. The method according to claim 1, characterized in that The loop detection is performed on the laser subgraph to be measured based on the descriptors of the feature points in the laser subgraph to be measured and the descriptors of the feature points in the historical laser subgraphs to obtain the loop detection result, including: Determine a correspondence between the feature points in the laser subgraph to be measured and the feature points in the historical laser subgraph based on the descriptors of the feature points in the laser subgraph to be measured and the descriptors of the feature points in the historical laser subgraph; Based on the corresponding relationship, determining an initial pose transformation matrix between the feature points in the laser sub-image to be measured and the feature points in the historical laser sub-image; Based on the initial pose transformation matrix, it is verified whether the scene corresponding to the laser sub-image to be measured is consistent with the scene corresponding to the historical laser sub-image.

8. The method according to claim 7, characterized in that The descriptor includes binary description information, type information and average Euclidean distance value within the neighborhood of the feature point; and The determining of the correspondence between the feature points in the laser subgraph to be measured and the feature points in the historical laser subgraph based on the descriptors of the feature points in the laser subgraph to be measured and the descriptors of the feature points in the historical laser subgraph includes: For each feature point in the laser sub-image to be measured, perform the following steps: Selecting a candidate feature point from the historical laser subgraph that has the same type information as the feature point and a difference in the average Euclidean distance value within the neighborhood that is less than a target value; Determine the distance between the feature point and the binary description information of each selected candidate feature point; A correspondence between the feature point and the candidate feature point corresponding to the minimum distance value is established.

9. The method according to claim 7, characterized in that Determining the initial pose transformation matrix of the feature points in the laser sub-image to be measured and the feature points in the historical laser sub-image based on the corresponding relationship includes: Repeat the following steps: randomly selecting multiple pairs of feature point pairs with corresponding relationships, and determining candidate pose transformation matrices based on the selected feature point pairs; and determining scores of the candidate pose transformation matrices based on each pair of feature point pairs with corresponding relationships; When the number of repeated executions reaches the target number, the candidate pose transformation matrix with the highest score is used as the initial pose transformation matrix.

10. The method according to claim 7, characterized in that The descriptor also includes the main gradient direction of the feature point; and Before determining the initial pose transformation matrix of the feature points in the laser subgraph to be measured and the feature points in the historical laser subgraph, performing loop detection on the laser subgraph to be measured based on the descriptors of the feature points in the laser subgraph to be measured and the descriptors of the feature points in the historical laser subgraph to obtain a loop detection result, the method further includes: For each pair of feature points having a corresponding relationship, the difference in the main gradient direction of the two feature points in the feature point pair is detected; Determine the proportion of feature point pairs whose difference values ​​are within the target range; Delete the corresponding relationship of the feature point pair whose proportion is less than the target value.

11. The method according to claim 9, characterized in that The method further comprises: Based on the initial pose transformation matrix, updating the correspondence between the feature points in the laser sub-image to be measured and the feature points in the historical laser sub-image; Based on the updated correspondence, the initial pose transformation matrix is ​​updated to obtain the target pose transformation matrix, and based on the target pose transformation matrix, the scene corresponding to the laser sub-image to be measured is re-checked to see whether it is consistent with the scene corresponding to the historical laser sub-image.

12. The method according to claim 11, characterized in that The updating of the correspondence between the feature points in the laser sub-image to be measured and the feature points in the historical laser sub-image based on the initial pose transformation matrix includes: For each feature point in the laser sub-image to be measured, perform the following steps: Determining a corresponding point of the feature point in the historical laser sub-image based on the initial pose transformation matrix; Determine a target area in the historical laser sub-graph with the corresponding point as the center; Selecting a target feature point within the target area based on a descriptor of the feature point and a descriptor of the feature point within the target area; A correspondence between the feature point and the target feature point is established.

13. A loop detection device, characterized in that: The device comprises: A transformation unit is configured to perform Euclidean distance transformation on the laser sub-image to be measured to obtain a Euclidean distance image; a determining unit configured to determine, based on the Euclidean distance map, feature points in the laser sub-map to be measured, and obtain descriptors of the feature points in the laser sub-map to be measured, wherein the descriptors are information for describing the feature points in the laser sub-map to be measured; a detection unit configured to perform loop closure detection on the laser subgraph to be measured based on descriptors of feature points in the laser subgraph to be measured and descriptors of feature points in historical laser subgraphs, to obtain a loop closure detection result, wherein the loop closure detection is performed based on a correspondence between the descriptors of feature points in the laser subgraph to be measured and the descriptors of feature points in historical laser subgraphs; The determination unit is further configured to: determine the Hessian matrix determinant of each pixel point in the above-mentioned Euclidean distance map; determine the saddle point, maximum point and minimum point in the above-mentioned Euclidean distance map based on the Hessian matrix determinant of each pixel point; and determine the characteristic points in the above-mentioned laser sub-map to be measured based on the determined saddle point, maximum point and minimum point.

14. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 12.

15. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images