Loopback positioning detection method and system and storage medium

Abstract

The invention discloses a loopback positioning detection method and system and a storage medium, and the method comprises the steps: carrying out the region division of a left eye image based on a preset proportion when a loopback detection condition is judged to be satisfied, and calculating a first feature point in each divided region image; determining a first matching point of the first feature point on the loopback frame image by using a descriptor matching mode; calculating a second feature point matched with the first matching point in the right eye image by using a similarity matching algorithm; calculating a first 3D coordinate corresponding to the first feature point in a world coordinate system based on the pixel coordinate of the first feature point and the pixel coordinate of the second feature point; and correcting a first position when the mobile device acquires the positioning image based on the first 3D coordinate. According to the technical scheme, the problem that loopback positioning detection is unavailable under the condition that the feature points are distributed unevenly is solved, and the precision of VSLAM loopback positioning detection is improved.

Description

Technical Field

[0001] The present application relates to the technical field of loop detection and positioning, and in particular, to a loop positioning detection method, a loop positioning detection system and a computer storage medium. Background Art

[0002] Mobile robots can use visual odometers to implement a vision-based real-time positioning and mapping system - VSLAM, but visual odometers usually only consider key frames in adjacent time periods, and the displacement errors generated during the movement of the mobile robot during this period will gradually accumulate, forming cumulative errors, resulting in unreliable positioning results of the visual odometer. Therefore, it is necessary to use loop detection and positioning technology to discover potential loops and correct the current positioning position of the mobile robot to reduce global errors.

[0003] In the process of loop detection and positioning, first, for the current frame image that needs to be judged, the descriptor is extracted according to its feature points that are tracked at the front end, and the bag-of-words model is used to find out whether there are similar scenes in the previous historical images. Then, more feature points are extracted again, the descriptors are calculated, and inserted into the bag-of-words model so that they can be searched by subsequent image frames; secondly, if a potential loop frame image is found through the bag-of-words, the descriptor matching is performed; then, when the number of feature points matched by the descriptor reaches a certain threshold, the PNP algorithm is used to calculate the inliers and the current position of the loop frame; finally, if the number of inliers reaches a certain threshold and the positions of the loop frame and the current frame are not far apart, the cumulative error is corrected.

[0004] In the prior art, after finding the loop frame and calculating the valid inner points (valid feature points), the number of feature points is already very small. Based on these few feature points, the PNP algorithm will calculate a large loop frame position deviation, which will lead to inaccurate calculation of the cumulative error. Then, the current frame position corrected based on this will also be inaccurate. Especially in some scenes with few feature points in the near distance and many feature points in the far distance, the error will be even greater, making the loop detection positioning unusable. Summary of the invention

[0005] The purpose of this application is to improve the accuracy of VSLAM loop positioning detection.

[0006] The technical solution of the first aspect of the present application is: a loop closure positioning detection method is provided, the method comprising: when it is determined that the loop closure detection condition is met, dividing the left eye image into regions based on a preset ratio, and respectively calculating the first feature points in each divided regional image; determining the first matching point of the first feature point on the loop closure frame image by using a descriptor matching method; calculating the second feature point matching the first matching point in the right eye image by using a similarity matching algorithm; calculating the first 3D coordinates corresponding to the first feature point in the world coordinate system based on the pixel coordinates of the first feature point and the pixel coordinates of the second feature point; and correcting the first position of the mobile device when acquiring the positioning image based on the first 3D coordinates, wherein the positioning image includes the left eye image and the right eye image.

[0007] In any of the above technical solutions, the method further includes: obtaining feature point pairs in the left-eye image and the right-eye image, and calculating the second 3D coordinates of the feature point pairs; when the left-eye image is determined to be the current key frame image, searching the historical key frame image with the highest similarity to the current key frame image in the historical key frame image set, and recording it as the loop frame image; using the descriptor matching method, determining the second matching point matching the feature point pair in the loop frame image; based on the second 3D coordinate corresponding to the second matching point, calculating the second position of the mobile device when acquiring the left-eye image; when the number of the counted second matching points is greater than the preset number, and the position deviation between the second position and the fourth position is less than the preset distance, it is determined that the loop detection condition is met, wherein the fourth position is the current position of the mobile device corresponding to the loop frame image calculated based on the first 3D coordinate.

[0008] In any of the above technical solutions, further, based on the first 3D coordinate, the first position of the mobile device when acquiring the positioning image is corrected, specifically including: determining whether the distance deviation between the fourth position and the third position is less than or equal to the cumulative error threshold, if not, discarding the positioning image, and if so, based on the distance deviation, correcting the first position of the mobile device when acquiring the positioning image, wherein the third position is the historical position of the mobile device when acquiring the loop frame image.

[0009] In any of the above technical solutions, further, a similarity matching algorithm is used to calculate a second feature point matching the first matching point in the right-eye image, specifically including: respectively calculating image integral images of pixel points in the left-eye image and the right-eye image, and generating an NCC operator query table; constructing a similarity matching calculation formula based on the NCC calculation principle and the preset disparity traversal interval; for any first matching point, based on the similarity matching calculation formula, in the right-eye image, by querying the operator in the NCC operator query table, the similarity corresponding to any preset disparity within a preset window is calculated, wherein the preset disparity belongs to the preset disparity traversal interval; selecting the preset disparity corresponding to the maximum similarity value, and based on the selected preset disparity, determining the feature point of the pixel point corresponding to the first matching point in the right-eye image, recorded as the second feature point.

[0010] In any of the above technical solutions, further, the similarity matching calculation formula is:

[0011]

[0012] Where d is the preset disparity, (i, j) is the pixel coordinate of the first matching point, Ncc(d, (i, j)) is the similarity corresponding to the first matching point with pixel coordinates (i, j) when the preset disparity is d, and I LR (i,j),L′ L , L′ R is the intermediate parameter, I L (i, j+d) is the pixel value of the pixel point in the left image after the pixel coordinate (i, j) is horizontally offset by the preset disparity d. R (i, j) is the pixel value of the pixel with coordinates (i, j) in the right image, m*n is the size of the preset window, They are the pixel means in the corresponding preset windows of the pixel with the pixel coordinates (i, j) in the left image and the right image respectively.

[0013] The technical solution of the second aspect of the present application is: a loop positioning detection system is provided, the system includes: an area division unit, the area division unit is configured to divide the left eye image into areas based on a preset ratio when it is determined that the loop detection condition is met, and respectively calculate the first feature points in each divided area image; a matching unit, the matching unit is configured to determine the first matching point of the first feature point on the loop frame image by using a descriptor matching method; the matching unit is also configured to: use a similarity matching algorithm to calculate the second feature point matching the first matching point in the right eye image; a coordinate calculation unit, the coordinate calculation unit is configured to calculate the first 3D coordinates corresponding to the first feature point in the world coordinate system based on the pixel coordinates of the first feature point and the pixel coordinates of the second feature point; a correction unit, the correction unit is configured to correct the first position of the mobile device when acquiring the positioning image based on the first 3D coordinates, wherein the positioning image includes the left eye image and the right eye image.

[0014] In any of the above technical solutions, further, the coordinate calculation unit is also configured to: obtain feature point pairs in the left-eye image and the right-eye image, and calculate the second 3D coordinates of the feature point pairs; the system also includes: an image search unit, the image search unit is configured to, when determining that the left-eye image is the current key frame image, search for the historical key frame image with the highest similarity to the current key frame image in the historical key frame image set, and record it as the loop frame image; the matching unit is also configured to: determine the second matching point matching the feature point pair in the loop frame image by using the descriptor matching method; the system also includes: a position calculation unit, the position calculation unit is configured to calculate the second position of the mobile device when acquiring the left-eye image based on the second 3D coordinate corresponding to the second matching point; a judgment unit, the judgment unit is configured to determine that the loop detection condition is met when the number of the counted second matching points is greater than the preset number and the position deviation between the second position and the fourth position is less than the preset distance, wherein the fourth position is the current position of the mobile device corresponding to the loop frame image calculated based on the first 3D coordinate.

[0015] In any of the above technical solutions, further, the correction unit is specifically configured to: determine whether the distance deviation between the fourth position and the third position is less than or equal to the cumulative error threshold; if not, discard the positioning image; if so, based on the distance deviation, correct the first position when the mobile device acquires the positioning image, wherein the third position is the historical position when the mobile device acquires the loop frame image.

[0016] In any of the above technical solutions, further, the matching unit is specifically configured to: calculate the image integral images of pixel points in the left-eye image and the right-eye image respectively, and generate an NCC operator query table; construct a similarity matching calculation formula based on the NCC calculation principle and the preset disparity traversal interval; for any first matching point, based on the similarity matching calculation formula, in the right-eye image, by querying the operator in the NCC operator query table, calculate the similarity corresponding to any preset disparity within the preset window, wherein the preset disparity belongs to the preset disparity traversal interval; select the preset disparity corresponding to the maximum similarity value, and based on the selected preset disparity, determine the feature point of the pixel point corresponding to the first matching point in the right-eye image, and record it as the second feature point.

[0017] The technical solution of the third aspect of the present application is: a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, a method for loop positioning detection as described in any one of the technical solutions of the first aspect of the present application is implemented.

[0018] The beneficial effects of this application are:

[0019] The technical solution in the present application divides the image into regions and re-extracts feature points before performing loop closure detection, and then matches the feature points between the left and right eye images and the loop closure frame image to obtain the position of the mobile device with higher accuracy. This helps to solve the problem of unavailable loop closure positioning detection when the feature points are unevenly distributed, and improves the accuracy of VSLAM loop closure positioning detection.

[0020] This application also proposes a method of using an integral graph to generate an NCC operator query table when calculating the similarity of pixel points in the left and right eye images, so as to quickly obtain the values ​​of relevant operators in the similarity matching calculation formula to reduce the amount of calculation and reduce the demand for hardware computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The advantages of the above and / or additional aspects of the present application will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:

[0022] Figure 1 is a schematic flow chart of a loop positioning detection method according to an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of a real bow-shaped trajectory according to an embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of a bow-shaped trajectory drawn by a mobile device in an existing conventional solution;

[0025] Figure 4is a schematic diagram of a bow-shaped trajectory drawn by a mobile device according to an embodiment of the present application;

[0026] Figure 5 It is a schematic block diagram of a loop closure positioning detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0028] In the following description, many specific details are elaborated to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited to the specific embodiments disclosed below.

[0029] Embodiment 1:

[0030] In some scenarios, a mobile device (such as a lawn mowing robot or a snow removal robot) needs to move along a set trajectory. When the mobile device moves along the trajectory for one circle and returns to the vicinity of the starting point, loop positioning detection is required to eliminate the accumulated errors during the movement of the mobile device.

[0031] Taking the lawn mowing scene as an example, since the scene near the mobile device is mainly lawn, and the scene far away may be trees, buildings, etc., the texture of the near image area in the image obtained by the mobile device in this scene is relatively simple, and most of the feature points are above the image, while the accuracy of the position information of the upper / far area is low. If loop positioning is performed, the positioning accuracy is low. Therefore, it is necessary to improve the accuracy of VSLAM loop positioning detection in such scenes.

[0032] like Figure 1 As shown, this embodiment provides a loop positioning detection method, the method comprising:

[0033] Step 10: When it is determined that the loop detection condition is met, the left eye image is divided into regions based on a preset ratio, and the first feature points in each divided region image are calculated respectively;

[0034] Specifically, if the calculated number of feature points, the current position of the mobile device, etc., it can be determined that the set loop detection conditions are met. At this time, in order to improve the accuracy of the loop positioning detection algorithm 3D-2D PNP, based on the resolution of the left eye image of the current frame, the left eye image is divided into two upper and lower regions, and then through feature point extraction methods, such as corner point detection algorithms, in the upper and lower area images, according to the set number of feature points, the same number of feature points are calculated to obtain more near feature points, and their descriptors are calculated, and these calculated / extracted feature points are recorded as first feature points.

[0035] It should be noted that when extracting feature points of the upper and lower area images, the same number of feature points can be set, or it can be adjusted according to actual conditions. When dividing the left eye image into regions, it can be divided into two parts, or into three parts: upper, middle and lower.

[0036] Step 20: Use the descriptor matching method to match the descriptors corresponding to the first feature points in the left-eye image with the loop-closure frame image to obtain points in the loop-closure frame image that match these first feature points, that is, determine the first matching points of the first feature points on the loop-closure frame image; wherein the loop-closure frame image is a historical key frame image (left-eye image).

[0037] Step 30: Calculate a second feature point matching the first matching point in the right image using a similarity matching algorithm;

[0038] Specifically, based on the binocular vision principle, the left and right images can be subjected to binocular image stereo correction using the calibrated internal and external parameters of the binocular camera, so that the left and right images can be aligned in rows. Then, for the first matching point corresponding to the first feature point of a row in the left image, the corresponding matching point is searched on the same row of the right image, and then sub-pixel difference is performed to determine the corresponding second feature point.

[0039] It should be noted that the related binocular stereo matching methods include SAD, SSD, CENSUS, NCC, etc. Considering that the light of the images captured by the left and right eye lenses in the binocular camera may be different, resulting in different image grayscales in the same area, therefore, this embodiment uses the NCC algorithm to match the feature points of the left and right eye images to avoid matching failures caused by image grayscale differences.

[0040] Step 40: Calculate a first 3D coordinate corresponding to the first feature point in the world coordinate system based on the pixel coordinates of the first feature point and the pixel coordinates of the second feature point;

[0041] Specifically, after finding the first feature point and the second feature point in the left and right eye images through the loop frame image, the corresponding disparity value can be calculated according to the pixel coordinates of these two points in the left and right eye images, and then combined with the current posture of the mobile device, the internal and external parameters of the binocular camera, the IMU information, etc., the 3D position of the first feature point in the left eye image after being transformed to the world coordinate system is calculated through coordinate transformation to obtain the first 3D coordinate.

[0042] Step 50: Based on the first 3D coordinate, correct the first position of the mobile device when acquiring the positioning image, wherein the positioning image includes a left-eye image and a right-eye image.

[0043] Specifically, the PNP algorithm can be used to calculate the more accurate position of the loop frame using the parameters obtained by the above calculation, which can be recorded as the fourth position. Then, the fourth position is used to perform loop positioning correction on the first position, so as to obtain a more accurate VSLAM loop positioning.

[0044] During the loop closure positioning correction process, the deviation between the fourth position and the historical position (third position) when the mobile device acquired the loop frame image at the historical moment can be calculated first, and then the deviation can be used to perform loop closure positioning correction on the first position.

[0045] In any of the above embodiments, further, this embodiment also shows a method that can accurately determine whether the loop detection condition is met. When the loop detection condition is met, the above process is executed to divide the left eye image; when the loop detection condition is not met, the loop positioning detection is not performed, and the currently acquired positioning image is discarded. The method also includes:

[0046] Step 101: Acquire feature point pairs in the left-eye image and the right-eye image, and calculate the second 3D coordinates of the feature point pairs;

[0047] Specifically, the real-time positioning part of the VSLAM system in this embodiment is divided into a front end and a back end, wherein the front end is responsible for processing the collected sensor image data in order to extract effective tracking features; the back end accepts the features tracked by the front end for optimization to obtain a more accurate trajectory and map.

[0048] After the mobile device in this embodiment acquires the positioning image (left eye image, right eye image) through the binocular camera, the front end of the VSLAM system can calculate the feature points (such as Shi-Tomasi feature points) on the left eye image through corner point detection.

[0049] Then, the obtained feature points are matched in the right image through a method such as optical flow tracking algorithm, and points matching the feature points in the left image are found to form feature point pairs. These feature point pairs are stored so that the VSLAM backend can be optimized to obtain the precise 2D pixel positions (pixel coordinates) and 3D positions of these feature point pairs. These 3D positions are recorded as 3D coordinates, which are recorded as second 3D coordinates.

[0050] It should be noted that, when calculating the feature points in the left-eye image, the number can be set within 200 to ensure real-time performance.

[0051] Step 102: when the left-eye image is determined to be the current key frame image, a historical key frame image with the highest similarity to the current key frame image is searched in the historical key frame image set, and recorded as a loop frame image;

[0052] Specifically, the VSLAM backend performs key frame judgment. The basis for the judgment can be the number of feature points tracked between the previous and next frames of the left-eye image (number of tracking points), the time interval with the previous key frame, the moving distance, the rotation angle, etc., to determine whether the currently acquired left-eye image is a key frame image. If not, the left-eye image and the corresponding right-eye image are discarded.

[0053] If so, the bag-of-words model DBOW3 is used to search for the historical key frame image with the highest similarity to the current key frame image in the historical key frame image set, and the image most similar to the key frame is recorded as the loop frame image.

[0054] It should be noted that the historical key frame image set is an image set consisting of key frames acquired by the mobile device at historical moments, and each time a key frame is inserted, the feature points and their related feature descriptors are recalculated again, wherein the number of recalculated feature points can be set to about 500.

[0055] Step 103: Determine a second matching point that matches the feature point pair in the loop frame image by using a descriptor matching method;

[0056] Specifically, after obtaining the loop frame image, a descriptor brute force matching method can be used to find matching points in the loop frame image that match the feature point pair in the above step 101, and these matching points are recorded as second matching points.

[0057] Step 104: Based on the second 3D coordinate corresponding to the second matching point, calculate the second position of the mobile device when acquiring the left eye image. When the number of the counted second matching points is greater than the preset number, and when the position deviation between the second position and the fourth position is less than the preset distance, it is determined that the loop detection condition is met, wherein the fourth position is the current position of the mobile device corresponding to the loop frame image calculated based on the first 3D coordinate.

[0058] It should be noted that when performing loop positioning correction, the first 3D coordinates, the second 3D coordinates and the corresponding feature points can be used together with the PNP algorithm to calculate the fourth position.

[0059] In any of the above embodiments, further, based on the first 3D coordinate, correcting the first position of the mobile device when acquiring the positioning image specifically includes:

[0060] Determine whether the distance deviation between the fourth position and the third position is less than or equal to the cumulative error threshold, if not, discard the positioning image,

[0061] If so, based on the distance deviation, the first position of the mobile device when acquiring the positioning image is corrected, and the accumulated result between the distance deviation and the first position is used as the corrected position, wherein the third position is the historical position of the mobile device when acquiring the loop frame image.

[0062] In any of the above embodiments, further, considering that there are a large number of operations such as calculating the sum of gray values, mean, and square sum in the image window in the NCC algorithm, which causes the NCC algorithm to consume more time than other local matching algorithms, this embodiment calculates the image integral map in advance, and performs a rapid similarity calculation by looking up a table. Therefore, using the similarity matching algorithm, calculating the second feature point that matches the first matching point in the right eye image specifically includes:

[0063] Step 301: Calculate the image integral images of the pixels in the left image and the right image respectively according to the preset window size, and generate an NCC operator query table;

[0064] Specifically, in the left and right eye images, take any pixel point as the center point of the preset window, calculate the image integral map based on the pixel value of the pixel point in the window, and extract the relevant data in the integral map to generate an NCC operator query table. The NCC operator query table may include two tables, corresponding to the left eye image and the right eye image, respectively, wherein the table corresponding to the left eye image is shown in Table 1 below.

[0065] Table 1

[0066]

[0067] It should be noted that since the size of the preset window m*n is a preset value, when the coordinates of any pixel point (i, j) are determined, the mean value of the pixel in the left image is Pixel Square Cumulative Sum The value of is also determined, where the pixel mean value of the left image is It is the average of the accumulated sum of the pixel values ​​of all pixels in the preset window range corresponding to the pixel with the pixel coordinate (i, j) in the left image. (I, J) is determined by the set image resolution.

[0068] Step 302: Based on the NCC calculation principle and the preset parallax traversal interval, a similarity matching calculation formula is constructed; wherein the constructed similarity matching calculation formula is:

[0069]

[0070]

[0071] Where d is the preset disparity, (i, j) is the pixel coordinate of the first matching point, Ncc(d, (i, j)) is the similarity corresponding to the first matching point with pixel coordinates (i, j) when the preset disparity is d, and I LR (i,j),L′ L , L′ R is the intermediate parameter, I L (i, j+d) is the pixel value of the pixel point in the left image after the pixel coordinate (i, j) is horizontally offset by the preset disparity d. R (i, j) is the pixel value of the pixel with coordinates (i, j) in the right image, m*n is the size of the preset window, They are the pixel means in the corresponding preset windows of the pixel with the pixel coordinates (i, j) in the left image and the right image respectively.

[0072] Step 303: For any first matching point, a traversal calculation is performed based on the similarity matching calculation formula. In the right-eye image, the similarity corresponding to any preset disparity in the preset window is calculated by querying the operator in the NCC operator query table, wherein the preset disparity belongs to the preset disparity traversal interval. Each traversal calculation changes within the preset disparity traversal interval according to the preset step size, and the similarity corresponding to each preset disparity is calculated.

[0073] Specifically, since the integral image can calculate the mean and sum of squares in a window of the image in advance, the above formula They can all be quickly obtained through the integral graph calculated in advance.

[0074] The remaining time is mainly spent on In order to improve the calculation speed of the NCC algorithm.

[0075] Step 304: Selecting a preset disparity corresponding to the maximum similarity value, and based on the selected preset disparity, determining a feature point of the pixel point corresponding to the first matching point in the right-eye image, recorded as a second feature point.

[0076] After the second feature point in the right eye image is obtained in this embodiment, it can form a point pair with the first feature point in the left eye image, and then the corresponding spatial coordinates, ie, the first 3D coordinates, are calculated from the point pair.

[0077] However, considering that the above calculation process assumes that the first and second feature points are located in the same row in the left and right eye images, and in the process of obtaining the left and right eye images by distortion correction based on the original image, there may be an offset in the pixel rows, therefore, after obtaining the point pair consisting of the first and second feature points, the matching relationship between the pixel points within a certain range around the second feature point and the first feature point can be calculated to complete the optimization of the point pair, and then obtain a more accurate first 3D coordinate, thereby improving the accuracy of loop positioning detection.

[0078] This embodiment adopts a group of Figure 2 The bow-shaped trajectory shown in FIG. 1 is used for comparative testing. In the existing conventional method, the obtained bow-shaped trajectory is as follows: Figure 3 As shown, where a loop occurs (such as Figure 3 As shown by the arrow in the figure, due to the low loop accuracy, the trajectory jump is more serious; after adopting the above scheme in this embodiment, the resulting bow-shaped trajectory is as follows Figure 4 As shown in the figure, where the loop occurs, the positioning is more accurate and more consistent with the actual trajectory.

[0079] Embodiment 2:

[0080] like Figure 5 As shown, this embodiment also provides a loop positioning detection system 100, characterized in that the system 100 includes:

[0081] The region division unit 10 is configured to divide the left eye image into regions based on a preset ratio when it is determined that the loop detection condition is met, and respectively calculate the first feature points in each divided region image;

[0082] A matching unit 20, the matching unit 20 is configured to determine a first matching point of the first feature point on the loop frame image by using a descriptor matching method;

[0083] The matching unit 20 is further configured to: calculate a second feature point matching the first matching point in the right eye image using a similarity matching algorithm;

[0084] A coordinate calculation unit 30, the coordinate calculation unit 30 is configured to calculate a first 3D coordinate corresponding to the first feature point in a world coordinate system based on the pixel coordinates of the first feature point and the pixel coordinates of the second feature point;

[0085] The correction unit 40 is configured to correct a first position of the mobile device when acquiring a positioning image based on the first 3D coordinate, wherein the positioning image includes a left-eye image and a right-eye image.

[0086] In any of the above embodiments, further, the coordinate calculation unit 30 is further configured to: obtain feature point pairs in the left-eye image and the right-eye image, and calculate the second 3D coordinates of the feature point pairs;

[0087] The system 100 further includes:

[0088] The image search unit 50 is configured to search for a historical key frame image with the highest similarity to the current key frame image in the historical key frame image set when determining that the left-eye image is the current key frame image, and record it as a loop frame image;

[0089] The matching unit 20 is further configured to: determine a second matching point matching the feature point pair in the loop frame image by using a descriptor matching method;

[0090] The system 100 further includes:

[0091] A position calculation unit 60, the position calculation unit 60 is configured to calculate a second position of the mobile device when acquiring the left-eye image based on the second 3D coordinates corresponding to the second matching point;

[0092] The judgment unit 70 is configured to determine that the loop detection condition is met when the number of second matching points counted is greater than a preset number and the position deviation between the second position and the fourth position is less than a preset distance, wherein the fourth position is the current position of the mobile device corresponding to the loop frame image calculated based on the first 3D coordinate.

[0093] In any of the above embodiments, further, the correction unit 40 is specifically configured to: determine whether the distance deviation between the fourth position and the third position is less than or equal to the cumulative error threshold; if not, discard the positioning image; if so, based on the distance deviation, correct the first position when the mobile device acquires the positioning image, wherein the third position is the historical position when the mobile device acquires the loop frame image.

[0094] In any of the above embodiments, further, the matching unit 20 is specifically configured to: respectively calculate the image integral images of the pixel points in the left-eye image and the right-eye image, and generate an NCC operator query table; based on the NCC calculation principle and the preset disparity traversal interval, construct a similarity matching calculation formula; for any first matching point, based on the similarity matching calculation formula, in the right-eye image, by querying the operator in the NCC operator query table, calculate the similarity corresponding to any preset disparity within the preset window, wherein the preset disparity belongs to the preset disparity traversal interval; select the preset disparity corresponding to the maximum similarity value, and based on the selected preset disparity, determine the feature point of the pixel point corresponding to the first matching point in the right-eye image, and record it as the second feature point.

[0095] Embodiment 3:

[0096] This embodiment further provides a computer-readable storage medium, in which a computer program is stored. When the stored computer program is executed by a processor, the loop positioning detection method of any one of the above-mentioned embodiments 1 is implemented.

[0097] Specifically, the computer program includes computer instructions, which are stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method steps in any one of the above-mentioned embodiments 1.

[0098] The above-mentioned computer-readable storage medium may include random access memory (RAM), read-only memory (ROM), compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage or other magnetic storage devices, etc.

[0099] The above-mentioned processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0100] So far, various embodiments of the present application have been described in detail. In order to avoid obscuring the concept of the present application, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0101] Although some specific embodiments of the present application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration rather than for limiting the scope of the present application.

[0102] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0103] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. A loop positioning detection method, characterized in that: The method comprises: When it is determined that the loop detection condition is met, the left eye image is divided into regions based on a preset ratio, and the first feature points in each divided region image are calculated respectively; Determine a first matching point of the first feature point on the loop frame image by using a descriptor matching method; Using a similarity matching algorithm, calculate a second feature point in the right eye image that matches the first matching point; Calculate a first 3D coordinate corresponding to the first feature point in a world coordinate system based on the pixel coordinates of the first feature point and the pixel coordinates of the second feature point; Based on the first 3D coordinates, a first position of the mobile device when acquiring a positioning image is corrected, wherein the positioning image includes the left-eye image and the right-eye image.

2. The loop positioning detection method according to claim 1, characterized in that: The method further comprises: Acquire feature point pairs in the left-eye image and the right-eye image, and calculate second 3D coordinates of the feature point pairs; When it is determined that the left-eye image is the current key frame image, searching the historical key frame image set for the one with the highest similarity to the current key frame image, and recording it as the loop frame image; Determine, in the loop frame image, a second matching point that matches the feature point pair by using a descriptor matching method; Calculating a second position of the mobile device when acquiring the left-eye image based on a second 3D coordinate corresponding to the second matching point; When the number of the second matching points counted is greater than the preset number, and When the position deviation between the second position and the fourth position is less than a preset distance, it is determined that the loop detection condition is met. The fourth position is a current position of the mobile device corresponding to the loop frame image and calculated based on the first 3D coordinates.

3. The loop positioning detection method according to claim 2, characterized in that: The correcting, based on the first 3D coordinate, the first position of the mobile device when acquiring the positioning image specifically includes: determining whether the distance deviation between the fourth position and the third position is less than or equal to a cumulative error threshold, and if not, discarding the positioning image; If yes, based on the distance deviation, the first position when the mobile device acquires the positioning image is corrected, The third position is a historical position when the mobile device acquires the loop frame image.

4. The loop positioning detection method according to any one of claims 1 to 3, characterized in that: The using a similarity matching algorithm to calculate a second feature point matching the first matching point in the right image specifically includes: Calculate the image integral images of the pixels in the left image and the right image respectively, and generate an NCC operator query table; Based on the NCC calculation principle and the preset parallax traversal interval, a similarity matching calculation formula is constructed; For any of the first matching points, based on the similarity matching calculation formula, in the right-eye image, by querying an operator in the NCC operator query table, the similarity corresponding to any preset disparity in the preset window is calculated, Wherein, the preset disparity belongs to the preset disparity traversal interval; A preset disparity corresponding to the maximum similarity value is selected, and based on the selected preset disparity, a feature point of the pixel point corresponding to the first matching point is determined in the right-eye image, and recorded as the second feature point.

5. The loop positioning detection method according to claim 4, characterized in that: The similarity matching calculation formula is: Where d is the preset disparity, (i, j) is the pixel coordinate of the first matching point, Ncc(d, (i, j)) is the similarity corresponding to the first matching point with pixel coordinates (i, j) when the preset disparity is d, and I LR (i,j),L ′ L , L ′ R is the intermediate parameter, I L (i, j+d) is the pixel value of the pixel point in the left image after the pixel coordinate (i, j) is horizontally offset by the preset disparity d. R (i, j) is the pixel value of the pixel with the coordinates (i, j) in the right image, m*n is the size of the preset window, They are respectively the pixel means within the preset windows corresponding to the pixel points with pixel coordinates (i, j) in the left image and the right image.

6. A loop positioning detection system, characterized in that: The system comprises: A region division unit, wherein the region division unit is configured to divide the left eye image into regions based on a preset ratio when it is determined that the loop detection condition is met, and respectively calculate the first feature points in each divided region image; A matching unit, wherein the matching unit is configured to determine a first matching point of the first feature point on the loop frame image by using a descriptor matching method; The matching unit is further configured to: calculate a second feature point matching the first matching point in the right eye image using a similarity matching algorithm; A coordinate calculation unit, wherein the coordinate calculation unit is configured to calculate a first 3D coordinate corresponding to the first feature point in a world coordinate system based on the pixel coordinates of the first feature point and the pixel coordinates of the second feature point; A correction unit is configured to correct a first position of a mobile device when acquiring a positioning image based on the first 3D coordinate, wherein the positioning image includes the left-eye image and the right-eye image.

7. The loop positioning detection system according to claim 6, characterized in that: The coordinate calculation unit is further configured to: Acquire feature point pairs in the left-eye image and the right-eye image, and calculate second 3D coordinates of the feature point pairs; The system further comprises: An image search unit, wherein when determining that the left-eye image is a current key frame image, the image search unit is configured to search for a historical key frame image with the highest similarity to the current key frame image in a set of historical key frame images, and record the historical key frame image as the loop frame image; The matching unit is further configured to: determine a second matching point matching the feature point pair in the loop frame image by using a descriptor matching method; The system further comprises: a position calculation unit, the position calculation unit being configured to calculate a second position of the mobile device when acquiring the left-eye image based on a second 3D coordinate corresponding to the second matching point; A judging unit, wherein the judging unit is configured to, when the number of the second matching points counted is greater than a preset number, When the position deviation between the second position and the fourth position is less than a preset distance, it is determined that the loop detection condition is met. The fourth position is a current position of the mobile device corresponding to the loop frame image and calculated based on the first 3D coordinates.

8. The loop positioning detection system according to claim 7, characterized in that: The correction unit is specifically configured as follows: determining whether the distance deviation between the fourth position and the third position is less than or equal to a cumulative error threshold, and if not, discarding the positioning image; If yes, based on the distance deviation, the first position when the mobile device acquires the positioning image is corrected, The third position is a historical position when the mobile device acquires the loop frame image.

9. The loop positioning detection system according to any one of claims 6 to 8, characterized in that: The matching unit is specifically configured as follows: Calculate the image integral images of the pixels in the left image and the right image respectively, and generate an NCC operator query table; Based on the NCC calculation principle and the preset parallax traversal interval, a similarity matching calculation formula is constructed; For any of the first matching points, based on the similarity matching calculation formula, in the right-eye image, by querying an operator in the NCC operator query table, the similarity corresponding to any preset disparity in the preset window is calculated, Wherein, the preset disparity belongs to the preset disparity traversal interval; A preset disparity corresponding to the maximum similarity value is selected, and based on the selected preset disparity, a feature point of the pixel point corresponding to the first matching point is determined in the right-eye image, and recorded as the second feature point.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images