A cross-station planetary rover visual positioning method, system, device and medium

Abstract

The present invention discloses a method, system, device and medium for visual positioning of a cross-station rover, which relates to the field of visual navigation and positioning technology. The method includes: confirming the regional scope of the content of the photos taken before and after the rover moves, first matching the distant area, and performing projection transformation on the area to be matched according to the matching result, gradually transitioning from far to near to match near, and finally obtaining the matching points of the target area and the positioning result of the rover. The present invention gradually brings the matching area closer to obtain the accurate projection transformation relationship of the area that is most conducive to positioning. The difficulty of feature matching of the area to be matched after the projection transformation is greatly reduced, and more matching points with higher quality can be obtained, thereby greatly improving the success rate and positioning accuracy of the cross-station visual positioning of the rover.

Description

Background Art

[0002] When the rover is driving on the surface of the planet, it needs to use the navigation camera images taken by different stations to visually locate the rover and determine its position in the working coordinate system, so as to support path planning and other tasks. Before visual positioning, it is necessary to perform feature matching on the overlapping areas in the images of the front and rear stations. The accuracy of feature matching directly affects the positioning accuracy.

[0003] Since the rover moves a long distance in a single move, there are large scale and shape differences between the same features in the images of the front and rear stations. It is difficult to match the original images of the front and rear stations directly, or only a small number of feature points can be matched. The traditional method uses planning or telemetry information to estimate the station posture and calculate the overlapping area of ​​the front and rear station images, thereby narrowing the matching range and reducing the difficulty of matching. However, this method is heavily dependent on the accuracy of the estimated station information. When the estimated position deviation is large, the overlapping area calculation is inaccurate and cannot be correctly matched. In addition, the traditional method only uses the scale-invariant and affine-invariant characteristics of the general feature descriptor to deal with the scale and shape differences of features between images, which is difficult to adapt to the large scale and shape differences of features between images of different rover stations. When the traditional automatic feature matching method fails, it is necessary to manually select matching feature points between images of different stations. This process is time-consuming and labor-intensive, and the positions of the manually selected matching feature points are not accurate enough, which easily leads to inaccurate visual positioning results. Summary of the invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the prior art, specifically the problem of inaccurate visual positioning results, and specifically provide a cross-station rover visual positioning method, system, device and medium, as follows:

[0005] 1) In the first aspect, the present invention provides a cross-station rover visual positioning method, and the specific technical solution is as follows:

[0006] S1, obtaining a first image taken by a rover equipped with a shooting device for a target area before movement and a second image taken for the target area after movement, performing binocular stereo matching on the first image and the second image respectively, obtaining first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, and determining a first initial area range and a first process area range corresponding to the first image and a second initial area range and a second process area range corresponding to the second image based on the first sparse depth information and the second sparse depth information;

[0007] S2, matching the image within the first initial area with the image within the second initial area to obtain an initial matching result;

[0008] S3, comparing the first distance and the second distance according to the position information of the initial matching result, wherein the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image, and projecting the image with a smaller distance value to the perspective of the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image;

[0009] S4, taking the third image as the first image or the second image, replacing the first initial area range and the second initial area range with the first process area range and the second process area range, and repeating S2 and S3 in a manner in which the distance between the matching area range and the shooting point gradually decreases, until the matching result meets the preset requirements, thereby completing the positioning of the rover.

[0010] The beneficial effects of the cross-station rover visual positioning method provided by the present invention are as follows:

[0011] By gradually bringing the matching area closer, we can obtain matching points in the area that is most conducive to positioning. The difficulty of feature matching in the area to be matched after projection transformation is greatly reduced, so more matching points can be obtained, thereby improving the positioning accuracy.

[0012] Based on the above solution, the present invention can also be improved as follows.

[0013] Furthermore, the image within the first initial area is matched with the image within the second initial area, and the initial matching result is obtained as follows:

[0014] If the first initial area range and the second initial area range only contain one distance range, corresponding matching is performed on areas with the same range value in the first initial area range and the second initial area range, and the coplanar assumption RANSAC method is used to filter and eliminate the matching results to obtain an initial matching result.

[0015] If the first initial area range and the second initial area range contain multiple distance ranges, corresponding matching is performed on multiple areas with the same range values ​​in the first initial area range and the second initial area range to obtain multiple matching results, and each matching result is screened and eliminated using the coplanar assumption RANSAC method to obtain multiple groups of matching point pairs, and all matching point pairs are collected and screened and eliminated again using the coplanar assumption RANSAC method to obtain initial matching results.

[0016] Further, the first distance is compared with the second distance, where the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image. The image with a smaller distance value is projected to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image in the perspective of the image, specifically:

[0017] Determine a first distance value between the initial matching result and the shooting point in the first image;

[0018] Determine a second distance value between the initial matching result and the shooting point in the second image;

[0019] When the first distance value is less than the second distance value, the first image is projected to the perspective of the second image through the homography matrix corresponding to the initial matching result to generate a third image;

[0020] When the first distance value is greater than the second distance value, the second image is projected to the perspective of the first image through the homography matrix corresponding to the initial matching result to generate a third image.

[0021] Further, S2 and S3 are repeatedly performed in a manner that the distance between the matching area and the shooting point gradually decreases until the matching result meets the preset requirements, specifically:

[0022] Arrange the first process area range and the second process area range in descending order of distance from the shooting point to generate a first sequence and a second sequence;

[0023] Using the third image as the first image or the second image, matching the image area corresponding to the first first area range in the first sequence and the image area corresponding to the first second area range in the second sequence to obtain a matching result and a homography matrix corresponding to the matching result;

[0024] Taking the first matching result as the initial matching result, taking the first homography matrix as the homography matrix corresponding to the initial matching result, executing S3, and obtaining a fourth image;

[0025] Using the fourth image as the first image or the second image, matching the image area corresponding to the current area range or the next area range in the first sequence and the second sequence until all area ranges in the first sequence and the second sequence are matched and the matching results meet the preset requirements;

[0026] One or more matches can be performed within the same area.

[0027] Furthermore, it also includes:

[0028] S5, screening and eliminating the matching results to obtain accurate matching point pairs, and using the accurate matching points to locate the rover.

[0029] 2) In the second aspect, the present invention also provides a cross-station rover visual positioning system, and the specific technical solution is as follows:

[0030] The acquisition module is used to: acquire a first image captured by a rover equipped with a shooting device for a target area before the rover moves and a second image captured for the target area after the rover moves, perform binocular stereo matching on the first image and the second image respectively, obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, and determine a first initial area range and a first process area range corresponding to the first image and a second initial area range and a second process area range corresponding to the second image based on the first sparse depth information and the second sparse depth information;

[0031] The matching module is used to: match the image within the first initial area with the image within the second initial area to obtain an initial matching result;

[0032] The projection module is used to: compare the first distance and the second distance according to the position information of the initial matching result, the first distance being the distance between the initial matching result and the shooting point in the first image, and the second distance being the distance between the initial matching result and the shooting point in the second image, and project the image with a smaller distance value to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image;

[0033] The positioning module is used to: use the third image as the first image or the second image, replace the first initial area range and the second initial area range with the first process area range and the second process area range, and repeat the execution content of the matching module and the projection module in a manner in which the distance between the matching area range and the shooting point gradually decreases until the matching result meets the preset requirements, thereby completing the positioning of the rover.

[0034] Based on the above solution, the present invention can also be improved as follows.

[0035] Furthermore, the image within the first initial area is matched with the image within the second initial area, and the initial matching result is obtained as follows:

[0036] If the first initial area range and the second initial area range only contain one distance range, corresponding matching is performed on areas with the same range value in the first initial area range and the second initial area range, and the coplanar assumption RANSAC method is used to filter and eliminate the matching results to obtain an initial matching result.

[0037] If the first initial area range and the second initial area range contain multiple distance ranges, corresponding matching is performed on multiple areas with the same range values ​​in the first initial area range and the second initial area range to obtain multiple matching results, and each matching result is screened and eliminated using the coplanar assumption RANSAC method to obtain multiple groups of matching point pairs, and all matching point pairs are collected and screened and eliminated again using the coplanar assumption RANSAC method to obtain initial matching results.

[0038] Further, the first distance is compared with the second distance, where the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image. The image with a smaller distance value is projected to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image in the perspective of the image, specifically:

[0039] Determine a first distance value between the initial matching result and the shooting point in the first image;

[0040] Determine a second distance value between the initial matching result and the shooting point in the second image;

[0041] When the first distance value is less than the second distance value, the first image is projected to the perspective of the second image through the homography matrix corresponding to the initial matching result to generate a third image;

[0042] When the first distance value is greater than the second distance value, the second image is projected to the perspective of the first image through the homography matrix corresponding to the initial matching result to generate a third image.

[0043] Further, the matching module and the projection module are executed repeatedly in a manner that the distance between the matching area and the shooting point gradually decreases until the matching result meets the preset requirements, specifically:

[0044] Arrange the first process area range and the second process area range in descending order of distance from the shooting point to generate a first sequence and a second sequence;

[0045] Using the third image as the first image or the second image, matching the image area corresponding to the first first area range in the first sequence and the image area corresponding to the first second area range in the second sequence to obtain a matching result and a homography matrix corresponding to the matching result;

[0046] Taking the first matching result as the initial matching result, taking the first homography matrix as the homography matrix corresponding to the initial matching result, executing the content of the projection module, and obtaining a fourth image;

[0047] Using the fourth image as the first image or the second image, matching the image area corresponding to the current area range or the next area range in the first sequence and the second sequence until all area ranges in the first sequence and the second sequence are matched and the matching results meet the preset requirements;

[0048] One or more matches can be performed within the same area.

[0049] Furthermore, it also includes:

[0050] The precise positioning module is used to screen and eliminate the matching results to obtain precise matching point pairs, and use the precise matching points to position the rover.

[0051] 3) In a third aspect, the present invention further provides a computer device, comprising a processor, wherein the processor is coupled to a memory, wherein the memory stores at least one computer program, and wherein the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above methods.

[0052] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above methods.

[0053] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0055] Figure 1 A schematic diagram of a flow chart of a cross-station rover visual positioning method according to an embodiment of the present invention;

[0056] Figure 2 A structural framework diagram of a cross-station rover visual positioning system according to an embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of the structure of the computer device of the present invention;

[0058] Figure 4 This is a schematic diagram of a real image. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0060] like Figure 1 As shown, a cross-station rover visual positioning method according to an embodiment of the present invention comprises the following steps:

[0061] S1, obtaining a first image taken by a rover equipped with a shooting device for a target area before the rover moves and a second image taken for the target area after the rover moves, performing binocular stereo matching on the first image and the second image respectively to obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, and determining a first initial area range and a first process area range corresponding to the first image and a second initial area range and a second process area range corresponding to the second image based on the first sparse depth information and the second sparse depth information;

[0062] S2, matching the image within the first initial area with the image within the second initial area to obtain an initial matching result;

[0063] S3, comparing the first distance and the second distance according to the position information of the initial matching result, wherein the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image, and projecting the image with a smaller distance value to the perspective of the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image;

[0064] S4, taking the third image as the first image or the second image, replacing the first initial area range and the second initial area range with the first process area range and the second process area range, and repeating S2 and S3 in a manner in which the distance between the matching area range and the shooting point gradually decreases, until the matching result meets the preset requirements, thereby completing the positioning of the rover.

[0065] The beneficial effects of the cross-station rover visual positioning method provided by the present invention are as follows:

[0066] By gradually bringing the matching area closer, we can obtain matching points in the area that is most conducive to positioning. The difficulty of feature matching in the area to be matched after projection transformation is greatly reduced, so more matching points can be obtained, thereby improving the positioning accuracy.

[0067] In S1, the photographing device may be a camera;

[0068] The movement direction includes but is not limited to: forward and backward.

[0069] Determining the first initial area range and the second initial area range corresponding to the first image and the second image based on the first sparse depth information and the second sparse depth information is specifically as follows:

[0070] In this example, three distance ranges of distant areas are determined in each image based on sparse depth information, that is, the first initial area range includes three, and the second initial area range also includes three; it should be noted that the value of each distance range is consistent in the two images.

[0071] For example, the initial regions for the three distance ranges are set to [d gif ,d gin(1) ]、[d gif ,d gin(2) ]、[d gif ,d gin(3) ], where d gif is the maximum distance, d gin(1) d gin(2) d gin(3) Represent different close distances, and d gin(1) >d gin(2) >d gin(3) .

[0072] Determining the first process area range and the second process area range corresponding to the first image and the second image based on the first sparse depth information and the second sparse depth information is specifically as follows:

[0073] In this example, three distance ranges for gradually narrowing the matching area are determined in each image based on the sparse depth information, and each distance range has the same value in the two images.

[0074] For example: The process area of ​​three distance ranges is set to [d gf(1) ,d gn(1) ]、[d gf(2) ,d gn(2) ]、[d gf(3) ,d gn(3) ],d gf(1) d gf(2) d gf(3) Represents the distance, d gn(1) d gn(2) d gn(3) represents the near distance, and d gf(1) ≥d gf(2) ≥d gf(3) , d gn(1) ≥d gn(2) ≥d gn(3) In order to better connect with the initial area range, d gf(1) Set to d gif Same, dgn(1) Set to d gin(3) same.

[0075] like Figure 4 As shown, S2, the image within the first initial area is matched with the image within the second initial area to obtain an initial matching result.

[0076] In each distance range, the image in the first initial area is matched with the image in the second initial area. Since three distance ranges are given in this scheme, three matches are required here. Each matching result is processed by eliminating wrong matches according to the homography hypothesis RANSAC method to obtain three sets of matching point pairs, specifically: M ginit(1) 、M ginit(2) 、M ginit(3) .

[0077] Among them, matching refers to extracting and describing features in the area to be matched by using a general feature extraction and description operator such as SIFT, and then obtaining matching point pairs using a matching algorithm such as brute force matching.

[0078] After obtaining three sets of matching results, the three sets of results are merged, and then the coplanar assumption RANSAC method is used to screen the merged matching point pairs to obtain the initial matching result M ginit And the homography matrix H corresponding to the initial matching result ginit .

[0079] The surface area of ​​the planet where the rover is traveling is usually relatively flat, so the local area can be approximated as a plane, and the homography assumption can better screen out the correct matching point pairs. By setting multiple distance ranges, matching the distant areas with multiple distance ranges is more conducive to obtaining a sufficient number of initial matching point pairs. Since the shape and scale changes of distant scenes in the images of the two stations are relatively small and easier to match, the initial matching areas are all selected as distant areas. Matching areas with multiple distance ranges help adapt to different terrains and increase the probability of successful matching.

[0080] S3, according to the location information of the initial matching result, compare the first distance and the second distance, the first distance is the distance between the initial matching result and the shooting point in the first image, the second distance is the distance between the initial matching result and the shooting point in the second image, and project the image with a smaller distance value to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image. Wherein:

[0081] The initial matching results contain the specific location information of the matching point pairs;

[0082] Determine a first distance value between the initial matching result and the shooting point in the first image;

[0083] Determine a second distance value between the initial matching result and the shooting point in the second image;

[0084] When the first distance value is less than the second distance value, the first image is projected to the perspective of the second image by using the first homography matrix corresponding to the initial matching result to generate a third image;

[0085] When the first distance value is greater than the second distance value, the second image is projected to the viewing angle of the first image through the first homography matrix corresponding to the initial matching result to generate a third image.

[0086] S4, using the third image as the first image or the second image, replacing the first initial area range and the second initial area range with the first process area range and the second process area range, and repeating S2 and S3 in a manner that the distance between the matching area range and the shooting point gradually decreases, until the matching result meets the preset requirements, thereby completing the positioning of the rover. Wherein:

[0087] When the first distance value is less than the second distance value, taking the third image as the first image;

[0088] When the first distance value is greater than the second distance value, the third image is used as the second image.

[0089] S2 and S3 are repeatedly executed in a manner that the distance between the matching area and the shooting point gradually decreases, specifically:

[0090] From the above example, we can see that the distance ranges of the three process areas are set to [d gf(1) ,d gn(1) ]、[d gf(2) ,d gn(2) ]、[d gf(3) ,d gn(3) ], and d gf(1) ≥d gf(2) ≥d gf(3) , d gn(1) ≥d gn(2) ≥d gn(3) Therefore, the three process areas are arranged in an order from far to near in the image. The homography matrix H corresponding to the initial matching result is used. ginit , obtain the third image according to the method given in S3, and use the third image as the first image or the second image according to the distance value, and perform the distance range [d gf(1) ,d gn(1) ], and obtain a matching point pair M g(1) And the homography matrix H corresponding to a matching point pair g(1) . So far, the matching in the first distance range is completed;

[0091] Using a homography matrix H g(1) , obtain the fourth image according to the method given in S3, and use the fourth image as the first image or the second image according to the distance value, and perform the distance range [d gf(2) ,d gn(2) ], and obtain the secondary temporary matching point pair M g(2)_t And the secondary temporary homography matrix H corresponding to the secondary temporary matching point pair g(2)_t Keeping the distance range unchanged, use the secondary temporary homography matrix H g(2)_t The fifth image is obtained according to the method given in S3, and the fifth image is used as the first image or the second image according to the distance value to perform a second matching in the distance range to obtain a second matching point pair M. g(2) And the quadratic homography matrix H corresponding to the quadratic matching point pair g(2) . So far, the matching under the second distance range is completed;

[0092] Using the quadratic homography matrix H g(2) , obtain the sixth image according to the method given in S3, and use the sixth image as the first image or the second image according to the distance value, and perform the distance range [d gf(3) ,d gn(3) ], and obtain three temporary matching point pairs M g(3)_t And the cubic temporary homography matrix H corresponding to the cubic temporary matching point pairs g(3)_t Keeping the distance range unchanged, use the cubic temporary homography matrix H g(3)_t The seventh image is obtained according to the method given in S3, and the seventh image is used as the first image or the second image according to the distance value to perform the second matching in the distance range, and three matching point pairs M are obtained. g(3) And the cubic homography matrix H corresponding to the cubic matching point pairs g(3) At this point, the matching under the third distance range is completed. g(3)_t And three matching points M g(3) Merge as the final valid matching point pair M gval .

[0093] After S4, the effective matching point pairs may be screened and eliminated, specifically:

[0094] Screening and elimination refer to: screening and eliminating matching points according to the number of neighboring points and feature significance.

[0095] For example: After obtaining the valid matching point pair M gvalAfter that, first calculate the number of neighboring points of each valid matching point pair. If the distance between a valid matching point pair and the target valid matching point pair is less than r (preset distance), then the valid matching point pair is considered to be a neighboring point of the target valid matching point pair; according to the above process, traverse all valid matching point pairs. Next, move the valid matching point pair with 0 neighbors into the candidate area, process the valid matching point pair with the largest number of neighbors, and move the valid matching point pair with the highest feature significance among the valid matching point pair and the neighboring point pair (the highest feature significance refers to the highest feature detection response value of the point under the selected general feature extraction algorithm) into the candidate, and delete other point pairs. Similarly, recalculate the number of neighboring points of the valid matching point pair to be processed, and process the valid matching point pair with 0 neighbors and the largest number of neighbors and the surrounding valid matching point pairs until the number of valid matching point pairs to be processed is 0. Finally, all valid matching point pairs in the candidate area are used as matching point pairs M for camera relative pose estimation. guse , determine the position of the rover according to the relative position of the camera.

[0096] The advantages of the present invention are mainly manifested in:

[0097] (1) More effective matching points: By gradually bringing the matching area closer, we can obtain the matching points in the area that is most conducive to positioning. The difficulty of feature matching in the area to be matched after projection transformation is greatly reduced, so more matching points can be obtained.

[0098] (2) More accurate matching point positions: Since the accurate projection transformation relationship of the area to be matched is obtained, there is almost no difference in perspective and scale in the area to be matched after projection. Therefore, the consistency of feature points is better and the matching point positions are more accurate.

[0099] (3) No prior position information is required: When matching distant areas in an image, there is no need to know the relative position of the rover before and after it moves. The matching points obtained from the first matching are used as landmarks to obtain a rough projection transformation relationship between images, thereby guiding subsequent matching.

[0100] Furthermore, the image within the first initial area is matched with the image within the second initial area, and the initial matching result is obtained as follows:

[0101] If the first initial area range and the second initial area range only include one distance range, corresponding matching is performed on areas with the same range value in the first initial area range and the second initial area range, and the matching results are screened and eliminated using the coplanar assumption RANSAC method to obtain an initial matching result;

[0102] If the first initial area range and the second initial area range contain multiple distance ranges, corresponding matching is performed on multiple areas with the same range values ​​in the first initial area range and the second initial area range to obtain multiple matching results, and each matching result is screened and eliminated using the coplanar assumption RANSAC method to obtain multiple groups of matching point pairs, and all matching point pairs are collected and screened and eliminated again using the coplanar assumption RANSAC method to obtain initial matching results.

[0103] Further, the first distance is compared with the second distance, where the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image. The image with a smaller distance value is projected to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image in the perspective of the image, specifically:

[0104] Determine a first distance value between the initial matching result and the shooting point in the first image;

[0105] Determine a second distance value between the initial matching result and the shooting point in the second image;

[0106] When the first distance value is less than the second distance value, the first image is projected to the perspective of the second image through the homography matrix corresponding to the initial matching result to generate a third image;

[0107] When the first distance value is greater than the second distance value, the second image is projected to the perspective of the first image through the homography matrix corresponding to the initial matching result to generate a third image.

[0108] Further, S2 and S3 are repeatedly performed in a manner that the distance between the matching area and the shooting point gradually decreases until the matching result meets the preset requirements, specifically:

[0109] Arrange the first process area range and the second process area range in descending order of distance from the shooting point to generate a first sequence and a second sequence;

[0110] Using the third image as the first image or the second image, matching the image area corresponding to the first first area range in the first sequence and the image area corresponding to the first second area range in the second sequence to obtain a matching result and a homography matrix corresponding to the matching result;

[0111] Taking the first matching result as the initial matching result, taking the first homography matrix as the homography matrix corresponding to the initial matching result, executing S3, and obtaining a fourth image;

[0112] Using the fourth image as the first image or the second image, matching the image area corresponding to the current area range or the next area range in the first sequence and the second sequence until all area ranges in the first sequence and the second sequence are matched and the matching results meet the preset requirements;

[0113] One or more matches can be performed within the same area.

[0114] Furthermore, it also includes:

[0115] S5, screening and eliminating the matching results to obtain accurate matching point pairs, and using the accurate matching points to locate the rover.

[0116] like Figure 2 As shown, the present invention also provides a cross-station rover visual positioning system 200, and the specific technical solution is as follows:

[0117] The acquisition module 210 is used to: acquire a first image captured by a rover equipped with a shooting device for a target area before the rover moves and a second image captured for the target area after the rover moves, perform binocular stereo matching on the first image and the second image respectively, obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, and determine a first initial area range and a first process area range corresponding to the first image and a second initial area range and a second process area range corresponding to the second image based on the first sparse depth information and the second sparse depth information;

[0118] The matching module 220 is used to: match the image within the first initial area with the image within the second initial area to obtain an initial matching result;

[0119] The projection module 230 is used to: compare the first distance and the second distance according to the position information of the initial matching result, wherein the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image, and project the image with a smaller distance value to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image;

[0120] The positioning module 240 is used to: use the third image as the first image or the second image, replace the first initial area range and the second initial area range with the first process area range and the second process area range, and repeat the execution content of the matching module and the projection module in a manner in which the distance between the matching area range and the shooting point gradually decreases until the matching result meets the preset requirements, thereby completing the positioning of the rover.

[0121] Furthermore, the image within the first initial area is matched with the image within the second initial area, and the initial matching result is obtained as follows:

[0122] If the first initial area range and the second initial area range only include one distance range, corresponding matching is performed on areas with the same range value in the first initial area range and the second initial area range, and the matching results are screened and eliminated using the coplanar assumption RANSAC method to obtain an initial matching result;

[0123] If the first initial area range and the second initial area range contain multiple distance ranges, corresponding matching is performed on multiple areas with the same range values ​​in the first initial area range and the second initial area range to obtain multiple matching results, and each matching result is screened and eliminated using the coplanar assumption RANSAC method to obtain multiple groups of matching point pairs, and all matching point pairs are collected and screened and eliminated again using the coplanar assumption RANSAC method to obtain initial matching results.

[0124] Further, the first distance is compared with the second distance, where the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image. The image with a smaller distance value is projected to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image in the perspective of the image, specifically:

[0125] Determine a first distance value between the initial matching result and the shooting point in the first image;

[0126] Determine a second distance value between the initial matching result and the shooting point in the second image;

[0127] When the first distance value is less than the second distance value, the first image is projected to the perspective of the second image through the homography matrix corresponding to the initial matching result to generate a third image;

[0128] When the first distance value is greater than the second distance value, the second image is projected to the perspective of the first image through the homography matrix corresponding to the initial matching result to generate a third image.

[0129] Further, the matching module and the projection module are executed repeatedly in a manner that the distance between the matching area and the shooting point gradually decreases until the matching result meets the preset requirements, specifically:

[0130] Arrange the first process area range and the second process area range in descending order of distance from the shooting point to generate a first sequence and a second sequence;

[0131] Using the third image as the first image or the second image, matching the image area corresponding to the first first area range in the first sequence and the image area corresponding to the first second area range in the second sequence to obtain a matching result and a homography matrix corresponding to the matching result;

[0132] Taking the first matching result as the initial matching result, taking the first homography matrix as the homography matrix corresponding to the initial matching result, executing the content of the projection module, and obtaining a fourth image;

[0133] Using the fourth image as the first image or the second image, matching the image area corresponding to the current area range or the next area range in the first sequence and the second sequence until all area ranges in the first sequence and the second sequence are matched and the matching results meet the preset requirements;

[0134] One or more matches can be performed within the same area.

[0135] Furthermore, it also includes:

[0136] The precise positioning module is used to screen and eliminate the matching results to obtain precise matching point pairs, and use the precise matching points to position the rover.

[0137] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0138] It should be noted that the beneficial effects of the cross-station rover visual positioning system provided by the above embodiment are the same as the beneficial effects of the cross-station rover visual positioning method, which will not be repeated here. In addition, when the system provided by the above embodiment realizes its functions, it only takes the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0139] like Figure 3As shown, a computer device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the computer device 300 implements any of the above methods, specifically:

[0140] The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein at least one computer program 330 is stored in the one or more memories 310, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the computer device 300 implements a cross-station rover visual positioning method provided by the above embodiment. Of course, the computer device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the computer device 300 may also include other components for implementing device functions, which will not be repeated here.

[0141] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.

[0142] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0143] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above methods.

[0144] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0145] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0146] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0147] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A cross-station rover visual positioning method, characterized in that: include: S1, obtaining a first image taken by a rover equipped with a shooting device for a target area before movement and a second image taken for the target area after movement, performing binocular stereo matching on the first image and the second image respectively, obtaining first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, and determining a first initial area range and a first process area range corresponding to the first image and a second initial area range and a second process area range corresponding to the second image based on the first sparse depth information and the second sparse depth information; S2, matching the image within the first initial area with the image within the second initial area to obtain an initial matching result; S3, comparing the first distance and the second distance according to the position information of the initial matching result, wherein the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image, and projecting the image with a smaller distance value to the perspective of the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image; S4, using the third image as the first image or the second image, replacing the first initial area range and the second initial area range with the first process area range and the second process area range, and repeating S2 and S3 in a manner in which the distance between the matching area range and the shooting point gradually decreases, until the matching result meets the preset requirements, thereby completing the positioning of the rover; Based on the sparse depth information, three distance ranges of distant areas are determined in each image, that is, the first initial area range includes three, and the second initial area range includes three; each distance range has the same value in the two images; Specifically, the initial regions of the three distance ranges are set to [d gif ,d gin(1) ]、[d gif ,d gin(2) ]、[d gif ,d gin(3) ], where d gif is the maximum distance, d gin(1) ,d gin(2) ,d gin(3) Represent different close distances, and d gin(1) >d gin(2) >d gin(3) ; Based on the sparse depth information, three distance ranges are determined in each image for gradually narrowing the matching area, and each distance range has the same value in the two images; Specifically, the process areas of the three distance ranges are set to [d gf(1) ,d gn(1) ]、[d gf(2) ,d gn(2) ]、[d gf(3) ,d gn(3) ],d gf(1) d gf(2) d gf(3) Represents the distance, d gn(1) d gn(2) d gn(3) represents the near distance, and d gf(1) ≥d gf(2) ≥d gf(3) , d gn(1) ≥d gn(2) ≥d gn(3) In order to better connect with the initial area range, d gf(1) Set to d gif Same, d gn(1) Set to d gin(3) same.

2. A cross-station rover visual positioning method according to claim 1, characterized in that: The image within the first initial area is matched with the image within the second initial area, and the initial matching result is obtained as follows: If the first initial area range and the second initial area range only include one distance range, corresponding matching is performed on areas with the same range value in the first initial area range and the second initial area range, and the matching results are screened and eliminated using the coplanar assumption RANSAC method to obtain an initial matching result; If the first initial area range and the second initial area range contain multiple distance ranges, corresponding matching is performed on multiple areas with the same range values ​​in the first initial area range and the second initial area range to obtain multiple matching results, and each matching result is screened and eliminated using the coplanar assumption RANSAC method to obtain multiple groups of matching point pairs, and all matching point pairs are collected and screened and eliminated again using the coplanar assumption RANSAC method to obtain initial matching results.

3. The visual positioning method for a cross-station rover according to claim 1, characterized in that: Comparing the first distance and the second distance, where the first distance is the distance between the initial matching result and the shooting point in the first image, and the second distance is the distance between the initial matching result and the shooting point in the second image, and projecting the image with a smaller distance value to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image, specifically: Determine a first distance value between the initial matching result and the shooting point in the first image; Determine a second distance value between the initial matching result and the shooting point in the second image; When the first distance value is less than the second distance value, the first image is projected to the perspective of the second image through the homography matrix corresponding to the initial matching result to generate a third image; When the first distance value is greater than the second distance value, the second image is projected to the perspective of the first image through the homography matrix corresponding to the initial matching result to generate a third image.

4. A cross-station rover visual positioning method according to claim 3, characterized in that: In a manner where the distance between the matching area and the shooting point gradually decreases, S2 and S3 are repeatedly executed until the matching result meets the preset requirements: Arrange the first process area range and the second process area range in descending order of distance from the shooting point to generate a first sequence and a second sequence; Using the third image as the first image or the second image, matching the image area corresponding to the first first area range in the first sequence and the image area corresponding to the first second area range in the second sequence to obtain a matching result and a homography matrix corresponding to the matching result; Taking the first matching result as the initial matching result, taking the first homography matrix as the homography matrix corresponding to the initial matching result, executing S3, and obtaining a fourth image; Using the fourth image as the first image or the second image, matching the image area corresponding to the current area range or the next area range in the first sequence and the second sequence until all area ranges in the first sequence and the second sequence are matched and the matching results meet the preset requirements; One or more matches can be performed within the same area.

5. The visual positioning method for a cross-station rover according to claim 1, characterized in that: Also includes: S5, screening and eliminating the matching results to obtain accurate matching point pairs, and using the accurate matching points to locate the rover.

6. A cross-station rover visual positioning system, characterized in that: include: The acquisition module is used to: acquire a first image captured by a rover equipped with a shooting device for a target area before the rover moves and a second image captured for the target area after the rover moves, perform binocular stereo matching on the first image and the second image respectively, obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, and determine a first initial area range and a first process area range corresponding to the first image and a second initial area range and a second process area range corresponding to the second image based on the first sparse depth information and the second sparse depth information; The matching module is used to: match the image within the first initial area with the image within the second initial area to obtain an initial matching result; The projection module is used to: compare the first distance and the second distance according to the position information of the initial matching result, the first distance being the distance between the initial matching result and the shooting point in the first image, and the second distance being the distance between the initial matching result and the shooting point in the second image, and project the image with a smaller distance value to the image with a larger distance value according to the homography matrix corresponding to the initial matching result to generate a third image; The positioning module is used to: use the third image as the first image or the second image, replace the first initial area range and the second initial area range with the first process area range and the second process area range, and repeat the execution content of the matching module and the projection module in a manner that the distance between the matching area range and the shooting point gradually decreases until the matching result meets the preset requirements, thereby completing the positioning of the rover; Based on the sparse depth information, three distance ranges of distant areas are determined in each image, that is, the first initial area range includes three, and the second initial area range includes three; each distance range has the same value in the two images; Specifically, the initial regions of the three distance ranges are set to [d gif ,d gin(1) ]、[d gif ,d gin(2) ]、[d gif ,d gin(3) ], where d gif is the maximum distance, d gin(1) d gin(2) d gin(3) Represent different close distances, and d gin(1) >d gin(2) >d gin(3) ; Based on the sparse depth information, three distance ranges are determined in each image for gradually narrowing the matching area, and each distance range has the same value in the two images; Specifically, the process areas of the three distance ranges are set to [d gf(1) ,d gn(1) ]、[d gf(2) ,d gn(2) ]、[d gf(3) ,d gn(3) ],d gf(1) ,d gf(2) ,d gf(3) Represents the distance, d gn(1) ,d gn(2) ,d gn(3) represents the near distance, and d gf(1) ≥d gf(2) ≥d gf(3) , d gn(1) ≥d gn(2) ≥d gn(3) In order to better connect with the initial area range, d gf(1) Set to d gif Same, d gn(1) Set to d gin(3) same.

7. A cross-station rover visual positioning system according to claim 6, characterized in that: The matching module is specifically used for: If the first initial area range and the second initial area range only include one distance range, corresponding matching is performed on areas with the same range value in the first initial area range and the second initial area range, and the matching results are screened and eliminated using the coplanar assumption RANSAC method to obtain an initial matching result; If the first initial area range and the second initial area range contain multiple distance ranges, corresponding matching is performed on multiple areas with the same range values ​​in the first initial area range and the second initial area range to obtain multiple matching results, and each matching result is screened and eliminated using the coplanar assumption RANSAC method to obtain multiple groups of matching point pairs, and all matching point pairs are collected and screened and eliminated again using the coplanar assumption RANSAC method to obtain initial matching results.

8. The cross-station rover visual positioning system according to claim 6, characterized in that: The projection module is specifically used for: Determine a first distance value between the initial matching result and the shooting point in the first image; Determine a second distance value between the initial matching result and the shooting point in the second image; When the first distance value is less than the second distance value, the first image is projected to the perspective of the second image through the homography matrix corresponding to the initial matching result to generate a third image; When the first distance value is greater than the second distance value, the second image is projected to the perspective of the first image through the homography matrix corresponding to the initial matching result to generate a third image.

9. A computer device, characterized in that: The computer device comprises a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements the method according to any one of claims 1 to 5.

Images