A method, system, device and medium for matching planetary surface images at different viewing angles

Abstract

The present invention discloses a planet surface image matching method, system, device and medium of different viewing angles, which relates to the technical field of visual image matching. The method comprises: using a sliding window to segment the navigation camera image before the planetary rover moves and the navigation camera image after the move; performing pairwise matching on the segmented areas in the image before the move and the image after the move, assuming that the to-be-matched area is the same area in the scene during matching, using the position information of the to-be-matched area in each image, projecting the to-be-matched area to the same viewing angle for matching, and obtaining an initial matching result; using the initial matching result to perform fine projection matching on the image before the move and the image after the move, and obtaining a valid matching point pair of the target area. This scheme greatly reduces the difficulty of feature matching.

Description

Background Art

[0002] Feature matching is one of the key steps for the rover to perform visual navigation and three-dimensional reconstruction on the surface of the planet. The quality of feature matching has a decisive influence on the accuracy of visual positioning and reconstruction. Due to the limited energy and communication conditions of the rover, in order to achieve higher mobility efficiency, the shooting locations of the rover's navigation camera before and after movement are usually far apart, which leads to a large difference in perspective between the front and rear station images, making image feature matching difficult.

[0003] Affine invariant feature matching has good performance in dealing with images with large viewing angle differences. By performing transformations such as tilting and rotating the original image at multiple angles, the changes in the shape of image features due to viewing angle differences are reduced, thereby achieving better matching. However, affine invariant feature matching uses a brute force approach to estimate the affine transformation relationship between images. It is necessary to consider a large number of affine transformations with different parameters for the same feature, resulting in low matching efficiency and inaccurate sampled affine transformations. In addition, affine invariant feature matching is not very adaptable to projection transformations, and the matching success rate is not high when dealing with images with large viewing angle differences on the surface of a planet. Summary of the invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the prior art, specifically the problems of low matching success rate and poor adaptability, and specifically provide a method, system, device and medium for matching planet surface images from different viewing angles, as follows:

[0005] 1) In the first aspect, the present invention provides a method for matching planet surface images at different viewing angles, and the specific technical solution is as follows:

[0006] S1, obtaining a first image taken by a rover equipped with a shooting device for a target area before the rover moves and a second image taken for the target area after the rover moves, performing binocular stereo matching on the first image and the second image respectively to obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, segmenting the first image and the second image according to a preset segmentation rule to obtain a first region sequence corresponding to the first image and a second region sequence corresponding to the second image, and presetting a first desired region and a second desired region;

[0007] S2, select a second area from the second area sequence, and match it with all the first areas. When matching, assume that the area to be matched is an overlapping area. Use the position information of the area to be matched in the first image and the second image to project the first image or the second image to the same viewing angle as the other image to match the corresponding area. If all matching instances of the selected second area are unsuccessful, select another second area and match it with all the first areas again until a successful matching instance is obtained, thereby obtaining an initial matching point pair and an initial homography matrix corresponding to the initial matching point pair.

[0008] S3, projecting the first image or the second image to the same viewing angle as the other image according to the position information of the initial matching point pair and the initial homography matrix, and performing matching in the first expected area and the second expected area to obtain a valid matching result;

[0009] The beneficial effects of the different-viewing angle planet surface image matching method provided by the present invention are as follows:

[0010] Taking full advantage of the fact that the driving area of ​​the planetary rover is relatively flat locally on the planetary surface, the overlapping area is searched in the image to be matched along the terrain surface manifold. The spatial position information of the local area is used to project the area to be matched to the same viewing angle, which greatly reduces the difficulty of feature matching. At the same time, it narrows the search range in the feature transformation space and reduces the matching complexity, making it easier to obtain the correct matching point pairs.

[0011] Based on the above solution, the present invention can also be improved as follows.

[0012] Furthermore, the preset segmentation rule is:

[0013] Image segmentation is performed by sliding the window according to the preset step size.

[0014] Further, a second region is selected from the second region sequence and matched with all the first regions. When matching, it is assumed that the region to be matched is an overlapping region. The position information of the region to be matched in the first image and the second image is used to project the first image or the second image to the same viewing angle as the other image to match the corresponding region. Specifically, the following steps are performed:

[0015] Approximate the to-be-matched region as a plane, and calculate the center position and normal vector of the current first region and the second region respectively according to the first sparse depth information and the second sparse depth information;

[0016] According to the center positions and normal vectors of the first area and the second area, respectively, a homography matrix for transforming the first image from an original viewing angle of the first area to a bird's-eye view perpendicular to the center of the first area and a homography matrix for transforming the second image from an original viewing angle of the second area to a bird's-eye view perpendicular to the center of the second area are calculated;

[0017] A rotation matrix of a preset simulation angle is used to represent a transformation matrix for transforming a top-down perspective perpendicular to the center of the second area to a top-down perspective perpendicular to the center of the first area;

[0018] Assuming that the area to be matched is the same area in the scene, the homography matrix from the original perspective of the second area to the original perspective of the first area is calculated according to the homography matrix from the original perspective of the first area to the overhead perspective perpendicular to the center of the first area, the homography matrix from the original perspective of the second area to the overhead perspective perpendicular to the center of the second area, and the transformation matrix from the overhead perspective of the center of the second area to the overhead perspective perpendicular to the center of the first area;

[0019] Compare the distance between the center position of the first area and the shooting point with the distance between the center position of the second area and the shooting point. If the distance between the center position of the first area and the shooting point is greater than the distance between the center position of the second area and the shooting point, project the second image to the perspective of the first image using a homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and perform feature matching of corresponding areas on the first image and the third image;

[0020] If the distance between the center position of the first area and the shooting point is smaller than the distance between the center position of the second area and the shooting point, the first image is projected to the perspective of the second image using the homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and feature matching of corresponding areas is performed on the third image and the second image.

[0021] 2) In the second aspect, the present invention also provides a planet surface image matching system with different viewing angles, and the specific technical solution is as follows:

[0022] The acquisition module is used to: acquire a first image captured by a rover equipped with a shooting device for a target area before the rover moves and a second image captured for the target area after the rover moves, perform binocular stereo matching on the first image and the second image respectively to obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, segment the first image and the second image according to a preset segmentation rule to obtain a first region sequence corresponding to the first image and a second region sequence corresponding to the second image, and preset a first expected region and a second expected region;

[0023] The initial matching module is used to: select a second area from the second area sequence, match it with all the first areas, assume that the area to be matched is an overlapping area during matching, and use the position information of the area to be matched in the first image and the second image to project the first image or the second image to the same viewing angle as the other image to match the corresponding area. If all matching instances of the selected second area are unsuccessful, select another second area and match it with all the first areas again until a successful matching instance is obtained, thereby obtaining an initial matching point pair and an initial homography matrix corresponding to the initial matching point pair;

[0024] The precise matching module is used to: project the first image or the second image to the same viewing angle as the other image according to the position information of the initial matching point pair and the initial homography matrix, and perform matching in the first expected area and the second expected area to obtain a valid matching result;

[0025] Based on the above solution, the present invention can also be improved as follows.

[0026] Furthermore, the preset segmentation rule is:

[0027] Image segmentation is performed by sliding the window according to the preset step size.

[0028] Further, a second region is selected from the second region sequence and matched with all the first regions. When matching, it is assumed that the region to be matched is an overlapping region. The position information of the region to be matched in the first image and the second image is used to project the first image or the second image to the same viewing angle as the other image to match the corresponding region. Specifically, the following steps are performed:

[0029] Approximate the to-be-matched region as a plane, and calculate the center position and normal vector of the current first region and the second region respectively according to the first sparse depth information and the second sparse depth information;

[0030] According to the center positions and normal vectors of the first area and the second area, respectively, a homography matrix for transforming the first image from an original viewing angle of the first area to a bird's-eye view perpendicular to the center of the first area and a homography matrix for transforming the second image from an original viewing angle of the second area to a bird's-eye view perpendicular to the center of the second area are calculated;

[0031] A rotation matrix of a preset simulation angle is used to represent a transformation matrix for transforming a top-down perspective perpendicular to the center of the second area to a top-down perspective perpendicular to the center of the first area;

[0032] Assuming that the area to be matched is the same area in the scene, the homography matrix from the original perspective of the second area to the original perspective of the first area is calculated according to the homography matrix from the original perspective of the first area to the overhead perspective perpendicular to the center of the first area, the homography matrix from the original perspective of the second area to the overhead perspective perpendicular to the center of the second area, and the transformation matrix from the overhead perspective of the center of the second area to the overhead perspective perpendicular to the center of the first area;

[0033] Compare the distance between the center position of the first area and the shooting point with the distance between the center position of the second area and the shooting point. If the distance between the center position of the first area and the shooting point is greater than the distance between the center position of the second area and the shooting point, project the second image to the perspective of the first image using a homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and perform feature matching of corresponding areas on the first image and the third image;

[0034] If the distance between the center position of the first area and the shooting point is smaller than the distance between the center position of the second area and the shooting point, the first image is projected to the perspective of the second image using the homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and feature matching of corresponding areas is performed on the third image and the second image.

[0035] 3) In a third aspect, the present invention further provides a computer device, comprising a processor, wherein the processor is coupled to a memory, wherein the memory stores at least one computer program, and wherein the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above methods.

[0036] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above methods.

[0037] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0039] Figure 1 A schematic diagram of a flow chart of a method for matching planetary surface images at different viewing angles according to an embodiment of the present invention;

[0040] Figure 2 A structural framework diagram of a planet surface image matching system with different viewing angles according to an embodiment of the present invention;

[0041] Figure 3 A computer device structure framework diagram of a method for matching planetary surface images at different viewing angles according to an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of a real image of a method for matching planet surface images at different viewing angles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0044] like Figure 1 As shown, a planet surface image matching method with different viewing angles according to an embodiment of the present invention comprises the following steps:

[0045] S1, obtaining a first image taken by a rover equipped with a shooting device for a target area before the rover moves and a second image taken for the target area after the rover moves, performing binocular stereo matching on the first image and the second image respectively to obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, segmenting the first image and the second image according to a preset segmentation rule to obtain a first region sequence corresponding to the first image and a second region sequence corresponding to the second image, and presetting a first desired region and a second desired region;

[0046] S2, select a second area from the second area sequence, and match it with all the first areas. When matching, assume that the area to be matched is an overlapping area. Use the position information of the area to be matched in the first image and the second image to project the first image or the second image to the same viewing angle as the other image to match the corresponding area. If all matching instances of the selected second area are unsuccessful, select another second area and match it with all the first areas again until a successful matching instance is obtained, thereby obtaining an initial matching point pair and an initial homography matrix corresponding to the initial matching point pair.

[0047] S3, projecting the first image or the second image to the same viewing angle as the other image according to the position information of the initial matching point pair and the initial homography matrix, and performing matching in the first expected area and the second expected area to obtain a valid matching result;

[0048] The beneficial effects of the different-viewing angle planet surface image matching method provided by the present invention are as follows:

[0049] Taking full advantage of the fact that the driving area of ​​the planetary rover is relatively flat locally on the planetary surface, the overlapping area is searched in the image to be matched along the terrain surface manifold. The spatial position information of the local area is used to project the area to be matched to the same viewing angle, which greatly reduces the difficulty of feature matching. At the same time, it narrows the search range in the feature transformation space and reduces the matching complexity, making it easier to obtain the correct matching point pairs.

[0050] like Figure 4 As shown, S1, obtain a first image taken by a rover equipped with a shooting device for a target area before moving and a second image taken for the target area after moving, perform binocular stereo matching on the first image and the second image respectively, obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, segment the first image and the second image according to a preset segmentation rule, obtain a first region sequence corresponding to the first image and a second region sequence corresponding to the second image, and preset a first expected region and a second expected region. Wherein:

[0051] The photographing device may be a camera;

[0052] The movement direction includes but is not limited to: forward and backward.

[0053] Segmenting the first image and the second image according to the preset segmentation rule is specifically as follows:

[0054] The sparse depth information represents the distance between the image feature and the shooting point along the optical axis;

[0055] For the first image and the second image, a first region sequence and a second region sequence are determined respectively by using a sliding window method;

[0056] Among them, the farthest matching area and other matching areas can be divided in the first image and the second image. The farthest matching area needs to be determined based on the farthest perception range of the shooting device and the distance moved by the rover. The first sparse depth information and the second sparse depth information are used to determine the matching area according to the distance range.

[0057] For example: In the first image: Set the distance range of the farthest matching area to [d rp(1) ,d rp(1) -l p ], where l p represents the length of the matching area in the first image along the optical axis, d rp(1) Indicates the farthest position that the camera can reach in the first image;

[0058] Set the distance range of other matching areas to [d rp(i) ,d rp(i)-l p ],(i=2,…,n),d rp(i) =d rp(i-1) -Δd p , Δd p is the step size of the first sliding window;

[0059] In the second image: Set the distance range of the farthest matching area to [d rc(1) ,d rc(1) -l c ],l c represents the length of the matching area in the second image along the optical axis, d rc(1) Indicates the farthest position that the camera can reach in the second image;

[0060] Set the distance range of other matching areas to [d rc(i) ,d rc(i) -l c ],(j=2,…,m),d rc(j) =d rc(j-1) -Δd c , Δd c is the step size of the second sliding window.

[0061] After segmentation, the purpose of matching any region in the first image with any region in the second image can be achieved.

[0062] In this solution, in order to balance the matching efficiency and matching quality, one of the first sliding window step size and the second sliding window step size can be set to a larger value and the other to a smaller value. The smaller the step size, the higher the overlap of adjacent matching areas and the more refined the area search. rp(1) and d rc(1) Both are 30 meters, l p is 10 meters, Δd p is 3 meters, and l c is 7 meters, Δd c is 7 meters.

[0063] S2, select a second area from the second area sequence and match it with all the first areas. When matching, assume that the area to be matched is an overlapping area. Use the position information of the area to be matched in the first image and the second image to project the first image or the second image to the same viewing angle as the other image to match the corresponding area. If all matching instances of the selected second area are unsuccessful, select another second area and match it with all the first areas again until a successful matching instance is obtained, and obtain the initial matching point pair and the initial homography matrix corresponding to the initial matching point pair. Where:

[0064] The overlapping area is the same area, and the same area means that in the entire space environment, or in the space environment where the rover is located, the first area can overlap with the second area after being enlarged or reduced.

[0065] Using the position information of the area to be matched in each image, the first image or the second image is projected to the same viewing angle as the other image to match the corresponding area, and the following processing scheme is used for auxiliary operation:

[0066] The geometric center of the region is taken as the region center, and the sparse depth information is used to obtain the three-dimensional coordinates of the stereo matching features in the first region and the second region relative to the shooting point. The region is fitted into a plane according to the three-dimensional coordinates of all features in the region, and the normal vectors of the first region and the second region are calculated;

[0067] According to the region normal vector and the center position of the region, the homography matrix of the original perspective transformed to the top-down perspective perpendicular to the center of the region is calculated to obtain the homography matrix H corresponding to the first region. rp(i)_ortho The homography matrix H corresponding to the second region rc(j)_ortho .

[0068] The calculation process is as follows: arbitrarily set four auxiliary points in the fitting plane of the region, calculate the projection coordinates of the four auxiliary points in the image under the original viewing angle and the projection coordinates under the overhead viewing angle perpendicular to the center of the region, and use the projection coordinates of the auxiliary points under the two viewing angles to solve the homography matrix transformed from the original viewing angle to the overhead viewing angle perpendicular to the center of the region.

[0069] To simulate the difference in viewing angle between the first and second regions in the original image, the transformation matrix corresponding to the rotation operation with an angle of θ around the center of the image is R θ , using R θ represents a transformation matrix for transforming a bird's-eye view perpendicular to the center of the second region to a bird's-eye view perpendicular to the center of the first region, where θ is a value of a preset set, for example, θ∈[-6,0,6].

[0070] Take the path of transforming the original perspective of the second area to the top-down perspective perpendicular to the center of the second area, transforming the top-down perspective perpendicular to the center of the second area to the top-down perspective perpendicular to the center of the first area, and then transforming the top-down perspective perpendicular to the center of the first area to the original perspective of the first area, and calculate the homography matrix H corresponding to the transformation from the original perspective of the second area to the original perspective of the first area rcp(i,j,θ) :

[0071] H rcp(i,j,θ) =(H rp(i)_ortho ) -1 R θ H rc(j)_ortho

[0072] If the distance between the center of the first region and the shooting point is greater than the distance between the center of the second region and the shooting point, use H rcp(i,j,θ) Transforming the second image to the same viewing angle as the first image to obtain a third image;

[0073] If the distance between the center of the first region and the shooting point is smaller than the distance between the center of the second region and the shooting point, use H rcp(i,j,θ) Transforming the first image to the same viewing angle as the second image to obtain a third image;

[0074] A general feature extraction and description method (such as SIFT) is used to perform feature extraction and matching in the matching area of ​​the third image and the first or second image, and the coplanar assumption RANSAC method is used to screen and eliminate the matching results to obtain a set of matching point pairs (which may be wrong matches).

[0075] The current second area is matched with all the first areas to obtain multiple sets of matching point pairs. All the matching point pairs are merged, and the coplanar assumption RANSAC method is applied again to filter and eliminate the matching point pairs. If the number of matching point pairs after screening exceeds the preset threshold, the match is considered successful, otherwise the match is considered failed.

[0076] If the current second region fails to match, another second region is selected and matched with all first regions again until the match succeeds, and the initial matching point pair M is obtained. rinit and the initial homography matrix H rinit .

[0077] S3, projecting the first image or the second image to the same viewing angle as the other image according to the position information of the initial matching point pair and the initial homography matrix, and performing matching in the first expected area and the second expected area to obtain a valid matching result.

[0078] Projecting the first image or the second image to the same viewing angle as the other image, matching is performed within the first desired area and the second desired area to obtain effective matching results, specifically:

[0079] Compare the distance between the initial matching point pair and the shooting point in the first image and the distance between the initial matching point pair and the shooting point in the second image. If the distance between the initial matching point pair and the shooting point in the first image is greater than the distance between the initial matching point pair and the shooting point in the second image, use the initial homography matrix H rinit Projecting the second image to the same viewing angle as the first image to obtain a fourth image;

[0080] Similarly, if the distance between the initial matching point pair and the shooting point in the first image is smaller than the distance between the initial matching point pair and the shooting point in the second image, the initial homography matrix H is used. rinitProjecting the first image to the same viewing angle as the second image to obtain a fourth image;

[0081] The fourth image and the first image or the second image are subjected to feature extraction and matching in the first desired area and the second desired area, and the matching results are screened and eliminated using the coplanar assumption RANSAC method to obtain the first matching point pair M r_t and the second homography matrix H r_t , using the second homography matrix H r_t The first image or the second image is projected to the same viewing angle as the other image to obtain a fifth image, and the fifth image is matched and screened with the first image or the second image in the first desired area and the second desired area to obtain a second matching point pair M. r and the second homography matrix H r .

[0082] Finally, M rval ={M r_t ,M r} as a valid matching point pair.

[0083] Since the driving area of ​​the rover on the surface of the planet is usually relatively flat, this feature can be fully utilized to reduce the transformation space of the image features, thereby achieving more efficient feature matching. The present invention approximates the local area in the image as a plane, and uses the center position of the local terrain, the normal vector and the simulated perspective change to perform a projection transformation on the image to be matched, so that the images to be matched are under the same perspective. The initial matching point pairs are obtained by searching for the overlapping areas in the images to be matched using a sliding window along the terrain surface. The initial matching point pairs are then used to obtain the precise projection transformation relationship of the overlapping areas of the images to be matched, and the images are projected and matched again to obtain a larger number and higher quality matching point pairs. The advantages of the present invention are mainly reflected in:

[0084] (1) Stronger adaptability to projection transformation: The images to be matched are projected to the same viewing angle, and the images after projection transformation are used for matching, making it easier to match image features with large viewing angle differences.

[0085] (2) Higher matching efficiency: The position and normal vector information of the local terrain is fully utilized. It is assumed that the area to be matched only simulates the parameter of the camera yaw change. The overlapping area is searched along the terrain surface manifold. The number of feature comparisons is reduced, so that the correct matching point pairs can be found faster.

[0086] (3) Loose application conditions: It only requires that the local terrain is relatively flat, and has a certain adaptability to the situation where the terrain is slowly undulating. There is no restriction on the image shooting angle and shooting range. In theory, as long as there is an overlapping area in the images to be matched, matching can be achieved.

[0087] Furthermore, the preset segmentation rule is:

[0088] Image segmentation is performed by sliding the window according to the preset step size.

[0089] Further, a second region is selected from the second region sequence and matched with all the first regions. When matching, it is assumed that the regions to be matched are the same regions in the scene. The first image or the second image is projected to the same viewing angle as the other image using the position information of the regions to be matched in the respective images to match the corresponding regions. Specifically, the following steps are performed:

[0090] Approximate the to-be-matched region as a plane, and calculate the center position and normal vector of the current first region and the second region respectively according to the first sparse depth information and the second sparse depth information;

[0091] According to the center positions and normal vectors of the first area and the second area, respectively, a homography matrix for transforming the first image from an original viewing angle of the first area to a bird's-eye view perpendicular to the center of the first area and a homography matrix for transforming the second image from an original viewing angle of the second area to a bird's-eye view perpendicular to the center of the second area are calculated;

[0092] The rotation matrix of the preset simulation angle is used to represent the transformation matrix of the top view angle perpendicular to the center of the second area to the top view angle perpendicular to the center of the first area.

[0093] Assuming that the area to be matched is the same area in the scene, the homography matrix from the original perspective of the second area to the original perspective of the first area is calculated according to the homography matrix from the original perspective of the first area to the overhead perspective perpendicular to the center of the first area, the homography matrix from the original perspective of the second area to the overhead perspective perpendicular to the center of the second area, and the transformation matrix from the overhead perspective of the center of the second area to the overhead perspective perpendicular to the center of the first area.

[0094] Compare the distance between the center position of the first area and the shooting point with the distance between the center position of the second area and the shooting point. If the distance between the center position of the first area and the shooting point is greater than the distance between the center position of the second area and the shooting point, project the second image to the perspective of the first image using a homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and perform feature matching of corresponding areas on the first image and the third image;

[0095] If the distance between the center position of the first area and the shooting point is smaller than the distance between the center position of the second area and the shooting point, the first image is projected to the perspective of the second image using the homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and feature matching of corresponding areas is performed on the third image and the second image.

[0096] like Figure 2As shown, the present invention also provides a planet surface image matching system 200 with different viewing angles, and the specific technical solution is as follows:

[0097] The acquisition module 210 is used to: acquire a first image captured by a rover equipped with a camera before the rover moves and a second image captured by the rover after the rover moves, perform binocular stereo matching on the first image and the second image respectively to obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, segment the first image and the second image according to a preset segmentation rule to obtain a first region sequence corresponding to the first image and a second region sequence corresponding to the second image, and preset a first desired region and a second desired region;

[0098] The initial matching module 220 is used to: select a second area from the second area sequence and match it with all the first areas. When matching, it is assumed that the area to be matched is the same area in the scene. The first image or the second image is projected to the same viewing angle as the other image using the position information of the area to be matched in each image to match the corresponding area. If all matching instances of the selected second area are unsuccessful, another second area is selected and matched with all the first areas again until a successful matching instance is obtained, thereby obtaining an initial matching point pair and an initial homography matrix corresponding to the initial matching point pair.

[0099] The precise matching module 230 is used to: project the first image or the second image to the same viewing angle as the other image according to the initial matching point pair position information and the initial homography matrix, and perform matching in the first expected area and the second expected area to obtain a valid matching result.

[0100] Furthermore, the preset segmentation rule is:

[0101] Image segmentation is performed by sliding the window according to the preset step size.

[0102] Further, a second region is selected from the second region sequence and matched with all the first regions. When matching, it is assumed that the regions to be matched are the same regions in the scene. The first image or the second image is projected to the same viewing angle as the other image using the position information of the regions to be matched in the respective images to match the corresponding regions. Specifically, the following steps are performed:

[0103] Approximate the to-be-matched region as a plane, and calculate the center position and normal vector of the current first region and the second region respectively according to the first sparse depth information and the second sparse depth information;

[0104] According to the center positions and normal vectors of the first area and the second area, the homography matrix for transforming the first image from the original viewing angle of the first area to the overhead viewing angle perpendicular to the center of the first area and the homography matrix for transforming the second image from the original viewing angle of the second area to the overhead viewing angle perpendicular to the center of the second area are respectively calculated.

[0105] The rotation matrix of the preset simulation angle is used to represent the transformation matrix of the top view angle perpendicular to the center of the second area to the top view angle perpendicular to the center of the first area.

[0106] Assuming that the area to be matched is the same area in the scene, the homography matrix from the original perspective of the second area to the original perspective of the first area is calculated according to the homography matrix from the original perspective of the first area to the overhead perspective perpendicular to the center of the first area, the homography matrix from the original perspective of the second area to the overhead perspective perpendicular to the center of the second area, and the transformation matrix from the overhead perspective of the center of the second area to the overhead perspective perpendicular to the center of the first area.

[0107] Compare the distance between the center position of the first area and the shooting point with the distance between the center position of the second area and the shooting point. If the distance between the center position of the first area and the shooting point is greater than the distance between the center position of the second area and the shooting point, project the second image to the perspective of the first image using a homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and perform feature matching of corresponding areas on the first image and the third image;

[0108] If the distance between the center position of the first area and the shooting point is smaller than the distance between the center position of the second area and the shooting point, the first image is projected to the perspective of the second image using the homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and feature matching of corresponding areas is performed on the third image and the second image.

[0109] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0110] It should be noted that the beneficial effects of the planet surface image matching system with different viewing angles provided in the above embodiment are the same as the beneficial effects of the planet surface image matching method with different viewing angles, which will not be described in detail here. In addition, when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be described in detail here.

[0111] like Figure 3 As shown, a computer device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the computer device 300 implements any of the above methods, specifically:

[0112] The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the computer device 300 implements a different-viewing angle planet surface image matching method provided in the above embodiment. Of course, the computer device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the computer device 300 may also include other components for implementing device functions, which will not be described in detail here.

[0113] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.

[0114] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0115] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above methods.

[0116] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0117] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0118] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0119] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for matching planetary surface images from different viewing angles, characterized in that: include: S1, obtaining a first image taken by a rover equipped with a shooting device for a target area before the rover moves and a second image taken for the target area after the rover moves, performing binocular stereo matching on the first image and the second image respectively to obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, segmenting the first image and the second image according to a preset segmentation rule to obtain a first region sequence corresponding to the first image and a second region sequence corresponding to the second image, and presetting a first desired region and a second desired region; S2, select a second area from the second area sequence, and match it with all the first areas. When matching, assume that the area to be matched is an overlapping area. Use the position information of the area to be matched in the first image and the second image to project the first image or the second image to the same viewing angle as the other image to match the corresponding area. If all matching instances of the selected second area are unsuccessful, select another second area and match it with all the first areas again until a successful matching instance is obtained, thereby obtaining an initial matching point pair and an initial homography matrix corresponding to the initial matching point pair. S3, projecting the first image or the second image to the same viewing angle as the other image according to the position information of the initial matching point pair and the initial homography matrix, and performing matching in the first expected area and the second expected area to obtain a valid matching result; Select one second region from the second region sequence and match it with all the first regions. When matching, assume that the regions to be matched are overlapping regions. Use the position information of the regions to be matched in the first image and the second image to project the first image or the second image to the same viewing angle as the other image to match the corresponding regions. Specifically, the following steps are performed: Approximate the area to be matched as a plane, and calculate the center position and normal vector of the current first area and the second area respectively according to the first sparse depth information and the second sparse depth information; According to the center positions and normal vectors of the first area and the second area, respectively, a homography matrix for transforming the first image from an original viewing angle of the first area to a bird's-eye view perpendicular to the center of the first area and a homography matrix for transforming the second image from an original viewing angle of the second area to a bird's-eye view perpendicular to the center of the second area are calculated; A rotation matrix of a preset simulation angle is used to represent a transformation matrix for transforming a top-down perspective perpendicular to the center of the second area to a top-down perspective perpendicular to the center of the first area; Assuming that the area to be matched is the same area in the scene, the homography matrix from the original perspective of the second area to the original perspective of the first area is calculated according to the homography matrix from the original perspective of the first area to the overhead perspective perpendicular to the center of the first area, the homography matrix from the original perspective of the second area to the overhead perspective perpendicular to the center of the second area, and the transformation matrix from the overhead perspective of the center of the second area to the overhead perspective perpendicular to the center of the first area; Compare the distance between the center position of the first area and the shooting point with the distance between the center position of the second area and the shooting point. If the distance between the center position of the first area and the shooting point is greater than the distance between the center position of the second area and the shooting point, project the second image to the perspective of the first image using a homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and perform feature matching of corresponding areas on the first image and the third image; If the distance between the center position of the first area and the shooting point is smaller than the distance between the center position of the second area and the shooting point, the first image is projected to the perspective of the second image using the homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and feature matching of corresponding areas is performed on the third image and the second image.

2. The method for matching planetary surface images at different viewing angles according to claim 1, characterized in that: The preset segmentation rule is: Image segmentation is performed by sliding the window according to the preset step size.

3. A planet surface image matching system with different viewing angles, characterized in that: include: The acquisition module is used to: acquire a first image captured by a rover equipped with a shooting device for a target area before the rover moves and a second image captured for the target area after the rover moves, perform binocular stereo matching on the first image and the second image respectively to obtain first sparse depth information corresponding to the first image and second sparse depth information corresponding to the second image, segment the first image and the second image according to a preset segmentation rule to obtain a first region sequence corresponding to the first image and a second region sequence corresponding to the second image, and preset a first expected region and a second expected region; The initial matching module is used to: select a second area from the second area sequence, assume that the area to be matched is an overlapping area when matching with all the first areas, use the position information of the area to be matched in the first image and the second image, project the first image or the second image to the same viewing angle as the other image to match the corresponding area, if all matching instances of the selected second area are unsuccessful, select another second area, match with all the first areas again, until a successful matching instance is obtained, and obtain an initial matching point pair and an initial homography matrix corresponding to the initial matching point pair; The precise matching module is used to: project the first image or the second image to the same viewing angle as the other image according to the position information of the initial matching point pair and the initial homography matrix, and perform matching in the first expected area and the second expected area to obtain a valid matching result; Select one second region from the second region sequence and match it with all the first regions. When matching, assume that the regions to be matched are overlapping regions. Use the position information of the regions to be matched in the first image and the second image to project the first image or the second image to the same viewing angle as the other image to match the corresponding regions. Specifically, the following steps are performed: Approximate the area to be matched as a plane, and calculate the center position and normal vector of the current first area and the second area respectively according to the first sparse depth information and the second sparse depth information; According to the center positions and normal vectors of the first area and the second area, respectively, a homography matrix for transforming the first image from an original viewing angle of the first area to a bird's-eye view perpendicular to the center of the first area and a homography matrix for transforming the second image from an original viewing angle of the second area to a bird's-eye view perpendicular to the center of the second area are calculated; A rotation matrix of a preset simulation angle is used to represent a transformation matrix for transforming a top-down perspective perpendicular to the center of the second area to a top-down perspective perpendicular to the center of the first area; Assuming that the area to be matched is the same area in the scene, the homography matrix from the original perspective of the second area to the original perspective of the first area is calculated according to the homography matrix from the original perspective of the first area to the overhead perspective perpendicular to the center of the first area, the homography matrix from the original perspective of the second area to the overhead perspective perpendicular to the center of the second area, and the transformation matrix from the overhead perspective of the center of the second area to the overhead perspective perpendicular to the center of the first area; Compare the distance between the center position of the first area and the shooting point with the distance between the center position of the second area and the shooting point. If the distance between the center position of the first area and the shooting point is greater than the distance between the center position of the second area and the shooting point, project the second image to the perspective of the first image using a homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and perform feature matching of corresponding areas on the first image and the third image; If the distance between the center position of the first area and the shooting point is smaller than the distance between the center position of the second area and the shooting point, the first image is projected to the perspective of the second image using the homography matrix that transforms the original perspective of the second area to the original perspective of the first area to obtain a third image, and feature matching of corresponding areas is performed on the third image and the second image.

4. The planet surface image matching system with different viewing angles according to claim 3, characterized in that: The preset segmentation rule is: Image segmentation is performed by sliding the window according to the preset step size.

5. A computer device, characterized in that: The computer device comprises a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements the method according to claim 1 or 2.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that the computer implements the method according to claim 1 or 2.

Images