Image geographic positioning method and system based on street view image super-radius cube projection three-dimensional reconstruction

Through the three-dimensional reconstruction method based on the super-radius cube projection of street scene images, the problem of low matching accuracy and difficult to take into account in image geolocation is solved, and higher image retrieval accuracy and positioning accuracy are achieved, which is suitable for images in complex scenarios Geographical positioning.

CN118918185APending Publication Date: 2024-11-08Chinese People's Liberation Army Cyberspace Force Information Engineering University
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410960306.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When the coordinates of the subject's geographic coordinates and the coordinates of the camera station are far away, the positioning error is large, and the matching accuracy is not high in scenarios such as seasonal lighting changes and large perspective differences, so it is impossible to take into account the image posture relationship and overlap rate. .

Method used

Using a three-dimensional reconstruction method based on superradius cube projection in street scene images, the distorted street scene images are corrected through superradius cube projection, serialized perspective images are generated, reference image database and feature point cloud database are established, and feature description vectors are extracted using SuperGlue. RANSAC The algorithm eliminates the wrong matching points, performs three-dimensional reconstruction by adjusting the beam method, calculates the matching score based on point density information, and obtains the image shooting position through the rear intersection solution in the space.

Benefits of technology

It improves image retrieval accuracy and positioning accuracy, enhances feature matching accuracy in complex scenarios, weakens the impact of noise on position accuracy, and improves the application prospects of image positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918185A_ABST
    Figure CN118918185A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image geographic positioning, in particular to an image geographic positioning method and system based on street view image super-radius cube projection three-dimensional reconstruction, and the method comprises the steps: correcting a distorted street view image through super-radius cube projection, and obtaining a serialized perspective image; establishing a reference image database by using the serialized perspective image; performing three-dimensional reconstruction on the image matching points in the reference image database to generate three-dimensional point clouds, and grouping the three-dimensional point clouds according to the streetscape images to establish a feature point cloud database; calculating matching scores of the query image and three-dimensional point clouds in the feature point cloud database based on the point density information of the image matching points, and selecting a group of point clouds with the highest matching score as candidate point clouds; and taking the three-dimensional coordinates of the candidate point clouds as the three-dimensional coordinates of the image points of the query image, and obtaining the shooting position of the query image through space resection solution. According to the method, higher image retrieval accuracy and positioning precision can be obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image geolocation technology, and in particular to an image geolocation method and system based on three-dimensional reconstruction of street view images using super-radius cube projection. Background Art

[0002] Image geolocation is used to determine the coordinates of the image shooting station. Currently, scholars at home and abroad mainly use two methods for accurate image positioning: image retrieval positioning and 2D-3D matching positioning. Among them, the image retrieval positioning method uses images containing geographic coordinates as the reference image database, and locates the query image by searching the image with the image. This type of method describes the image by extracting local features, global features, or a combination of the two features. After retrieving the matching image using the similarity measure between images, the geographic location of the matching image is used as the positioning result. This method has high image retrieval efficiency and stable accuracy, but its positioning result depends on the similarity of the objects in the query image and the street view image. There are cases where the objects in the two images are similar but the distance between the shooting stations is far. This situation usually occurs in images where the coordinates of the main object in the image are far away from the coordinates of the shooting station, which can be called images with a long shooting distance. See Figure 2 As shown in the figure, (b) and (c) are two query images of the same object. The shooting distance of (c) is farther, and the coordinates of the two shooting stations are quite different. Even if the correct reference image (a) is retrieved based on the same object, using the coordinates of (a) as the coordinates of the two query images at the same time will obviously cause a large positioning error.

[0003] To address this shortcoming, some researchers have converted image geolocation into a 2D-3D matching problem, established a correspondence between 2D image points and 3D ground points, and solved the location of the image camera station through the geometric relationship between the two. For example, Liu et al. matched the query image with a large-scale 3D point cloud map and solved the image shooting location through the P4P method; Zeisl et al. proposed a voting-based posture estimation strategy, which filtered out incorrect matches through the geometric relationship of matching points and solved the problem of outlier matching in large-scale geolocation; Yuan Yi et al. used the query image and the reference image for 3D reconstruction to solve the location of the camera station of the query image. This type of method can effectively deal with the challenges of image geolocation caused by similar objects in the image and the long distance between the camera stations, but existing studies have used traditional manually designed descriptors for feature extraction and matching, which can only extract and represent low-level features of the image, and the matching accuracy is not high in scenes such as seasonal lighting changes and large viewing angle differences. Summary of the invention

[0004] To this end, the present invention provides an image geolocation method and system based on three-dimensional reconstruction of street view images with super-radius cube projection, which solves the problems in existing image geolocation, such as low matching in Kyoto and failure to take into account both image posture relationship and overlap rate.

[0005] According to the design scheme provided by the present invention, on the one hand, an image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection is provided, comprising:

[0006] Obtain street view images, correct the distorted street view images using hyperradius cube projection to obtain serialized perspective images, and establish a reference image database using the serialized perspective images;

[0007] Perform three-dimensional reconstruction on the image matching points in the reference image database to generate a three-dimensional point cloud, and group the three-dimensional point cloud according to the street view image to establish a feature point cloud database;

[0008] Based on the point density information of the image matching points, the matching scores between the query image and the three-dimensional point cloud in the feature point cloud database are calculated, and a group of point clouds with the highest matching scores are selected as candidate point clouds;

[0009] The 3D coordinates of the candidate point cloud are used as the 3D coordinates of the query image points, and the shooting position of the query image is obtained through spatial resection solution.

[0010] As an image geolocation method based on three-dimensional reconstruction of street view images by hyperradius cube projection of the present invention, further, the distorted street view images are corrected by using hyperradius cube projection, comprising:

[0011] The planar street view image is converted into a spherical panoramic image, the spherical panoramic image is projected into a designated space, and the spherical polar coordinates are converted into spatial rectangular coordinates based on a coordinate conversion rule. The designated space is a cubic surface tangent to the sphere and having a side length greater than the diameter of the panoramic sphere.

[0012] As an image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection of the present invention, further, the spherical panoramic image is projected to a designated space, comprising:

[0013] Projecting the pixel plane of the panoramic image onto the surface of the cube according to the geometric relationship between the pixel plane and the surface of the cube;

[0014] Set the proportional relationship between the cube side length and the panoramic sphere radius, and project the points on the cube surface to the panoramic sphere according to the geometric relationship between the cube surface and the panoramic sphere surface.

[0015] As the image geolocation method based on the three-dimensional reconstruction of the street view image by the hyperradius cube projection of the present invention, further, the three-dimensional reconstruction of the image matching points in the reference image database comprises:

[0016] The feature matching network SuperGlue is used to extract the same-name image points in the adjacent camera station images in the benchmark image database and save the feature description vectors of the same-name image points;

[0017] The RANSAC algorithm is used to eliminate mismatched points among the same-name image points, and the camera pose matrix is ​​obtained based on the relationship between the street view image's own Euler angle and the perspective image pose.

[0018] The bundle adjustment is used to perform three-dimensional reconstruction, and the three-dimensional coordinates of the image points corresponding to the ground points are obtained, and the feature vectors and three-dimensional coordinates of the image points with the same name are saved.

[0019] As an image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection of the present invention, further, a matching score between a query image and a three-dimensional point cloud in a feature point cloud database is calculated based on point density information of image matching points, including:

[0020] The image matching point density is obtained according to the number of image matching points and the area of ​​the closed polygon formed by the image matching points;

[0021] The matching scores of the query image and the three-dimensional point cloud in the feature point cloud database are obtained based on the matching point density, so as to screen the image matching points according to the matching point scores and select a group of point clouds with the highest matching point scores as candidate point clouds for the query image feature points.

[0022] As an image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection of the present invention, further, obtaining the shooting position of the query image by spatial resection solution includes:

[0023] Based on the center consistency principle, the candidate point cloud of the query image is noise filtered and the key point cloud is obtained;

[0024] Based on the key point cloud and through spatial resection solution, the shooting position of the query image is obtained.

[0025] As an image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection of the present invention, further, noise filtering is performed on the candidate point cloud of the query image based on the center consistency principle, comprising:

[0026] Randomly select multiple pairs of matching points from the candidate point cloud, and obtain the photographic center deviation based on the image point, ground point and photographic center through cosine distance;

[0027] Based on the inlier threshold and using the photographic center deviation, it is determined whether an image point is an inlier, and the candidate point cloud is filtered according to the inlier ratio, so as to select the key point cloud that meets the coordinate accuracy requirements based on the filtering results.

[0028] In another aspect, the present invention further provides an image geolocation system based on three-dimensional reconstruction of street view images by super-radius cube projection, comprising: a reference acquisition module, a three-dimensional reconstruction module, a point cloud matching module and a geolocation module, wherein:

[0029] A reference acquisition module is used to acquire street view images, correct the distorted street view images using hyperradius cube projection, obtain serialized perspective images, and establish a reference image database using the serialized perspective images;

[0030] A 3D reconstruction module is used to perform 3D reconstruction on the image matching points in the reference image database to generate a 3D point cloud, and to group the 3D point cloud according to the street view image to establish a feature point cloud database;

[0031] A point cloud matching module is used to calculate the matching score between the query image and the three-dimensional point cloud in the feature point cloud database based on the point density information of the image matching points, and select a group of point clouds with the highest matching score as candidate point clouds;

[0032] The geolocation module is used to use the three-dimensional coordinates of the candidate point cloud as the three-dimensional coordinates of the query image point, and obtain the shooting position of the query image through spatial back intersection solution.

[0033] Beneficial effects of the present invention:

[0034] The present invention corrects the distorted street view image through super-radius cube projection, generates a perspective image with a certain degree of overlap and a fixed posture relationship, expands the field of view of the perspective image by increasing the side length of each face of the cube, improves the overlap between panoramic images of different camera stations, matches more image points with the same name, and obtains higher image retrieval accuracy and positioning accuracy; uses the depth features of the corrected street view image to generate a 3D feature point cloud database and aligns it with the query image, thereby improving the accuracy of query image feature matching in complex scenes; and introduces the query image matching point density information into the matching score of the 3D feature point cloud, in order to cope with the problem of a large number of mismatched points due to the complex content of the street view image, solves the position information of the query image based on the central consistency principle through secondary adjustment, further reduces the influence of noise on the position accuracy of the rear intersection image, and has good application prospects in the field of image positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of the image geolocation process based on 3D reconstruction of street view images with super-radius cube projection in an embodiment;

[0036] Figure 2 This is an example of images taken of the same ground object but at different shooting station distances in the embodiment;

[0037] Figure 3 This is a schematic diagram of the hyperradius cube projection process in the embodiment;

[0038] Figure 4 Schematic diagram of the geometric relationship between the planar panoramic image and the spherical panoramic image in the embodiment;

[0039] Figure 5 This is a schematic diagram of the hyperradius cube projection in the embodiment;

[0040] Figure 6 Schematic diagram of the geometric relationship between the pixel plane and the cube surface in the embodiment;

[0041] Figure 7 Schematic diagram of the geometric relationship between the cube surface and the panoramic spherical surface in the embodiment;

[0042] Figure 8 This is a schematic diagram of the super radius cube projection effect in the embodiment;

[0043] Fig. 9 This is a schematic diagram of the central consistent geometric relationship in the embodiment;

[0044] Fig.10 It is a schematic diagram of the image position effect of spatial resection solution in the embodiment;

[0045] Fig.11 This is a schematic diagram of the super radius cube projection effect in the embodiment;

[0046] Fig.12 It is a schematic diagram of the three-dimensional reconstruction effect of the feature point cloud in the embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and technical solutions.

[0048] Street view images have the advantages of 360° field of view, location information of camera stations, and wide database distribution, making them a commonly used benchmark dataset for image geolocation. However, street view images are usually stored in equidistant cylindrical projection, while query images are mainly perspective projection images. Therefore, it is difficult to correctly align the same feature in street view images and query images. To this end, street view images are divided into blocks and cropped so that each block has less distortion, so as to perform image geolocation. In essence, distorted images are still used. In addition, street view images can also be projected onto the six faces of a cube to eliminate the distortion of street view images. However, after projection, the overlap of the images on each face of the cube is 0, and the wide viewing angle characteristics of street view images cannot be well utilized. To address this problem, sundial projection can be used to set the overlap rate to 50% to correct the buildings in blocks. Although this correction method has a high overlap rate, the posture angle of the reprojected image is difficult to correctly represent, and the posture information of the street view image is not used in visual positioning. Therefore, in order to solve the problem that the street view image and the perspective image are difficult to match due to different projection methods, the panoramic image is converted into a perspective image by projecting the spherical panoramic image onto a cube tangent to the sphere. At this time, the side length of the cube is equal to the diameter of the panoramic sphere, and the horizontal field of view of the perspective image is only 90°. The large viewing angle characteristic of the panoramic image is not fully utilized, which affects the 3D reconstruction effect and the query image retrieval accuracy. In the embodiments of the present invention, see Figure 1 As shown, an image geolocation method based on three-dimensional reconstruction of street view images with super-radius cube projection is provided, comprising:

[0049] S101, acquiring a street view image, correcting the distorted street view image using a hyperradius cube projection to obtain a serialized perspective image, and establishing a reference image database using the serialized perspective image.

[0050] Specifically, the distorted street view image is corrected using the hyperradius cube projection, which can be designed to include:

[0051] The planar street view image is converted into a spherical panoramic image, the spherical panoramic image is projected into a designated space, and the spherical polar coordinates are converted into spatial rectangular coordinates based on a coordinate conversion rule. The designated space is a cubic surface tangent to the sphere and having a side length greater than the diameter of the panoramic sphere.

[0052] The projecting of the spherical panoramic image to a specified space may include:

[0053] Projecting the pixel plane of the panoramic image onto the surface of the cube according to the geometric relationship between the pixel plane and the surface of the cube;

[0054] Set the proportional relationship between the cube side length and the panoramic sphere radius, and project the points on the cube surface to the panoramic sphere according to the geometric relationship between the cube surface and the panoramic sphere surface.

[0055] like Figure 3 As shown in , the planar panoramic image is first converted into a spherical panoramic image, and the image is projected into a three-dimensional space. Figure 4 As shown, O-XYZ is the camera coordinate system, O is the photography center (the center of the sphere), o-uv is the plane coordinate system, P O is a certain image point on the plane panoramic image, P S is the corresponding image point of the spherical panoramic image, α is OP S The angle between the projection on the XOY plane and OX, β is OP S The angle with OZ, like point P O The conversion formula between spherical polar coordinates (α, β, γ) and plane coordinates (u, v) is:

[0056]

[0057] Where γ is the radius of the sphere, u is the horizontal coordinate of the image point on the plane panoramic image, v is the vertical coordinate, W is the width of the film, and H is the height (all in pixels).

[0058] Then, the spherical polar coordinates are converted to spatial rectangular coordinates using formula (2).

[0059]

[0060] like Figure 5 As shown, different from the traditional cube projection (cube surface formed by dotted lines), the hyperradius cube projection projects the spherical panoramic image to a square surface (square surface formed by solid lines) that is tangent to the sphere but has a side length greater than the diameter of the panoramic sphere. It can be divided into two projection processes: the pixel plane to the cube surface and the cube surface to the spherical surface.

[0061] by Y+ The projection process is introduced using the surface as an example. The principles and formulas for the projection of other surfaces are similar. Figure 6 As shown, in Y+ In the plane, O'-X'Z' is the cube surface coordinate system, which is the camera coordinate system O-XYZ in Y+ The projection of the surface, o-uv is the pixel plane coordinate system, and the conversion relationship between the two is:

[0062]

[0063] Where l is the side length of the Y+ plane.

[0064] Then project the points on the cube surface onto the panoramic sphere, as Figure 7 As shown, P W is the ground point, P S , P C PW Projection on the panoramic sphere and the cube surface, according to the principle of similar triangles, P C (X',R,Z') and P S The conversion relationship of (X,Y,Z) is:

[0065]

[0066] Where R is the radius of the panoramic sphere and s is the similarity scale factor.

[0067] By setting the proportional relationship between the cube side length and the panoramic sphere radius, the field of view of the cube projection is increased, and the overlap rate between the projected images is improved. The effect is as follows Figure 8 As shown, the area outside the red box is the enlarged area.

[0068] The spherical panoramic image can be projected onto the six faces of the cube, of which the Z+ face and the Z- face mainly contain invalid information such as the panoramic image shooting vehicle and the sky. Therefore, in the embodiment of this case, the spherical panoramic image can be projected onto only the remaining four faces during the panoramic image processing process.

[0069] S102, performing three-dimensional reconstruction on the image matching points in the reference image database to generate a three-dimensional point cloud, and grouping the three-dimensional point cloud according to the street view image to establish a feature point cloud database.

[0070] Specifically, the three-dimensional reconstruction of the image matching points in the reference image database can be designed to include:

[0071] The feature matching network SuperGlue is used to extract the same-name image points in the adjacent camera station images in the benchmark image database and save the feature description vectors of the same-name image points;

[0072] The RANSAC algorithm is used to eliminate mismatched points among the same-name image points, and the camera pose matrix is ​​obtained based on the relationship between the street view image's own Euler angle and the perspective image pose.

[0073] The bundle adjustment is used to perform three-dimensional reconstruction, and the three-dimensional coordinates of the image points corresponding to the ground points are obtained, and the feature vectors and three-dimensional coordinates of the image points with the same name are saved.

[0074] In order to obtain the query image feature point cloud database, in the embodiment of this case, based on the serialized perspective image after hyperradius cube projection, SuperGlue is used to extract the same-name image points from the adjacent camera station images and save the feature description vectors of the same-name image points. Then, the mismatched points among the same-name image points are eliminated based on the RANSAC algorithm. Then, the camera pose matrix is ​​calculated using the posture relationship between the street view image's own Euler angle and the perspective image. Finally, three-dimensional reconstruction is performed through bundle adjustment to obtain the three-dimensional coordinates of the image points corresponding to the ground points.

[0075] In order to improve the retrieval efficiency of the query image, only the feature vectors and the three-dimensional coordinates of the image points with the same name in the world coordinate system can be saved, and the three-dimensional point cloud is evenly thinned according to the density. Considering that the field of view of the query image is usually much smaller than that of the panoramic image, in the embodiment of this case, the point cloud database is grouped based on the original panoramic image, that is, the feature point clouds corresponding to the image points on the same panoramic image are stored in the same file to establish a feature point cloud database, so as to improve the positioning efficiency and accuracy.

[0076] S103 . Calculate the matching scores between the query image and the three-dimensional point cloud in the feature point cloud database based on the point density information of the image matching points, and select a group of point clouds with the highest matching scores as candidate point clouds.

[0077] Specifically, the image matching point density can be obtained based on the number of image matching points and the area of ​​the closed polygon formed by the image matching points; the matching score between the query image and the three-dimensional point cloud in the feature point cloud database is obtained based on the matching point density, so that the image matching points are screened according to the matching point scores and a group of point clouds with the highest matching point scores are selected as candidate point clouds for the query image feature points.

[0078] In large-scale positioning retrieval of urban street scenes, images contain a large number of non-fixed targets such as pedestrians and vehicles, as well as objects without information such as the sky and the ground, which can easily cause mismatches. Therefore, it is necessary to filter the matching results of image points with the same name. In this embodiment, the confidence of the SuperGlue matching points can be used for filtering. The filtering function is shown in the following formula (5).

[0079]

[0080] Where V i Is the mark of whether the feature point i is correctly matched. If V i If it is 1, it means that the feature point is correctly matched and participates in the matching score calculation. Otherwise, it is a wrong match and does not participate in the calculation.

[0081] The matching score is a decisive factor in image matching. In order to make the correct matching result more distinguishable, in the embodiment of this case, a matching score calculation method based on the density of feature points is used. This is because the original panoramic image is large in size and has a large number of feature points, which may produce a large number of mismatched points, but these mismatched points will not appear in clusters on the image. At the same time, some scholars believe that correct matching points usually have the characteristics of neighboring points, and propose a matching algorithm for the same-name image points based on the co-occurrence matrix to improve the matching accuracy and efficiency. Therefore, the closed boundary line of the matching point can be extracted according to formula (6) and the density ρ of the matching point can be calculated.

[0082]

[0083] Where S is the area of ​​the closed polygon formed by the matching points.

[0084] Define the matching score based on the matching point density M The expression is:

[0085]

[0086] The set of point clouds with the highest matching scores is selected as the matching results of the query image feature points.

[0087] S104: Using the three-dimensional coordinates of the candidate point cloud as the three-dimensional coordinates of the query image points, and obtaining the shooting position of the query image through spatial resection solution.

[0088] Specifically, the shooting position of the query image is obtained by spatial resection solution, which can be designed to include:

[0089] Based on the center consistency principle, the candidate point cloud of the query image is noise filtered and the key point cloud is obtained;

[0090] Based on the key point cloud and through spatial resection solution, the shooting position of the query image is obtained.

[0091] Among them, multiple pairs of matching points can be randomly selected from the candidate point cloud, and the photographic center deviation can be obtained based on the image point, ground point and photographic center and through the cosine distance; based on the inlier threshold and using the photographic center deviation to determine whether the image point belongs to an inlier, and the candidate point cloud is filtered according to the proportion of inliers, so as to select the key point cloud that meets the coordinate accuracy requirements based on the filtering results.

[0092] The three-dimensional coordinates of the matching point cloud are used as the three-dimensional coordinates of the image points of the query image, and the shooting position of the query image is solved through spatial resection. However, in the process of calculating the three-dimensional coordinates of the point cloud, there are problems such as mismatching, large camera posture errors, and too large or too small intersection angles of the same-name image points, which result in poor accuracy of the three-dimensional coordinates of some point clouds and become noise. Since the result of solving the camera station position through spatial resection is the optimal solution for the adjustment of all the image points involved in the calculation, the noise has a great influence on the accuracy of solving the image position. In order to solve this problem, in the embodiment of this case, the RANSAC algorithm can be used to filter the noise based on the central consistency principle.

[0093] like Fig. 9 As shown, under ideal conditions, the photography center is O and the image point is P C , ground point P W There is a certain amount of offset δO between the photographic center O' and O calculated by adjustment. If the three-dimensional coordinate accuracy of the image point involved in solving O' is high, δO is small, otherwise δO is large. Therefore, the point cloud can be filtered according to the size of δO. This filtering principle can be called the center consistency principle. For the convenience of calculation, the cosine distance ∠OP CO' represents δO:

[0094]

[0095] Randomly select n (n>6) pairs of matching points to calculate the position of O' by spatial resection, and then determine the image point P by formula (9). C Is it an interior point?

[0096]

[0097] Where ε is the inlier threshold, which is 0.008.

[0098] Then, according to the proportion of internal points, iterative circular filtering is performed, and image points with high coordinate accuracy are selected to participate in the spatial resection to solve the image shooting position. The center consistency principle better solves the influence of noise on the position of the image shooting station. The effect before and after the center consistency processing is as follows: Fig.10 shown.

[0099] Furthermore, based on the above method, an embodiment of the present invention also provides an image geolocation system based on 3D reconstruction of street view images by super-radius cube projection, comprising: a reference acquisition module, a 3D reconstruction module, a point cloud matching module and a geolocation module, wherein:

[0100] A reference acquisition module is used to acquire street view images, correct the distorted street view images using hyperradius cube projection, obtain serialized perspective images, and establish a reference image database using the serialized perspective images;

[0101] A 3D reconstruction module is used to perform 3D reconstruction on the image matching points in the reference image database to generate a 3D point cloud, and to group the 3D point cloud according to the street view image to establish a feature point cloud database;

[0102] A point cloud matching module is used to calculate the matching score between the query image and the three-dimensional point cloud in the feature point cloud database based on the point density information of the image matching points, and select a group of point clouds with the highest matching score as candidate point clouds;

[0103] The geolocation module is used to use the three-dimensional coordinates of the candidate point cloud as the three-dimensional coordinates of the query image point, and obtain the shooting position of the query image through spatial back intersection solution.

[0104] Specifically, the distorted street view image is corrected by using the hyperradius cube projection according to a certain degree of overlap. Fig.11 As shown, one street view image can obtain four perspective images with fixed poses (excluding the cube Z+ Surface and Z- surface), establish a reference image database. The reference image matching points extracted by SuperPoint and SuperGlue are reconstructed in three dimensions through bundle adjustment to generate a feature point cloud with three-dimensional coordinates in the world coordinate system, such as Fig.12 As shown in the figure, the point cloud is grouped according to the original street view image, that is, the feature point cloud corresponding to the image points on the same panoramic image is stored in the same file to establish a feature point cloud database. The matching score is calculated based on the point density information of the query image and the 3D point cloud, and the group of point clouds with the highest matching score is used as the candidate point cloud result; the three-dimensional coordinates of the candidate point cloud are used as the three-dimensional coordinates of the query image image points, and the coordinates of the image shooting position are obtained by spatial rear intersection based on the center consistency principle.

[0105] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0106] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0107] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0108] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0109] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image geolocation method based on three-dimensional reconstruction of street view images using super-radius cube projection, characterized in that: Include: Obtain street view images, correct the distorted street view images using hyperradius cube projection to obtain serialized perspective images, and establish a reference image database using the serialized perspective images; Perform three-dimensional reconstruction on the image matching points in the reference image database to generate a three-dimensional point cloud, and group the three-dimensional point cloud according to the street view image to establish a feature point cloud database; Based on the point density information of the image matching points, the matching scores between the query image and the three-dimensional point cloud in the feature point cloud database are calculated, and a group of point clouds with the highest matching scores are selected as candidate point clouds; The 3D coordinates of the candidate point cloud are used as the 3D coordinates of the query image points, and the shooting position of the query image is obtained through spatial resection solution.

2. The image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection according to claim 1, characterized in that: The distorted street view image is corrected using the hypercube projection, including: The planar street view image is converted into a spherical panoramic image, the spherical panoramic image is projected into a designated space, and the spherical polar coordinates are converted into spatial rectangular coordinates based on a coordinate conversion rule. The designated space is a cubic surface tangent to the sphere and having a side length greater than the diameter of the panoramic sphere.

3. The image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection according to claim 2, characterized in that: Project the spherical panoramic image to the specified space, including: Projecting the pixel plane of the panoramic image onto the surface of the cube according to the geometric relationship between the pixel plane and the surface of the cube; Set the proportional relationship between the cube side length and the panoramic sphere radius, and project the points on the cube surface to the panoramic sphere according to the geometric relationship between the cube surface and the panoramic sphere surface.

4. The image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection according to claim 1, characterized in that: Perform 3D reconstruction of image matching points in the reference image database, including: The feature matching network SuperGlue is used to extract the same-name image points in the adjacent camera station images in the benchmark image database and save the feature description vectors of the same-name image points; The RANSAC algorithm is used to eliminate mismatched points among the same-name image points, and the camera pose matrix is ​​obtained based on the relationship between the street view image's own Euler angle and the perspective image pose. The bundle adjustment is used to perform three-dimensional reconstruction, and the three-dimensional coordinates of the image points corresponding to the ground points are obtained, and the feature vectors and three-dimensional coordinates of the image points with the same name are saved.

5. The image geolocation method based on three-dimensional reconstruction of street view images by super-radius cube projection according to claim 1, characterized in that: The matching score between the query image and the 3D point cloud in the feature point cloud database is calculated based on the point density information of the image matching points, including: The image matching point density is obtained according to the number of image matching points and the area of ​​the closed polygon formed by the image matching points; The matching scores of the query image and the three-dimensional point cloud in the feature point cloud database are obtained based on the matching point density, so as to screen the image matching points according to the matching point scores and select a group of point clouds with the highest matching point scores as candidate point clouds for the query image feature points.

6. The image geolocation method based on 3D reconstruction of street view images by super-radius cube projection according to claim 1, characterized in that: The shooting position of the query image is obtained through spatial resection solution, including: Based on the center consistency principle, the candidate point cloud of the query image is noise filtered and the key point cloud is obtained; Based on the key point cloud and through spatial resection solution, the shooting position of the query image is obtained.

7. The image geolocation method based on 3D reconstruction of street view images by super-radius cube projection according to claim 6, characterized in that: Based on the center consistency principle, the candidate point cloud of the query image is noise filtered, including: Randomly select multiple pairs of matching points from the candidate point cloud, and obtain the photographic center deviation based on the image point, ground point and photographic center through cosine distance; Based on the inlier threshold and using the photographic center deviation, it is determined whether an image point is an inlier, and the candidate point cloud is filtered according to the inlier ratio, so as to select the key point cloud that meets the coordinate accuracy requirements based on the filtering results.

8. An image geolocation system based on three-dimensional reconstruction of street view images by super-radius cube projection, characterized in that: It includes: benchmark acquisition module, 3D reconstruction module, point cloud matching module and geographic positioning module, among which, A reference acquisition module is used to acquire street view images, correct the distorted street view images using hyperradius cube projection, obtain serialized perspective images, and establish a reference image database using the serialized perspective images; A 3D reconstruction module is used to perform 3D reconstruction on the image matching points in the reference image database to generate a 3D point cloud, and to group the 3D point cloud according to the street view image to establish a feature point cloud database; A point cloud matching module is used to calculate the matching score between the query image and the three-dimensional point cloud in the feature point cloud database based on the point density information of the image matching points, and select a group of point clouds with the highest matching score as candidate point clouds; The geolocation module is used to use the three-dimensional coordinates of the candidate point cloud as the three-dimensional coordinates of the query image point, and obtain the shooting position of the query image through spatial back intersection solution.

9. An electronic device, characterized in that: include: at least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, and the computer program can be executed by the at least one processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 can be implemented.

Citation Information

Cited By

  • Geographic positioning method and device based on multi-modal large model, equipment and medium

    CN120256489A

  • Automatic relocation matching method, device and equipment for cross-period inspection panoramic image, medium and product

    CN122492829A