Gps-based visual positioning method, system, computer and storage medium

Abstract

The application provides a GPS-based visual positioning method, system, computer and storage medium, which comprises the following steps: acquiring the GPS coordinates of a query image, and screening reference images with close geographical positions from a database based on the GPS coordinates; screening reference images with a geographical distance less than or equal to a preset radius as a candidate set; determining several similar images based on k-nearest neighbor search; clustering according to the co-visibility relationship of each similar image to generate a plurality of locations corresponding to the candidate image, each location containing commonly observed 3D points; performing 2D-3D matching on each location, and estimating and outputting a six-degree-of-freedom camera pose through a perspective n-point algorithm and a random sample consensus algorithm to realize visual positioning. The GPS coordinates are used to dynamically screen reference images, the search time is reduced, the quick response requirement of real-time scene application is met, and the cross-environment high robustness is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual positioning, in particular to a GPS-based visual positioning method and system, a computer and a storage medium. BACKGROUND

[0002] Visual positioning is a core technology in the fields of computer vision and robot navigation, aiming to determine the 6-DoF pose of a camera in a known 3D model through images captured by the camera. This technology is widely used in the following scenarios: autonomous driving: achieving accurate navigation without GPS or weak GPS signals in urban environments. Mobile robots: supporting simultaneous localization and mapping (SLAM) and path planning. Augmented reality (AR): providing centimeter-level precision pose to achieve immersive experience.

[0003] In the prior art, the traditional method includes the HF-Net network architecture, which realizes the means of generating global descriptors and local features at the same time through a single CNN, first performs global retrieval to determine candidate positions, and then performs local matching to estimate the accurate pose. However, the global retrieval of HF-Net needs to traverse the entire database, and the computational complexity increases significantly with the increase of the database size, especially in urban scenarios, the efficiency is limited. SUMMARY

[0004] In view of the deficiencies of the prior art, the purpose of the present application is to provide a GPS-based visual positioning method, system, computer and storage medium, aiming to solve the technical problem of low efficiency of visual positioning in the prior art.

[0005] To achieve the above-mentioned purpose, in a first aspect, the present application provides a GPS-based visual positioning method, comprising the following steps:

[0006] Obtaining the GPS coordinates of the query image, and filtering the reference images close in geographical position from the database based on the GPS coordinates;

[0007] Calculating the geographical distance of the reference image according to the Haversine formula, and filtering the reference images with a distance less than or equal to a preset radius as a candidate set based on the geographical distance;

[0008] Extracting the global descriptor, local descriptor and key point score of the query image based on the HF-Net network architecture, performing k-nearest neighbor search in the candidate set according to the global descriptor to determine a plurality of similar images;

[0009] Clustering according to the co-visibility relationship of each similar image based on the SfM model, generating a plurality of locations corresponding to the candidate images, each location containing commonly observed 3D points;

[0010] 2D-3D matching is performed on each of the locations based on the local descriptor and the key point score, and a six-degree-of-freedom camera pose is estimated and output through a perspective-n-point algorithm and a random sample consensus algorithm, so as to realize visual positioning based on the six-degree-of-freedom camera pose.

[0011] According to an aspect of the above technical solution, before the step of realizing visual positioning based on the six-degree-of-freedom camera pose, the method further comprises:

[0012] determining whether the six-degree-of-freedom camera pose is valid, and if the six-degree-of-freedom camera pose is invalid, determining whether the number of candidate images in the candidate set is greater than a preset value;

[0013] if the number of candidate images is less than the preset value, the preset radius is expanded to re-screen and update the candidate set.

[0014] According to an aspect of the above technical solution, the method further comprises:

[0015] if the number of candidate images is less than the preset value, performing k-nearest neighbor search in the database according to the global descriptor to re-determine similar images.

[0016] According to an aspect of the above technical solution, the step of 2D-3D matching each of the locations based on the local descriptor and the key point score, and estimating and outputting a six-degree-of-freedom camera pose through a perspective-n-point algorithm and a random sample consensus algorithm specifically comprises:

[0017] extracting 2D key points with a confidence higher than a threshold in a query image based on a local descriptor for feature matching and a key point score for quantifying key point confidence;

[0018] matching 3D points corresponding to the 2D key points in each of the locations, and retaining valid matches;

[0019] solving a camera pose based on the matching relationship through a perspective-n-point algorithm, and outputting an initial pose estimate;

[0020] optimizing the initial pose estimate through iterative optimization and inlier selection based on a random sample consensus algorithm, to obtain a six-degree-of-freedom camera pose that meets the valid condition and has the earliest output time.

[0021] According to an aspect of the above technical solution, the GPS coordinates include longitude, latitude and height, and the method further comprises:

[0022] In the step of matching 2D key points and 3D points in each of the locations, preferential matching is performed according to the order of similar heights.

[0023] According to an aspect of the above technical solution, the condition for determining whether the six-degree-of-freedom camera pose is valid comprises:

[0024] The reprojection error in the perspective-n-point algorithm is less than a first preset value, and the inlier ratio in the random sample consensus algorithm is greater than a second preset value.

[0025] In a second aspect, the present application provides a GPS-based visual positioning system, comprising:

[0026] A query module is configured to obtain GPS coordinates of a query image, and filter reference images with close geographical positions from a database based on the GPS coordinates;

[0027] A candidate module is configured to calculate geographical distances of the reference images according to a Haversine formula, and filter reference images with distances less than or equal to a preset radius as a candidate set based on the geographical distances;

[0028] A similarity module is configured to extract global descriptors, local descriptors and key point scores of the query image based on a HF-Net network architecture, perform a k-nearest neighbor search in the candidate set according to the global descriptors, and determine a plurality of similar images;

[0029] A co-view module is configured to cluster the similar images according to co-view relationships of the similar images based on a SfM model, and generate a plurality of locations corresponding to the candidate images, wherein each of the locations contains 3D points commonly observed;

[0030] A positioning module is configured to perform 2D-3D matching on the locations based on the local descriptors and the key point scores, estimate and output a six-degree-of-freedom camera pose by a perspective-n-point algorithm and a random sample consensus algorithm, and realize visual positioning based on the six-degree-of-freedom camera pose.

[0031] According to an aspect of the above technical solution, the system further comprises:

[0032] A judgment module is configured to judge whether the six-degree-of-freedom camera pose is valid, and if the six-degree-of-freedom camera pose is invalid, judge whether the number of candidate images in the candidate set is greater than a preset value;

[0033] If the number of candidate images is less than the preset value, the preset radius is enlarged to re-filter to update the candidate set.

[0034] According to an aspect of the above technical solution, the system further comprises:

[0035] An update module is configured to, if the number of candidate images is less than the preset value, perform a k-nearest neighbor search in the database according to the global descriptors to re-determine similar images.

[0036] According to an aspect of the above technical solution, the positioning module is specifically configured to:

[0037] Based on the local descriptor for feature matching and the key point score for quantifying the key point confidence, 2D key points with confidence higher than a threshold in the query image are extracted;

[0038] 3D points corresponding to the 2D key points are matched in each of the locations, and valid matches are retained;

[0039] Based on the matching relationship, the camera pose is solved by a perspective n-point algorithm, and an initial pose estimate is output;

[0040] Based on a random sample consensus algorithm, the initial pose estimate is optimized by iteration and inlier selection, a six-degree-of-freedom camera pose that meets the effective condition and has the earliest output time is obtained.

[0041] According to an aspect of the above technical solution, the GPS coordinates include longitude, latitude and height, and the system further comprises:

[0042] The priority module is configured to, in the step of matching the 2D key points and the 3D points in each of the locations, perform priority matching according to the order of the proximity of the heights.

[0043] Compared with the prior art, the present application has the beneficial effects that: the reference images are dynamically screened by using GPS coordinates, the search time is reduced, the fast response requirement of real-time scene application is met, the global descriptor, the local descriptor and the key point score are simultaneously output by the HF-Net single CNN architecture, the anti-interference ability of the deep learning features to light changes (such as night) and seasonal changes (such as winter-summer) is significantly enhanced, the cross-environment high robustness is ensured, the feature extraction time is reduced by the lightweight network design, and the method is suitable for mobile devices such as smartphones; when the GPS signal is weak or the pose is invalid, the screening radius is automatically expanded or the full database search is returned, the height information is combined with the priority matching mechanism to ensure the positioning continuity in complex scenes such as urban canyons and tunnels, and the compatibility and adaptability of the scheme are improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A flowchart of a GPS-based visual positioning method in the first embodiment of the present application;

[0045] Figure 2 A structural block diagram of a GPS-based visual positioning system in the second embodiment of the present application;

[0046] Figure 3 A hardware structure diagram of a computer in the third embodiment of the present application;

[0047] The following specific embodiments will further illustrate the present application in conjunction with the above drawings. DETAILED DESCRIPTION

[0048] For the purposes of promoting an understanding of the principles of the application, reference will now be made to the embodiments illustrated in the drawings. The embodiments shown are intended to be illustrative only and not limiting of the present application. The present application will become more fully understood from the following detailed description, taken in conjunction with the accompanying drawings.

[0049] It is to be understood that where an element is referred to as being "on" another element, it can be directly on the other element or intervening elements can also be present. Where an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can also be present. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0051] Embodiment One

[0052] Referring to Figure 1 , a flow chart of a GPS-based visual positioning method in a first embodiment of the present application is shown. As shown in the figure, the method includes the following steps:

[0053] In step S100, the GPS coordinates of a query image are obtained, and reference images with close geographical locations are filtered from a database based on the GPS coordinates. Specifically, reference images with close geographical locations are filtered from the database based on the longitude and latitude information of the GPS coordinates, so as to narrow the global search range.

[0054] In step S200, the geographical distance of the reference images is calculated according to the Haversine formula, and reference images with distances less than or equal to a preset radius are selected as a candidate set based on the geographical distance. Specifically, the Haversine formula is used to calculate the geographical distance, and candidate images within a radius (e.g. 100 meters) are selected. The expression of the Haversine formula is as follows:

[0055] ;

[0056] wherein, , R = 6371 km is the average radius of the earth, 、 is the GPS coordinates (longitude and latitude) of the query image, , GPS coordinates of the reference image.

[0057] At step S300, a global descriptor, a local descriptor and a key point score of the query image are extracted based on the HF-Net network architecture, so as to perform a k-nearest neighbor search in the candidate set according to the global descriptor, so as to determine a plurality of similar images:

[0058] Specifically, in the embodiment, the HF-Net network architecture simultaneously outputs the global descriptor, the local descriptor and the key point score through a single compact model, so as to realize efficient feature calculation in hierarchical localization:

[0059] In the embodiment, the HF-Net network architecture is optimized based on a lightweight MobileNetV2 architecture, integrates global retrieval and local matching capabilities through multi-task distillation training, and is divided into four parts: an encoder (MobileNetV2) for extracting multi-scale feature maps of a backbone network, a global descriptor head for generating an image-level feature vector, a local descriptor head for outputting a dense local feature map, and a key point score head for predicting sparse key point confidence. The compact MobileNetV2 and multi-task distillation technology are suitable for mobile devices, have low memory and computing occupancy, and meet the application scenarios with low resource requirements.

[0060] At step S400, the similar images are clustered based on a SfM model according to a co-view relationship, so as to generate a plurality of places corresponding to the candidate images, and each place contains a common observed 3D point. Specifically, in the embodiment, the SFM (Structure-from-Motion, motion recovery structure) model is a core basis for constructing a known 3D environment, and is an environment prior knowledge base for visual localization, which supports the entire hierarchical localization process: the candidate images are clustered into "places" based on co-view images, each place contains a subset of images sharing 3D points, which is used to provide 3D point coordinates as input for a PnP (perspective n-point) algorithm, so as to realize matching between 2D key points of the query image and 3D coordinates of the SFM model.

[0061] At step S500, 2D-3D matching is performed on each place based on the local descriptor and the key point score, and a six-degree-of-freedom camera pose is estimated and output through a perspective n-point algorithm and a random sample consensus algorithm, so as to realize visual localization based on the six-degree-of-freedom camera pose.

[0062] Specifically, based on the local descriptor for feature matching and the key point score for quantifying the key point confidence, 2D key points in the query image with a confidence higher than a threshold are extracted.

[0063] matching 3D points (derived from the SfM model) corresponding to the 2D key points (derived from the query image) within each of the locations and retaining valid matches; wherein the valid matches are filtered by an improved ratio test for retaining only strong matches of different 3D points;

[0064] solving camera poses based on the matching relationship by a perspective-n-point algorithm and outputting an initial pose estimate;

[0065] optimizing the initial pose estimate by iterative optimization and inlier selection based on a random sample consensus algorithm to obtain a six-degree-of-freedom camera pose that satisfies an effective condition and outputs the earliest in time.

[0066] Preferably, in the embodiment, before the step of implementing visual positioning based on the six-degree-of-freedom camera pose, the method further comprises:

[0067] determining whether the six-degree-of-freedom camera pose is valid, and if the six-degree-of-freedom camera pose is invalid, determining whether the number of candidate images in the candidate set is greater than a preset value;

[0068] if the number of candidate images is less than the preset value, expanding the preset radius to re-screen to update the candidate set.

[0069] Further, the method further comprises: if the number of candidate images is less than the preset value, performing a k-nearest neighbor search in the database according to the global descriptor to re-determine similar images. This way can ensure the reliability of the scheme when GPS is unavailable, and is suitable for different GPS accuracies and has a wide range of applications.

[0070] Further, the condition for determining whether the six-degree-of-freedom camera pose is valid comprises:

[0071] the re-projection error in the perspective-n-point algorithm is less than a first preset value, and the inlier ratio in the random sample consensus algorithm is greater than a second preset value. The first preset value is preferably 1.5 pixels, and the second preset value is preferably 50%.

[0072] Further, the GPS coordinates comprise longitude, latitude and height, and the method further comprises:

[0073] In the step of matching 2D key points and 3D points within each of the locations, preferential matching is performed according to the order of similar heights.

[0074] In summary, the GPS-based visual positioning method in the above embodiments of the present application uses GPS coordinates to dynamically filter reference images, reduces search time, meets the rapid response requirements of real-time scene applications, simultaneously outputs global descriptors, local descriptors and key point scores through a single CNN architecture of HF-Net, significantly enhances the anti-interference ability of deep learning features to changes in light (such as at night) and seasonal changes (such as winter-summer), has high robustness across environments, and reduces feature extraction time through lightweight network design, which is suitable for mobile devices such as smartphones; when the GPS signal is weak or the pose is invalid, the filtering radius is automatically expanded or the full database search is returned, combined with the height information priority matching mechanism, to ensure the positioning continuity in complex scenes such as urban canyons and tunnels, and improve the compatibility and adaptability of the scheme.

[0075] Embodiment Two

[0076] The second embodiment of the present application also provides a GPS-based visual positioning system for implementing the above embodiments and preferred embodiments, which have been described and will not be repeated. As used below, the terms "module", "unit", "sub-unit" and the like can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and contemplated.

[0077] As shown in Figure 2 , the system comprises a query module 100, a candidate module 200, a similarity module 300, a common view module 400 and a positioning module 500.

[0078] The query module 100 is configured to obtain the GPS coordinates of a query image, and filter reference images close in geographical location from a database based on the GPS coordinates;

[0079] The candidate module 200 is configured to calculate the geographical distance of the reference images according to the Haversine formula, and filter reference images with a distance less than or equal to a preset radius as a candidate set based on the geographical distance;

[0080] The similarity module 300 is configured to extract global descriptors, local descriptors and key point scores of the query image based on the HF-Net network architecture, to perform k-nearest neighbor search in the candidate set according to the global descriptors, and to determine a plurality of similar images;

[0081] The common view module 400 is configured to cluster the similar images according to the common view relationship of each similar image based on the SfM model, to generate a plurality of locations corresponding to the candidate images, and each location contains a common observed 3D point;

[0082] The positioning module 500 is configured to perform 2D-3D matching on each of the locations based on the local descriptor and the key point score, estimate and output a six-degree-of-freedom camera pose by using a perspective-n-point algorithm and a random sample consensus algorithm, and perform visual positioning based on the six-degree-of-freedom camera pose.

[0083] Preferably, in the embodiment, the system further comprises:

[0084] The judging module is configured to judge whether the six-degree-of-freedom camera pose is valid, and if the six-degree-of-freedom camera pose is invalid, judge whether the number of candidate images in the candidate set is greater than a preset value.

[0085] If the number of candidate images is less than the preset value, the preset radius is enlarged to re-screen and update the candidate set.

[0086] Preferably, in the embodiment, the system further comprises:

[0087] The updating module is configured to, if the number of candidate images is less than the preset value, perform k-nearest neighbor search in the database according to the global descriptor to re-determine similar images.

[0088] Preferably, in the embodiment, the positioning module 500 is specifically configured to:

[0089] extract 2D key points with a confidence higher than a threshold in a query image based on a local descriptor for feature matching and a key point score for quantifying key point confidence;

[0090] match 3D points corresponding to the 2D key points in each of the locations and retain valid matches;

[0091] solve a camera pose based on a matching relationship by using a perspective-n-point algorithm, and output an initial pose estimate;

[0092] optimize the initial pose estimate by using a random sample consensus algorithm, obtain a six-degree-of-freedom camera pose that meets an effective condition and has the earliest output time, and output the six-degree-of-freedom camera pose.

[0093] Preferably, in the embodiment, the GPS coordinates include longitude, latitude and height, and the system further comprises:

[0094] The prioritizing module is configured to, in the step of matching the 2D key points with the 3D points in each of the locations, perform preferential matching according to an order of similar heights.

[0095] It should be noted that the modules can be functional modules or program modules, and can be implemented by software or hardware. For the modules implemented by hardware, the modules can be located in the same processor, or the modules can be located in different processors in any combination.

[0096] Embodiment three

[0097] The third embodiment of the present application provides a computer, which can include a processor 81 and a memory 82 storing computer program commands.

[0098] Specifically, the processor 81 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0099] The memory 82 can include mass storage for data or commands. By way of example, and not limitation, memory 82 can include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, a USB drive, a magneto-optical disk, optical disk, a tape drive, or a combination of two or more of these. Storage 82 can be removable. Storage 82 can be internal or external. In some embodiments, storage 82 is non-volatile memory. In some embodiments, storage 82 includes read-only memory (ROM) and random-access memory (RAM). The ROM can be mask- programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or FLASH, or a combination of two or more of these. The RAM can be static RAM (SRAM) or dynamic RAM (DRAM), where the DRAM can be Fast Page Mode DRAM (FPM DRAM), Extended Data Output DRAM (EDO DRAM), synchronous DRAM (SDRAM), etc.

[0100] The memory 82 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program commands executed by the processor 81.

[0101] The processor 81 reads and executes the computer program commands stored in the memory 82 to implement any of the above-mentioned GPS-based visual positioning methods.

[0102] In some embodiments, the computer can further include a communication interface 83 and a bus 80. In which, as shown, the processor 81, the memory 82, the communication interface 83 are connected through the bus 80 and complete the communication between each other. Figure 3

[0103] The communication interface 83 is used to realize the communication between each module, device, unit and / or equipment in the embodiments of the present application. The communication interface 83 can also realize the data communication between other components, such as: external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations, etc.

[0104] ​Bus 80 includes hardware, software, or both, to couple components of computer 10 to each other and to couple them both to other devices. What is considered a component of computer 10 can vary depending on the specific implementation of computer 10. Practically speaking, any system that can accept data, process data, output data, or some combination of the three includes a component of a computer. Bus 80 includes, but is not limited to, a Data Bus, Address Bus, Control Bus, Expansion Bus, Local Bus, etc. By way of example, and without limitation, bus 80 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 80 can include one or more buses. Although this application describes and shows a particular bus, this application contemplates any suitable bus or interconnect.

[0105] Embodiment Four

[0106] A readable storage medium is provided in the fourth embodiment of the application. The readable storage medium stores computer program commands. The computer program commands are executed by a processor to implement any of the above-mentioned GPS-based visual positioning methods.

[0107] The technical features of the above-mentioned embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the embodiments are described. However, as long as the combinations of the technical features do not contradict each other, they should be considered as falling within the scope of the present application.

[0108] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A GPS-based visual positioning method, characterized by, The method comprises the following steps: obtaining GPS coordinates of a query image, and screening reference images close in geographical position from a database based on the GPS coordinates; calculating geographical distances of the reference images according to a Haversine formula, and screening reference images with distances less than or equal to a preset radius as a candidate set based on the geographical distances; extracting global descriptors, local descriptors and key point scores of the query image based on an HF-Net network architecture, performing k-nearest neighbor search in the candidate set according to the global descriptors to determine a plurality of similar images; performing clustering according to the co-visibility of each of the similar images based on an SfM model to generate a plurality of locations corresponding to the candidate images, each of the locations containing commonly observed 3D points; performing 2D-3D matching on each of the locations based on the local descriptors and the key point scores, and estimating and outputting a six-degree-of-freedom camera pose through a perspective n-point algorithm and a random sample consensus algorithm to realize visual positioning based on the six-degree-of-freedom camera pose; Before the step of realizing visual positioning based on the six-degree-of-freedom camera pose, the method further comprises: determining whether the six-degree-of-freedom camera pose is valid, and if the six-degree-of-freedom camera pose is invalid, determining whether the number of candidate images in the candidate set is greater than a preset value; if the number of candidate images is less than the preset value, expanding the preset radius to re-screen to update the candidate set; The step of performing 2D-3D matching on each of the locations based on the local descriptors and the key point scores, and estimating and outputting a six-degree-of-freedom camera pose through a perspective n-point algorithm and a random sample consensus algorithm specifically comprises: extracting 2D key points in the query image with a confidence higher than a threshold based on local descriptors for feature matching and key point scores for quantifying key point confidence; matching 3D points corresponding to the 2D key points in each of the locations, and retaining valid matches; solving a camera pose based on the matching relationship through a perspective n-point algorithm, and outputting an initial pose estimate; optimizing the initial pose estimate through iteration and inlier selection based on a random sample consensus algorithm to obtain a six-degree-of-freedom camera pose that meets the effective condition and has the earliest output time.

2. The GPS-based visual positioning method according to claim 1, characterized in that, The method further comprises: if the number of candidate images is less than the preset value, performing k-nearest neighbor search in the database according to the global descriptors to redetermine similar images.

3. The GPS-based visual positioning method according to claim 1, characterized in that, The GPS coordinates include longitude, latitude and altitude, and the method further comprises: In the step of matching 2D key points and 3D points in each of the locations, preferentially matching according to the order of close altitudes.

4. The GPS-based visual positioning method according to claim 1, characterized in that, The condition for determining whether the six-degree-of-freedom camera pose is valid comprises: the re-projection error in the perspective n-point algorithm is less than a first preset value, and the inlier ratio in the random sample consensus algorithm is greater than a second preset value.

5. A GPS-based visual positioning system, characterized by The method comprises: a query module configured to obtain GPS coordinates of a query image, and screen reference images close in geographical position from a database based on the GPS coordinates; a candidate module configured to calculate a geographic distance of the reference image according to a Haversine formula, and filter reference images with a distance less than or equal to a preset radius as a candidate set based on the geographic distance; a similarity module configured to extract a global descriptor, a local descriptor and a key point score of the query image based on a HF-Net network architecture, perform a k-nearest neighbor search in the candidate set according to the global descriptor, and determine a plurality of similar images; a co-view module configured to cluster the similar images according to a co-view relationship of each of the similar images based on a SfM model, and generate a plurality of locations corresponding to the candidate images, each of the locations containing a 3D point observed in common; a positioning module configured to perform 2D-3D matching on each of the locations based on the local descriptor and the key point score, estimate and output a six-degree-of-freedom camera pose by a perspective-n-point algorithm and a random sample consensus algorithm, and realize visual positioning based on the six-degree-of-freedom camera pose; a judgment module configured to judge whether the six-degree-of-freedom camera pose is valid, and if the six-degree-of-freedom camera pose is invalid, judge whether a number of candidate images in the candidate set is greater than a preset value; if the number of candidate images is less than the preset value, enlarge the preset radius to update the candidate set; the positioning module is specifically configured to: extract 2D key points in the query image with a confidence higher than a threshold based on a local descriptor for feature matching and a key point score for quantifying key point confidence; match 3D points corresponding to the 2D key points in each of the locations, and retain valid matches; solve a camera pose based on a matching relationship by a perspective-n-point algorithm, and output an initial pose estimate; optimize the initial pose estimate by iteration and inlier selection based on a random sample consensus algorithm, obtain a six-degree-of-freedom camera pose that meets an effective condition, and output the six-degree-of-freedom camera pose with the earliest time.

6. A computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the GPS-based visual positioning method of any one of claims 1-4.

7. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to realize the GPS-based visual positioning method of any one of claims 1-4.

Images