Ground camera trusted visual positioning method and device based on satellite reference base map

Abstract

The invention relates to the technical field of visual geographic positioning, in particular to a ground camera credible visual positioning method and device based on a satellite reference base map. The method comprises the following steps: acquiring a ground panoramic image and a satellite reference base map of a target scene; dividing the ground panoramic image into a plurality of slice images, and determining a pose positioning result of the target camera according to the plurality of slice images and the satellite reference base map; and obtaining a geometric error and a gross error identification result of each slice image, obtaining a current positioning false alarm number according to the geometric error and the gross error identification result of each slice image, and obtaining the credibility of a pose positioning result according to the current positioning false alarm number. Therefore, the problems of dependence on global information, single observation, lack of positioning credibility verification capability and insufficient reliability in practical application in related technologies are solved.

Description

Technical Field

[0001] This application relates to the field of visual geolocation technology, and in particular to a reliable visual positioning method and apparatus for ground cameras based on satellite reference maps. Background Technology

[0002] Satellite-referenced ground camera visual localization methods typically encode the entire ground query image and input it along with the encoded reference base map into an end-to-end deep learning decoding model to infer the spatial location of the camera.

[0003] However, these technologies primarily rely on global visual information from ground imagery, outputting only single location information. They lack independent redundant observations to assess the reliability of space-to-ground positioning results, leading to significant errors when positioning fails. This issue severely restricts the reliability and deployability of space-to-ground visual positioning technology in practical applications. Summary of the Invention

[0004] This application provides a reliable visual positioning method and apparatus for ground cameras based on satellite reference base maps, in order to solve the problems of relying on global information, single observation, lack of positioning reliability verification capability, and insufficient reliability in practical applications in related technologies.

[0005] To achieve the above objectives, the first aspect of this application proposes a reliable visual positioning method for ground cameras based on satellite reference maps, comprising the following steps: Acquire ground panoramic images and satellite reference base maps of the target scene; The ground panoramic image is divided into multiple slice images, and the pose positioning result of the target camera is determined based on the multiple slice images and the satellite reference base map; Obtain the geometric error and gross error identification results for each slice image, and obtain the current false alarm number based on the geometric error and gross error identification results for each slice image, and obtain the reliability of the pose localization result based on the current false alarm number.

[0006] According to one embodiment of this application, determining the pose positioning result of the target camera based on the plurality of slice images and the satellite reference base map includes: The image coordinates and relative rotation angle of each slice image on the satellite reference base map are determined respectively, and a slice image pose set is generated based on the image coordinates and relative rotation angle of each slice image on the satellite reference base map. The rotation angle of the target camera is determined based on the relative rotation angles of all slice images in the slice image pose set, and the inlier pose in the slice image pose set is determined. Based on the inlier pose in the slice image pose set, the image-side coordinates of the target camera are determined. The pose positioning result of the target camera is obtained based on the rotation angle of the target camera and the image coordinates of the target camera.

[0007] According to one embodiment of this application, determining the rotation angle of the target camera based on the relative rotation angles of all slice images in the slice image pose set includes: Calculate the mean of the relative rotation angles of all slice images in the slice image pose set; The mean value is used as the rotation angle of the target camera.

[0008] According to one embodiment of this application, determining the pose of interior points in the slice image pose set includes: Determine the initial pose of each slice image in the slice image pose set; The initial poses of multiple slice images are combined in pairs to obtain a set of slice pairs, and the back intersection of each set of slice pairs is performed to obtain a set of candidate camera positions. Input each candidate camera position into the localization background model and output the geometric error of each candidate camera position; The final camera position is obtained by filtering the candidates based on their geometric errors. The interior point set is obtained based on the final camera position. The interior point pose is obtained based on the interior point set.

[0009] According to one embodiment of this application, the positioning background model includes: ; in, The location background model, K The normalization coefficient is... e For geometric error, Let be the probability density of the error. x It is the integral variable.

[0010] According to one embodiment of this application, dividing the ground panoramic image into multiple slice images includes: Determine the total number of sliced ​​images, and the horizontal and vertical field of view of the ground panoramic image; Based on the total number of slice images and the horizontal and vertical field of view of the ground panoramic image, the horizontal field of view, vertical field of view, and center coordinates of each slice image are determined. Based on the horizontal field of view, vertical field of view, and center coordinates of each slice image, the sampling range of each slice image in the ground panoramic image is determined, and the ground panoramic image is sampled according to the sampling range to obtain the multiple slice images.

[0011] The ground camera reliable visual positioning method based on satellite reference base map proposed in this application divides the ground panoramic image into multiple slice images, and determines the pose positioning result of the target camera by combining the satellite reference base map. The geometric error and gross error identification results of each slice image are obtained to obtain the current false alarm number, and thus the reliability of the pose positioning result is obtained. This solves the problems of relying on global information, single observation, lack of positioning reliability verification capability, and insufficient reliability in practical applications in related technologies.

[0012] To achieve the above objectives, a second aspect of this application provides a reliable visual positioning device for a ground camera based on a satellite reference map, comprising: The acquisition module acquires ground panoramic images and satellite reference base maps of the target scene; The determination module divides the ground panoramic image into multiple slice images and determines the pose positioning result of the target camera based on the multiple slice images and the satellite reference base map; The positioning module acquires the geometric error and gross error identification results of each slice image, obtains the current false alarm number based on the geometric error and gross error identification results of each slice image, and obtains the reliability of the pose positioning result based on the current false alarm number.

[0013] According to one embodiment of this application, the determining module is specifically used for: The image coordinates and relative rotation angle of each slice image on the satellite reference base map are determined respectively, and a slice image pose set is generated based on the image coordinates and relative rotation angle of each slice image on the satellite reference base map. The rotation angle of the target camera is determined based on the relative rotation angles of all slice images in the slice image pose set, and the inlier pose in the slice image pose set is determined. Based on the inlier pose in the slice image pose set, the image-side coordinates of the target camera are determined. The pose positioning result of the target camera is obtained based on the rotation angle of the target camera and the image coordinates of the target camera.

[0014] According to one embodiment of this application, the determining module is specifically used for: Calculate the mean of the relative rotation angles of all slice images in the slice image pose set; The mean value is used as the rotation angle of the target camera.

[0015] According to one embodiment of this application, the determining module is specifically used for: Determine the initial pose of each slice image in the slice image pose set; The initial poses of multiple slice images are combined in pairs to obtain a set of slice pairs, and the back intersection of each set of slice pairs is performed to obtain a set of candidate camera positions. Input each candidate camera position into the localization background model and output the geometric error of each candidate camera position; The final camera position is obtained by filtering the candidates based on their geometric errors. The interior point set is obtained based on the final camera position. The interior point pose is obtained based on the interior point set.

[0016] According to one embodiment of this application, the positioning background model includes: ; in, The location background model, K The normalization coefficient is... e For geometric error, Let be the probability density of the error. x It is the integral variable.

[0017] According to one embodiment of this application, the determining module is specifically used for: Determine the total number of sliced ​​images, and the horizontal and vertical field of view of the ground panoramic image; Based on the total number of slice images and the horizontal and vertical field of view of the ground panoramic image, the horizontal field of view, vertical field of view, and center coordinates of each slice image are determined. Based on the horizontal field of view, vertical field of view, and center coordinates of each slice image, the sampling range of each slice image in the ground panoramic image is determined, and the ground panoramic image is sampled according to the sampling range to obtain the multiple slice images.

[0018] The ground camera reliable visual positioning device based on satellite reference base map proposed in this application divides the ground panoramic image into multiple slice images, and determines the pose positioning result of the target camera by combining the satellite reference base map. It then obtains the geometric error and gross error identification results for each slice image to obtain the current false alarm number, and finally obtains the reliability of the pose positioning result. This solves the problems of relying on global information, single observation, lack of positioning reliability verification capability, and insufficient reliability in practical applications in related technologies.

[0019] To achieve the above objectives, a third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the ground camera reliable visual positioning method based on satellite reference base map as described in the above embodiments.

[0020] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the ground camera reliable visual positioning method based on a satellite reference base map as described in the above embodiments.

[0021] To achieve the above objectives, a fifth aspect of this application provides a computer program product, which, when executed by a processor, implements the ground camera reliable visual positioning method based on a satellite reference map as described in the above embodiments.

[0022] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating a reliable visual positioning method for ground cameras based on satellite reference maps, according to an embodiment of this application. Figure 2 This is a flowchart of a ground camera reliable visual positioning method based on a satellite reference base map according to an embodiment of this application; Figure 3 This is a block diagram of a ground camera reliable visual positioning device based on a satellite reference base map provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0025] As those skilled in the art will understand, in recent years, with the successive proposal of multiple space-to-ground perspective datasets, researchers have developed feature-matching-based space-to-ground visual localization methods. By uniformly dividing and encoding the reference base map for localization, the spatial granularity and accuracy of localization have been effectively improved. Simultaneously, a series of mitigation strategies have been proposed to address issues such as perspective differences, resolution differences, and rotation inconsistencies between space-to-ground images, significantly enhancing the robustness of localization under limited field-of-view conditions. With the continuous advancements in accuracy and efficiency of deep learning-based space-to-ground localization methods, current technology has the capability to accurately locate local sub-images within panoramic ground images, thus providing a feasible path for constructing redundant observations.

[0026] The following describes, with reference to the accompanying drawings, a reliable visual positioning method and apparatus for ground cameras based on satellite reference maps according to embodiments of this application. First, the reliable visual positioning method for ground cameras based on satellite reference maps according to embodiments of this application will be described with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart of a ground camera reliable visual positioning method based on a satellite reference base map according to an embodiment of this application.

[0028] like Figure 1 As shown, the reliable visual localization method for ground cameras based on satellite reference maps includes the following steps: In step S101, ground panoramic images and satellite reference base maps of the target scene are acquired.

[0029] Among them, terrestrial panoramic imagery refers to holographic imagery data that completely records the actual ground scene, texture of ground features, and spatial geometric information of the target scene at ground points. Satellite reference base map refers to satellite remote sensing imagery base map data that covers the geographic space of the target scene and carries unified geographic coordinate reference information.

[0030] Specifically, embodiments of this application use panoramic imaging equipment (such as a ground panoramic camera system) to capture ground panoramic images of the target scene. The horizontal field of view of the ground panoramic image is 360° and the vertical field of view is 180°. The camera's height, pitch angle, and sway angle relative to the ground are collected during the shooting process. Embodiments of this application obtain satellite digital orthophoto maps covering the scene captured by the ground camera under the same target scene, i.e., satellite reference base maps, which have three channels: RGB (Red-Green-Blue).

[0031] In step S102, the ground panoramic image is divided into multiple slice images, and the pose positioning result of the target camera is determined based on the multiple slice images and the satellite reference base map.

[0032] The pose localization result refers to the set of spatial position parameters and attitude parameters of the target camera, representing the orientation and direction of the target camera in the geospatial coordinate system. The target camera can be a ground camera.

[0033] Specifically, in this embodiment of the application, the acquired panoramic ground image of the target scene is divided into multiple slice images, and the pose positioning result of the target camera is determined by using a satellite reference base map carrying geospatial reference information as a spatial positioning reference.

[0034] Optionally, in some embodiments, the ground panoramic image is divided into multiple slice images, including: determining the total number of slice images, the horizontal field of view and the vertical field of view of the ground panoramic image; determining the horizontal field of view, the vertical field of view and the center coordinates of each slice image based on the total number of slice images, the horizontal field of view and the vertical field of view of the ground panoramic image; determining the sampling range of each slice image in the ground panoramic image based on the horizontal field of view, the vertical field of view and the center coordinates of each slice image, and sampling the ground panoramic image according to the sampling range to obtain multiple slice images.

[0035] Specifically, the acquired ground panoramic image G has a horizontal field of view of 2π and a vertical field of view of π. The ground panoramic image is divided into a set of uniformly distributed slice images, with the total number of slice images being... n Then the first n The horizontal field of view of the slice image is The vertical field of view is Then the set of sliced ​​images is: The center coordinates of the slice image are: ,in: ; in, For the first k The horizontal angular coordinates of the center of the slice image. For the first k The vertical angular coordinates of the center of the slice image.

[0036] The pixel sampling range of the slice image in the ground panoramic image is determined by the center coordinates, horizontal field of view, and vertical field of view of the slice image: Multiple slice images are obtained by sampling the ground panoramic image according to the sampling range. The sampling function is: .

[0037] Optionally, in some embodiments, determining the pose localization result of the target camera based on multiple slice images and a satellite reference base map includes: determining the image-side coordinates and relative rotation angle of each slice image on the satellite reference base map; generating a slice image pose set based on the image-side coordinates and relative rotation angle of each slice image on the satellite reference base map; determining the rotation angle of the target camera based on the relative rotation angles of all slice images in the slice image pose set; determining the interior point pose in the slice image pose set; and determining the image-side coordinates of the target camera based on the interior point pose in the slice image pose set; and obtaining the pose localization result of the target camera based on the rotation angle and the image-side coordinates of the target camera.

[0038] In this context, the image-side coordinates of each slice image on the satellite reference base map refer to the two-dimensional coordinates in the two-dimensional image plane coordinate system of the satellite reference base map, representing the pixel position of each slice image in that base map. The relative rotation angle of each slice image on the satellite reference base map refers to the rotation offset of each slice image relative to the reference base map in the image plane coordinate system of the satellite reference base map. The interior point pose refers to the reliable slice image pose obtained through screening from the set of slice image poses and that conforms to the overall pose model constraints of the target camera.

[0039] Specifically, in order to determine the slice image I k Scene coordinates ( x k , y k This application embodiment constructs a convolutional network localization framework based on feature similarity comparison. This application embodiment performs localization on sliced ​​images. I k Encode the high-order visual feature description tensor to obtain the high-order visual feature description tensor. f k At the same time, satellite reference base map R After uniformly dividing the grid, each grid is encoded separately, and each grid yields a high-dimensional visual feature description tensor. D ( i,j By describing the feature tensor of the sliced ​​image. f k Grid feature description tensor of reference base map D By comparing the cosine similarity values ​​one by one, a spatial distribution heatmap is obtained: ; in, For spatial distribution heatmap, For the first i Line number j List, ·) is the cosine similarity function.

[0040] Furthermore, spatial distribution heatmap With feature description tensor set D The images are stitched together and then decoded using a convolutional network to finally output the scene coordinates of the sliced ​​images: ; in, For the first k slice image I k Scene coordinates, ·) represents a convolutional decoding network. ·) represents the tensor splicing operation.

[0041] Furthermore, in this embodiment, the relative rotation angle is decoded through a second convolutional network, and the relative rotation angle is directly output. The cosine and sine values: ; in, For the first k The relative rotation angle of the sliced ​​image; These are the sine and cosine values ​​of the relative rotation angle; (·) represents a convolutional decoding network; (·) is the dimension copy function.

[0042] Specifically, the dimension copy function (·) Describe tensors by copying the features of sliced ​​images The feature tensor of the satellite reference base map is obtained. D Tensors of consistent shape are used to achieve matching and stitching of feature dimensions. In this embodiment, the image-side coordinates of the scene described by each slice image in the satellite reference base map are obtained through annotation. With relative rotation angle This is used as the ground truth pose of the sliced ​​image, which is then used to train the parameters of the pose estimation model PM. ; in, This is the set of trainable parameters for the pose estimation model. ·) is the model training function. This represents the true pose of the sliced ​​image.

[0043] Furthermore, for each slice image, based on the satellite reference base map R, this embodiment uses the constructed pose estimation model to determine the image-side coordinates and relative rotation angles of the scene in the slice image, obtaining a 3DoF (3 Degrees of Freedom) pose set for the slice angles, i.e., the slice image pose set: .

[0044] in, Given the set of poses of sliced ​​images, For the first k 3DoF pose of a slice image.

[0045] Furthermore, in this embodiment, the global rotation angle of the target camera is determined by the relative rotation angles of all slice images in the slice image pose set, and the slice image pose set is robustly filtered to obtain interior point poses that are highly consistent with the target camera pose model. Based on the image-side coordinate information contained in these reliable interior point poses, the image-side coordinates of the target camera are obtained. In this embodiment, the rotation angle of the target camera and the image-side coordinates are fused to obtain a target camera pose localization result that simultaneously contains position and orientation information.

[0046] Furthermore, in some embodiments, determining the rotation angle of the target camera based on the relative rotation angles of all slice images in the slice image pose set includes: calculating the mean of the relative rotation angles of all slice images in the slice image pose set; and using the mean as the rotation angle of the target camera.

[0047] Specifically, in this embodiment, the relative rotation angles of all slice images in the slice image pose set are obtained, and the average value of the relative rotation angles of all slice images is calculated. This can effectively fuse the observation information of the slice images and suppress the error caused by matching noise or abnormal disturbances in a single slice. In this embodiment, the average value is used as the rotation angle of the target camera.

[0048] Optionally, in some embodiments, determining the inlier pose in the slice image pose set includes: determining the initial pose of each slice image in the slice image pose set; combining the initial poses of multiple slice images in pairs to obtain a slice pair set, and performing a back intersection on each slice pair to obtain a candidate camera position set; inputting each candidate camera position into the localization background model and outputting the geometric error of each candidate camera position; filtering based on the geometric error of each candidate camera position to obtain the final camera position, obtaining the inlier set based on the final camera position, and obtaining the inlier pose based on the inlier set.

[0049] Optionally, in some embodiments, locating the background model includes: ; in, To locate the background model, K The normalization coefficient is... e For geometric error, Let be the probability density of the error. x It is the integral variable.

[0050] Here, the initial pose refers to the original 3DoF pose of each slice image in the slice image pose set before subsequent optimization processing. The candidate camera position refers to the potential target camera spatial position.

[0051] Specifically, in this embodiment of the application, the initial poses of multiple slice images in the slice image pose set are sampled in pairs to obtain a slice pair set. Each pair of 3DoF poses is rear-intersected to obtain a set of candidate camera positions. .

[0052] Furthermore, embodiments of this application collect ground-to-ground positioning-related data, combine ground panoramic images with mismatched satellite reference base maps to obtain multiple positioning inputs with inconsistent scene descriptions, perform 3DoF pose estimation of sliced ​​images for these input samples, and calculate geometric errors based on the true pose values ​​and estimated pose values ​​of the sliced ​​images. ; Where e is the geometric error, Let be the true pose of the slice image. This represents the pose estimate of the sliced ​​image.

[0053] Furthermore, in this embodiment, the geometric error sampling distribution is used as the probability distribution of the error. And calculate the probability distribution of the error. As a background model for positioning.

[0054] Specifically, for each candidate camera position, the geometric error of all slice poses at the candidate camera position is calculated, and the slice set is sorted in ascending order based on the geometric error. k The elements are combined with the first (k-1) elements to form a slice set. Calculate the geometric consistency measure of this set: ; in, n The total number of slices. To calculate the number of combinations, k The number of elements in the subset. For the first k Geometric error of each slice, From n Choose any slice k The total number of combinations of , From k The total number of combinations of any two elements selected from a slice.

[0055] Specifically, regarding candidate camera positions traversal k From 3 to n ,choose The minimum value is used as a geometric consistency measure for candidate camera positions. k A set of elements as optimal subset The location with the smallest geometric consistency measure among the candidate camera locations is selected as the final localization result. The corresponding optimal subset is taken as the interior point set. The interior point poses are obtained from the interior point set. The image-side coordinates of the target camera are obtained based on the image-side coordinate information contained in these interior point poses. In this embodiment, the target camera rotation angle and image-side coordinates are fused to obtain a target camera pose localization result that simultaneously includes position and orientation information.

[0056] In step S103, the geometric error and gross error identification results of each slice image are obtained, and the current false alarm number is obtained based on the geometric error and gross error identification results of each slice image. The confidence level of the pose localization result is obtained based on the current false alarm number.

[0057] The current false alarm count refers to the estimated number of slice observations that are misjudged as reliable interior points, but are actually gross errors or abnormal matches, based on the geometric error and gross error identification results of each slice image.

[0058] Specifically, in this embodiment, based on the geometric error and gross error identification results of each slice image output after robust estimation, the credibility verification of the positioning result is carried out based on the theory of oppositional reasoning: In this embodiment, the geometric consistency measure corresponding to the final positioning result is used as the upper limit of the false alarm number of positioning, and the measure is used as the core indicator for quantifying credibility and compared with the value 0. If the geometric consistency measure is less than 0, the positioning result is determined to be credible; otherwise, it is determined to be unreliable.

[0059] To facilitate a better understanding of the reliable visual positioning method for ground cameras based on satellite reference base maps proposed in this application, the following is a detailed explanation. Figure 2 Further explanation is needed.

[0060] like Figure 2 As shown, Figure 2This is a flowchart of a reliable visual positioning method for ground cameras based on satellite reference base maps, according to an embodiment of this application. As shown in the figure, this embodiment acquires ground panoramic images and satellite reference base maps. The ground panoramic images are uniformly distributed in the horizontal direction. 3DoF pose estimation is performed on the sliced ​​sub-images and the reference base map respectively to obtain redundant observations. Robust estimation is then performed based on the geometric constraints between the sliced ​​sub-images and the camera to identify inliers and gross errors. Inliers are used to estimate the camera's 3DoF pose to improve positioning accuracy. Finally, a positioning reliability index is calculated based on the geometric errors of each sliced ​​sub-image and the inlier identification results. This index is compared with 0. If it is greater than 0, a warning is issued to the user to prevent the introduction of large positioning errors.

[0061] The ground camera reliable visual positioning method based on satellite reference base map proposed in this application divides the ground panoramic image into multiple slice images, and determines the pose positioning result of the target camera by combining the satellite reference base map. The geometric error and gross error identification results of each slice image are obtained to obtain the current false alarm number, and thus the reliability of the pose positioning result is obtained. This solves the problems of relying on global information, single observation, lack of positioning reliability verification capability, and insufficient reliability in practical applications in related technologies.

[0062] Next, referring to the accompanying drawings, a ground camera reliable visual positioning device based on a satellite reference base map, according to an embodiment of this application, is described.

[0063] Figure 3 This is a block diagram of a ground camera reliable visual positioning device based on a satellite reference map according to an embodiment of this application.

[0064] like Figure 3 As shown, the ground camera reliable visual positioning device 10 based on satellite reference base map includes: acquisition module 100, determination module 200 and positioning module 300.

[0065] Module 100 acquires ground panoramic images and satellite reference base maps of the target scene; The determination module 200 divides the ground panoramic image into multiple slice images and determines the pose and positioning result of the target camera based on the multiple slice images and the satellite reference base map; The positioning module 300 acquires the geometric error and gross error recognition results of each slice image, obtains the current false alarm number based on the geometric error and gross error recognition results of each slice image, and obtains the reliability of the pose positioning result based on the current false alarm number.

[0066] According to one embodiment of this application, the determining module 200 is specifically used for: The image coordinates and relative rotation angle of each slice image on the satellite reference base map are determined respectively, and a slice image pose set is generated based on the image coordinates and relative rotation angle of each slice image on the satellite reference base map. The rotation angle of the target camera is determined based on the relative rotation angles of all slice images in the slice image pose set, and the inlier pose in the slice image pose set is determined. Based on the inlier pose in the slice image pose set, the image-side coordinates of the target camera are determined. The pose localization result of the target camera is obtained based on the rotation angle and image coordinates of the target camera.

[0067] According to one embodiment of this application, the determining module 200 is specifically used for: Calculate the mean of the relative rotation angles of all slice images in the slice image pose set; The mean value is used as the rotation angle of the target camera.

[0068] According to one embodiment of this application, the determining module 200 is specifically used for: Determine the initial pose of each slice image in the slice image pose set; The initial poses of multiple slice images are combined in pairs to obtain a set of slice pairs, and the back intersection of each set of slice pairs is performed to obtain a set of candidate camera positions. Input each candidate camera position into the localization background model and output the geometric error of each candidate camera position; The final camera position is obtained by filtering the candidates based on their geometric errors. The interior point set is then obtained based on the final camera position, and the interior point pose is obtained based on the interior point set.

[0069] According to one embodiment of this application, the positioning background model includes: ; in, To locate the background model, K The normalization coefficient is... e For geometric error, Let be the probability density of the error. x It is the integral variable.

[0070] According to one embodiment of this application, the determining module 200 is specifically used for: Determine the total number of slice images, and the horizontal and vertical field of view of the ground panoramic image; Based on the total number of slice images and the horizontal and vertical field of view of the ground panoramic image, determine the horizontal field of view, vertical field of view, and center coordinates of each slice image; Based on the horizontal field of view, vertical field of view, and center coordinates of each slice image, the sampling range of each slice image in the ground panoramic image is determined, and multiple slice images are obtained by sampling the ground panoramic image according to the sampling range.

[0071] It should be noted that the foregoing explanation of the embodiment of the ground camera trusted visual positioning method based on satellite reference map also applies to the ground camera trusted visual positioning device based on satellite reference map in this embodiment, and will not be repeated here.

[0072] The ground camera reliable visual positioning device based on satellite reference base map proposed in this application divides the ground panoramic image into multiple slice images, and determines the pose positioning result of the target camera by combining the satellite reference base map. It then obtains the geometric error and gross error identification results for each slice image to obtain the current false alarm number, and finally obtains the reliability of the pose positioning result. This solves the problems of relying on global information, single observation, lack of positioning reliability verification capability, and insufficient reliability in practical applications in related technologies.

[0073] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. The electronic device may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0074] When the processor 402 executes the program, it implements the ground camera reliable visual positioning method based on satellite reference base map provided in the above embodiments.

[0075] Furthermore, electronic devices also include: Communication interface 403 is used for communication between memory 401 and processor 402.

[0076] The memory 401 is used to store computer programs that can run on the processor 402.

[0077] The memory 401 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0078] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0079] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0080] Processor 402 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention.

[0081] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described reliable visual positioning method for ground cameras based on satellite reference maps.

[0082] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the ground camera reliable visual positioning method based on a satellite reference base map.

[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0084] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0085] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A reliable visual positioning method for ground cameras based on satellite reference base maps, characterized in that, include: Acquire ground panoramic images and satellite reference base maps of the target scene; The ground panoramic image is divided into multiple slice images, and the pose positioning result of the target camera is determined based on the multiple slice images and the satellite reference base map; Obtain the geometric error and gross error identification results for each slice image, and obtain the current false alarm number based on the geometric error and gross error identification results for each slice image, and obtain the reliability of the pose localization result based on the current false alarm number.

2. The method according to claim 1, characterized in that, The step of determining the pose positioning result of the target camera based on the multiple slice images and the satellite reference base map includes: The image coordinates and relative rotation angle of each slice image on the satellite reference base map are determined respectively, and a slice image pose set is generated based on the image coordinates and relative rotation angle of each slice image on the satellite reference base map. The rotation angle of the target camera is determined based on the relative rotation angles of all slice images in the slice image pose set, and the inlier pose in the slice image pose set is determined. Based on the inlier pose in the slice image pose set, the image-side coordinates of the target camera are determined. The pose positioning result of the target camera is obtained based on the rotation angle of the target camera and the image coordinates of the target camera.

3. The method according to claim 2, characterized in that, Determining the rotation angle of the target camera based on the relative rotation angles of all slice images in the slice image pose set includes: Calculate the mean of the relative rotation angles of all slice images in the slice image pose set; The mean value is used as the rotation angle of the target camera.

4. The method according to claim 2, characterized in that, Determining the pose of interior points in the slice image pose set includes: Determine the initial pose of each slice image in the slice image pose set; The initial poses of multiple slice images are combined in pairs to obtain a set of slice pairs, and the back intersection of each set of slice pairs is performed to obtain a set of candidate camera positions. Input each candidate camera position into the localization background model and output the geometric error of each candidate camera position; The final camera position is obtained by filtering the candidates based on their geometric errors. The interior point set is obtained based on the final camera position. The interior point pose is obtained based on the interior point set.

5. The method according to claim 4, characterized in that, The positioning background model includes: ； in, The location background model, K The normalization coefficient is... e For geometric error, Let be the probability density of the error. x It is the integral variable.

6. The method according to claim 1, characterized in that, The step of dividing the ground panoramic image into multiple slice images includes: Determine the total number of sliced ​​images, and the horizontal and vertical field of view of the ground panoramic image; Based on the total number of slice images and the horizontal and vertical field of view of the ground panoramic image, the horizontal field of view, vertical field of view, and center coordinates of each slice image are determined. Based on the horizontal field of view, vertical field of view, and center coordinates of each slice image, the sampling range of each slice image in the ground panoramic image is determined, and the ground panoramic image is sampled according to the sampling range to obtain the multiple slice images.

7. A reliable visual positioning device for ground cameras based on satellite reference maps, characterized in that, include: The acquisition module acquires ground panoramic images and satellite reference base maps of the target scene; The determination module divides the ground panoramic image into multiple slice images and determines the pose positioning result of the target camera based on the multiple slice images and the satellite reference base map; The positioning module acquires the geometric error and gross error identification results of each slice image, obtains the current false alarm number based on the geometric error and gross error identification results of each slice image, and obtains the reliability of the pose positioning result based on the current false alarm number.

8. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the reliable visual positioning method for ground cameras based on satellite reference maps as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the ground camera reliable visual positioning method based on satellite reference base map as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the reliable visual positioning method for ground cameras based on satellite reference maps as described in any one of claims 1-6.

Images