Unmanned aerial vehicle visual positioning method and system based on geographic reference image matching

Abstract

The application discloses a UAV visual positioning method and system based on geographic reference image matching, and relates to the technical field of UAV visual positioning. The method comprises the following steps: acquiring a ground image collected by a UAV in real time, and converting the ground image into a BEV overhead image; a candidate matching region is cropped out; multi-scale feature extraction is performed to generate a key point set and a feature descriptor set; photometric disambiguation intervention is performed to generate an optimized primitive set; adaptive local regularization constraint is performed to output a corrected robust feature space; adaptive geometric consistency checking is used to remove false matching points, and actual geographic position coordinates are obtained, error evaluation information and positioning results are output. The application solves the technical problems that the existing UAV visual positioning is easily disturbed by scene photometric ambiguity, the feature false matching rate is high, and the positioning stability is poor, realizes the matching of real-time images of a UAV and geographic reference images, and improves the technical effects of robustness and positioning accuracy of UAV visual positioning.

Description

Technical Field

[0001] This invention relates to the field of UAV visual positioning technology, and specifically to a UAV visual positioning method and system based on geographic reference image matching. Background Technology

[0002] Current UAV visual positioning often uses feature matching between real-time aerial images and pre-stored georeferenced images. However, outdoor scenes such as rooftops and open fields have problems such as strong reflections, monotonous local textures, and drastic changes in lighting. Scene photometric ambiguity can cause feature descriptor distortion, leading to a large number of mismatches. Conventional feature matching methods lack fine-grained modeling of 3D scene geometry and photometric ambiguity, making it difficult to distinguish between real textures and lighting artifacts. This results in a low proportion of matched inliers, and the positioning results are prone to jumps and lack robustness. Relying solely on traditional RANSAC geometric verification cannot completely eliminate erroneous feature pairs caused by photometric ambiguity, making it difficult to meet the operational requirements of high-precision and high-reliability visual positioning for UAVs.

[0003] Existing UAV visual positioning suffers from technical problems such as susceptibility to scene light ambiguity, high feature mismatch rate, and poor positioning stability. Summary of the Invention

[0004] This application provides a method and system for UAV visual positioning based on geographic reference image matching, which is used to address the technical problems of existing UAV visual positioning being susceptible to interference from scene light ambiguity, high feature mismatch rate, and poor positioning stability.

[0005] In view of the above problems, this application provides a method and system for UAV visual positioning based on geographic reference image matching.

[0006] The first aspect of this application provides a UAV visual positioning method based on georeferenced image matching, the method comprising: The system acquires real-time ground images from the UAV, constructs a measurement coordinate system by combining flight status information and camera parameters, and converts the ground images into a BEV (Battlefield Elevated Vehicle) top-view image. Candidate matching regions are cropped from a georeferenced image database based on prior location and field of view. Multi-scale feature extraction is performed on both the BEV top-view image and the candidate matching regions to generate a keypoint set and a feature descriptor subset. For the rendering scene composed of Gaussian elements in the BEV top-view image, photometric disambiguation is performed by directly discarding the edge regions of each Gaussian element and applying ray-color consistency weighted variance regularization to generate an optimized base. The system employs a set of primitives; calculates a polysemy indicator based on the magnitude of the coefficients of higher-order spherical harmonic functions, and identifies primitives at the top and bottom percentiles as fuzzy primitives within the polysemy range; applies adaptive local regularization constraints to the fuzzy primitives, and outputs a modified robust feature space; performs candidate matching screening on the set of key points and the feature descriptor subset, and uses adaptive geometric consistency verification to eliminate mismatched points in combination with the optimized primitive set and robust feature space; maps image coordinates to a georeferenced image through a spatial transformation matrix, calculates the actual geographic location coordinates, and outputs error assessment information and positioning results.

[0007] A second aspect of this application provides a UAV visual positioning system based on georeferenced image matching, the system comprising: The system includes a ground image acquisition module for acquiring real-time ground images from the UAV, constructing a measurement coordinate system by combining flight status information and camera parameters, and converting the ground images into a BEV top-view image. A candidate matching region acquisition module is used to crop candidate matching regions from a georeferenced image database based on prior location and field of view. A multi-scale feature extraction module is used to perform multi-scale feature extraction on both the BEV top-view image and the candidate matching regions, generating a keypoint set and a feature descriptor subset. An optimized primitive set generation module is used to perform photometric disambiguation on the rendering scene composed of Gaussian primitives in the BEV top-view image by directly discarding the edge regions of each Gaussian primitive and applying ray-color consistency weighted variance regularization. The system generates an optimized primitive set; a fuzzy primitive judgment module is used to calculate the ambiguity indicator based on the magnitude of the coefficients of the higher-order spherical harmonic function, and to determine the primitives at the top percentile and bottom percentile as fuzzy primitives in the ambiguity interval; a robust feature space output module is used to apply adaptive local regularization constraints to the fuzzy primitives and output the corrected robust feature space; a positioning result acquisition module is used to perform candidate matching and screening on the key point set and feature descriptor subset, and to use adaptive geometric consistency verification to eliminate mismatched points in combination with the optimized primitive set and robust feature space, and to map the image coordinates to the georeferenced image through a spatial transformation matrix, convert the actual geographical location coordinates, and output error evaluation information and positioning results.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: The system acquires real-time ground images from a UAV and converts them into a BEV (Balance of Vehicle) top-down image. Candidate matching regions are cropped. Multi-scale feature extraction is performed to generate a keypoint set and a feature descriptor subset. For the rendering scene composed of Gaussian primitives in the BEV top-down image, photometric disambiguation is performed by directly discarding the edge regions of each Gaussian primitive and applying ray-color consistency weighted variance regularization to generate an optimized primitive set. An ambiguity indicator is calculated, and primitives at the top and bottom percentiles are identified as fuzzy primitives in the ambiguity interval. Adaptive local regularization constraints are applied, and a corrected robust feature space is output. Candidate matching is screened on the keypoint set and feature descriptor subset, and mismatched points are eliminated using adaptive geometric consistency verification combined with the optimized primitive set and robust feature space. Image coordinates are mapped to a georeferenced image using a spatial transformation matrix, and the actual geographic location coordinates are calculated. Error assessment information and positioning results are output. This achieves the technical effect of matching real-time UAV images with georeferenced images, improving the robustness and accuracy of UAV visual positioning. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of the UAV visual positioning method based on geographic reference image matching provided in the embodiments of this application; Figure 2 A bar chart comparing the comprehensive errors of different algorithm levels in the UAV visual positioning method based on geographic reference image matching provided in the embodiments of this application.

[0011] Figure 3 A schematic diagram of the structure of a UAV visual positioning system based on geographic reference image matching provided in an embodiment of this application.

[0012] Figure labeling: 10 Ground image acquisition module, 20 Candidate matching region acquisition module, 30 Multi-scale feature extraction module, 40 Optimized primitive set generation module, 50 Fuzzy primitive judgment module, 60 Robust feature space output module, 70 Localization result acquisition module. Detailed Implementation

[0013] This application provides a UAV visual positioning method and system based on geographic reference image matching, which addresses the technical problems of existing UAV visual positioning being susceptible to interference from scene light ambiguity, high feature mismatch rate, and poor positioning stability.

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0015] Example 1, as Figure 1 As shown, this application provides a UAV visual positioning method based on geographic reference image matching, the method comprising: Step S100: Acquire ground images collected in real time by the UAV, construct a measurement coordinate system by combining flight status information and camera parameters, and convert the ground images into BEV top view images.

[0016] Specifically, the system synchronously reads real-time ground images output by the UAV's onboard sensors, and simultaneously collects flight status information such as UAV altitude, attitude angle, and heading angle, as well as pre-calibrated intrinsic and extrinsic parameters of the onboard camera. Based on this, a measurement coordinate system is constructed. The acquired ground images are first subjected to grayscale processing and size normalization operations, and image distortion correction is performed in conjunction with camera distortion parameters to eliminate pixel shifts caused by lens distortion. Then, the flight altitude, attitude angle, heading angle, and camera intrinsic and extrinsic parameters are integrated to obtain a homography transformation matrix. Using this homography transformation matrix, all original image pixels on the original ground image are projected and mapped one by one, converting the original pixel coordinates under the perspective view to the pixel coordinates under the top view, completing perspective distortion correction, and finally outputting a BEV top view image (Bird's-EyeView), providing a top view image source for subsequent image matching processing.

[0017] Step S200: Crop candidate matching regions from the georeferenced image database based on prior location and field of view.

[0018] Specifically, the prior position, current flight altitude, and camera field of view output by the UAV are read and used as cropping input parameters for the georeferenced image. The prior position of the UAV is used as the central reference point, and the physical size coverage boundary corresponding to the ground side is calculated by combining the current flight altitude and field of view. Then, the map resolution parameters stored in the georeferenced image database are retrieved. Based on the map resolution, a local image that can completely cover the above coverage boundary is extracted from the georeferenced image database as a candidate matching region. At the same time, the map pixel coordinates, real geographic coordinates, and meter / pixel ratio corresponding to the candidate matching region are recorded simultaneously, providing reference map data and scale conversion basis for subsequent image feature matching and pixel-geographic coordinate conversion.

[0019] Step S300: Perform multi-scale feature extraction on the BEV top view image and the candidate matching region respectively to generate a key point set and a feature descriptor set.

[0020] Specifically, the output BEV top-view image and the acquired candidate matching regions are fed into a feature extraction network built on XFeat (X-Feature, a cross-scale fast feature extraction network) as input data. The TensorRT deep learning inference engine (Tensor Runtime, a deep learning inference acceleration engine) is called to perform forward inference acceleration processing on the XFeat feature extraction network. With the help of the network's multi-scale feature extraction capabilities, feature detection and feature encoding operations are performed on the BEV top-view image and the candidate matching regions at multiple scales, respectively. The corresponding key point sets and feature descriptor sets for each are calculated and output, providing a complete feature data foundation for subsequent feature matching and screening between images.

[0021] Step S400: For the rendering scene composed of Gaussian primitives in the BEV top view image, perform photometric disambiguation intervention by directly discarding the edge regions of each Gaussian primitive and applying ray-color consistency weighted variance regularization to generate an optimized primitive set.

[0022] Specifically, for the Gaussian primitive rendering scene corresponding to the BEV top-view image, the standard deviation is used to define the core boundary range of each Gaussian primitive. In the rendering opacity calculation stage, the edge region of the Gaussian primitive is directly discarded, and only the core region is retained to participate in the rendering contribution calculation. This cuts off the dilation deviation caused by weak gradients. At the same time, the weighted variance regularization term of the emitted color along the ray direction relative to the desired color is calculated. The ray-color consistency weighted variance regularization is applied to complete the photometric disambiguation intervention, which constrains the Gaussian primitives at the intersection of the same physical surface to have highly similar optical properties, suppressing redundant artifacts generated in the rendering process. This generates an optimized primitive set, providing the primitive basic data after photometric disambiguation processing for subsequent fuzzy primitive determination and feature space correction.

[0023] Step S500: Calculate the ambiguity indicator based on the magnitude of the coefficients of the higher-order spherical harmonic function, and determine the primitives at the top percentile and bottom percentile as fuzzy primitives in the ambiguity interval.

[0024] Specifically, leveraging the inherent property that the variance of viewpoint-related color is proportional to the sum of squares of the coefficients of higher-order spherical harmonic functions, the calculated magnitude of the coefficients of higher-order spherical harmonic functions is defined as an ambiguity indicator. The ambiguity indicator values ​​of all primitives are statistically sorted using a dual-end indicator module. Primitives with ambiguity indicators in the top 5% represent primitives with high-risk geometrical abrupt changes or strong reflections, while primitives with ambiguity indicators in the bottom 10% represent under-constrained primitives lacking photometric supervision. These primitives are collectively identified as fuzzy primitives falling into the ambiguity range, thus completing the identification and labeling of primitives prone to ambiguity and providing a basis for subsequent adaptive local regularization constraints on fuzzy primitives.

[0025] Step S600: Apply adaptive local regularization constraints to the fuzzy primitives and output the corrected robust feature space.

[0026] Specifically, a pre-trained depth estimation network is invoked to render the current scene, obtaining a pixel-level depth map. Partial derivative operations are performed on this pixel-level depth map to generate corresponding surface normal vectors, which serve as prior constraints for pixel-level normals. For the identified fuzzy primitives, the 3D center position and 3D covariance matrix of the fuzzy primitives are used as input variables to construct a parameter-level fine-grained regularization function. The cosine angle penalty term between the model rendering normal and the surface normal of the fuzzy primitive is solved. Preset weight coefficients are introduced to weight and sum the cosine angle penalty term to construct an adaptive local regularization loss function. Furthermore, during the gradient backpropagation optimization iteration in the Gaussian sputtering rendering scene, the spatial scaling parameters and opacity parameters of the fuzzy primitives are forcibly frozen to cut off the invalid geometric drift caused by weak supervision conditions. Iterative optimization is carried out by minimizing the adaptive local regularization loss function to correct the surface artifacts and geometric deformation problems caused by the photometric ambiguity of the fuzzy primitives. Finally, a corrected robust feature space is output, providing reliable geometric prior conditions for subsequent feature matching to eliminate mismatched points.

[0027] Step S700: Perform candidate matching and screening on the key point set and feature descriptor subset, and combine the optimized primitive set and robust feature space to use adaptive geometric consistency verification to eliminate mismatched points. Map the image coordinates to the geographic reference image through the spatial transformation matrix, calculate the actual geographic location coordinates, and output error evaluation information and positioning results.

[0028] Specifically, using the BEV top-view image and the keypoint set and feature descriptor set output from the candidate matching region as input, an initial candidate matching set containing multiple sets of feature point pairs is generated by calculating the Euclidean distance score between the feature descriptors or performing bidirectional matching filtering based on the mutual nearest neighbor relationship. This initial candidate matching set is then input into the RANSAC algorithm (Random Samples Algorithm). Consensus (a random sampling consensus algorithm) performs iterative calculations and simultaneously performs adaptive geometric consistency verification by combining the geometric prior constraints of the optimized primitive set and robust feature space output. Outlier mismatches are eliminated, and the set of interior points that satisfy the spatial geometric constraints is output. The validity of the current image matching status is determined based on the number of interior points, reprojection error, and matching stability threshold. The center point or target point of the BEV top view image is mapped to the candidate matching region using the obtained spatial transformation matrix to obtain the target pixel coordinates. Combined with the meter / pixel ratio pre-recorded in the candidate matching region, the target pixel coordinates are converted into actual geographical location coordinates represented by eastward and northward distances. The error evaluation information and the final positioning result are output simultaneously, including eastward positioning error, northward positioning error, horizontal comprehensive error, heading angle error, altitude error, total number of candidate matching points, number of interior points in the RANSAC algorithm, and matching validity status flag.

[0029] In one possible implementation, step S100 further includes: Step S110: Simultaneously acquire the altitude, attitude angle, heading angle, and internal and external parameters of the onboard camera during the UAV's flight.

[0030] Step S120: Perform grayscale conversion, size normalization, and distortion correction based on distortion parameters on the ground image in sequence.

[0031] Step S130: Calculate the homography transformation matrix based on the height, attitude angle, heading angle, camera intrinsic parameters and extrinsic parameters, and map the original image pixels into top view image pixels to obtain the BEV top view image.

[0032] Specifically, during UAV flight operations, the flight status data of flight altitude, attitude angle, and heading angle output in real time from the UAV's onboard navigation sensors are read simultaneously. At the same time, the camera's internal parameters obtained from the pre-calibrated onboard camera and the external installation parameters reflecting the camera's installation relationship with the UAV body are retrieved to complete the time-aligned acquisition of the above-mentioned multi-source parameters. This ensures that the altitude, attitude angle, heading angle and camera parameters are matched in time sequence, providing complete and reliable original input parameters for subsequent construction of the measurement coordinate system and solving the homography transformation matrix to realize the conversion of ground images to BEV top view images.

[0033] The system reads ground images acquired in real time by the UAV, firstly performing grayscale processing to convert the three-channel color image into a single-channel grayscale image to reduce subsequent computational load; then performing size normalization and scaling processing according to a preset standard resolution to unify the image input size; subsequently, retrieving the distortion parameters obtained from camera calibration, and performing distortion correction calculations on each pixel of the image based on radial and tangential distortion models to correct pixel position offsets caused by lens distortion, outputting preprocessed image data to provide preprocessed image input for subsequent homography transformation matrix projection to generate BEV top view images.

[0034] Based on the UAV's flight altitude, attitude angle, and heading angle, a rotation and translation matrix relative to the world coordinate system is constructed. The attitude rotation matrix is ​​calculated using the attitude angle and heading angle, with flight altitude serving as the vertical dimension parameter of the translation component. The transformation matrix from the UAV coordinate system to the camera coordinate system is obtained using the extrinsic parameters of the onboard camera. The camera intrinsic parameters are assembled to form the camera's inherent imaging intrinsic parameter matrix. The UAV-world rotation and translation matrix, the UAV-camera extrinsic parameter transformation matrix, and the camera intrinsic parameter matrix are combined. Based on the assumption of an approximate ground plane, the depth dimension is eliminated, and a homography transformation matrix for perspective projection to top-view perspective is derived. All pixels of the preprocessed original ground image are traversed, and the homography transformation matrix is ​​multiplied by the homography transformation matrix to obtain the mapped homography pixel coordinates. Homography normalization is then performed to obtain the top-view image pixel coordinates. Pixel resampling is used to fill in the grayscale information of each top-view pixel position, completing the perspective deformation transformation and finally obtaining the BEV top-view image, providing top-view image input for subsequent multi-scale feature extraction.

[0035] In one possible implementation, step S200 further includes: Step S210: Obtain the prior position, current flight altitude, and field of view of the UAV as input parameters for the clipping space.

[0036] Step S220: Using the prior position as the center, calculate the coverage boundary of the corresponding ground physical dimensions in combination with the current flight altitude and field of view.

[0037] Step S230: Based on the map resolution, extract a local roof image covering the coverage boundary from the geographic reference image database as the candidate matching area, and simultaneously record the map pixel coordinates, real geographic coordinates, and meter / pixel ratio corresponding to the candidate matching area.

[0038] Specifically, the prior location information output by the UAV's onboard positioning module is read. This prior location can be a rough geographic coordinate output by GPS / inertial navigation. The UAV's real-time current flight altitude data is collected simultaneously, and the field of view corresponding to the camera is calculated by combining the inherent optical parameters of the onboard camera. The aforementioned prior location, current flight altitude, and field of view are used as the clipping space input parameters for the candidate matching region. The multi-source parameters are timestamped and aligned, and a set of parameters that matches the current ground image acquisition time is selected to avoid parameter mismatch at different times. This provides an accurate input data source for subsequent calculation of the ground physical coverage boundary and clipping candidate matching regions from the georeferenced image database.

[0039] Using the prior geographical location of the UAV after time alignment as the center reference point of the plane, the horizontal and vertical physical coverage distances projected by the camera on the ground are calculated based on the camera's field of view angle and the current flight altitude using similar triangular projection relationships. The corresponding physical coverage distances are extended to the east, west, south, and north directions of the center reference point, respectively. The maximum and minimum eastern and northern boundaries of the ground physical area are obtained by solving these problems. These boundaries are combined to form a rectangular ground physical size coverage boundary, which limits the actual geographical range of the ground corresponding to the current UAV imaging. This provides a basis for calculating the physical boundary for subsequent extraction of corresponding candidate matching areas from the georeferenced image database.

[0040] The map resolution stored in the georeferenced image database is retrieved. Based on this map resolution, the previously calculated ground physical size coverage boundary is converted into the corresponding pixel boundary range on the reference image. According to the converted pixel boundary, a local roof image completely surrounding the pixel boundary is extracted from the georeferenced image database as a candidate matching region. The map pixel coordinates corresponding to the candidate matching region in the georeferenced image are located and recorded simultaneously. The real geographic coordinates corresponding to the four corner points of the region are extracted. The meter / pixel ratio is calculated by dividing the ground physical distance by the image pixel span. The map pixel coordinates, real geographic coordinates, and meter / pixel ratio are saved together to provide scale and location reference parameters for the subsequent conversion of pixel coordinates to actual geographic coordinates after image feature matching is completed.

[0041] In one possible implementation, step S300 further includes: Step S310: Input the BEV top view image and the candidate matching region into the feature extraction network built based on XFeat, using the BEV top view image and the candidate matching region as network input data.

[0042] Step S320: Accelerate the forward inference of the XFeat feature extraction network using the TensorRT deep learning inference engine, and output the corresponding key point set and feature descriptor subset.

[0043] Specifically, the XFeat native network base model is loaded, and the feature extraction network is built based on this native network. The XFeat native multi-scale convolutional backbone network, keypoint detection head, and descriptor decoding head structure are retained, and the interface specifications such as network input image size and channel arrangement are determined. The obtained BEV top view image and the obtained candidate matching region are read, and the channel dimension and pixel value range of the two images are adapted to meet the network input interface requirements. The adapted BEV top view image and candidate matching region are input into the feature extraction network built based on XFeat to complete the image data loading. This provides a complete and usable network model and input image data for subsequent calls to the TensorRT deep learning inference engine to accelerate the forward inference of the network and output the corresponding keypoint set and feature descriptor set.

[0044] The feature extraction network built based on XFeat is exported as a general model file of ONNX (Open Neural Network Exchange), imported into the TensorRT deep learning inference engine, and then subjected to model structure parsing, computational layer fusion, operator replacement, precision calibration, and memory allocation optimization. This process is then compiled to generate a TensorRT inference engine instance for deployment on hardware. The format-adapted BEV top-view image and candidate matching regions are assembled into an input tensor according to the inference engine requirements and fed into the compiled inference engine instance for forward inference. TensorRT performs hardware-accelerated computations on operators such as convolution operations, multi-scale downsampling, keypoint detection, and feature descriptor encoding within the network. After inference, the original tensor output from the engine is read, and the output tensor is parsed and decoded to extract the keypoint sets and feature descriptor sets corresponding to the BEV top-view image and candidate matching regions, providing inference-accelerated feature data for subsequent feature matching and selection.

[0045] In one possible implementation, step S400 further includes: Step S410: Use the standard deviation to define the core boundary of each Gaussian primitive. When calculating the rendering opacity, discard the edge region directly and retain only the contribution of the core region to cut off the inflation bias caused by weak gradients.

[0046] Step S420: Calculate the weighted variance regularization term of the emission color along the ray relative to the desired color of the mixture, force the primitives at the intersection of the same physical surface to have highly similar optical properties, suppress redundant artifacts, and thus generate the optimized primitive set.

[0047] Specifically, all Gaussian primitives in the BEV top-view image rendering scene are traversed, and the standard deviation parameter of the Gaussian distribution of each Gaussian primitive is obtained. The core boundary range of each Gaussian primitive is defined by the standard deviation multiple, distinguishing the core effective region and the edge region of the Gaussian primitive. During the process of calculating the contribution of each Gaussian primitive to the rendering opacity of the pixel in the rendering pipeline, region judgment is performed. The rendering contribution of the part determined to be the edge region is directly discarded, and only the core region is included in the weighted calculation of rendering opacity. The weak gradient regions at the edge are no longer mixed and superimposed, thereby cutting off the primitive expansion deviation caused by the weak gradient, completing the screening of the effective rendering region of the primitive, and providing a regional constraint basis for the subsequent introduction of light-color consistency weighted variance regularization to carry out photometric disambiguation intervention.

[0048] For each pixel in the rendering pipeline, project a ray, find all valid Gaussian elements that intersect the scene with the ray, and denote the emission color of the i-th intersecting Gaussian element as . Opacity weight is First, follow the formula Solving for the desired color of the mixture Then, based on the opacity of each primitive... As a weighting coefficient, it is used in the formula Calculate the weighted variance regularization term for ray-color consistency. The regularization term is incorporated into the overall loss function of Gaussian sputtering to participate in parameter iteration optimization. When the variance of primitive color at the intersection of the same physical surface is too large, the loss penalty is increased. This forces the optical properties such as emission color of all Gaussian primitives at the intersection of the same physical surface to tend to be consistent, suppressing redundant rendering artifacts such as ghosting and color distortion caused by color mixing. This completes the photometric disambiguation optimization of Gaussian primitives in the scene and generates an optimized primitive set.

[0049] In one possible implementation, step S500 further includes: Step S510: Utilizing the property that the variance of viewpoint-related color is proportional to the sum of squares of the coefficients of the higher-order spherical harmonic function, the modulus of the coefficients of the higher-order spherical harmonic function is defined as an ambiguity indicator.

[0050] Step S520: Design a dual-end indicator module to jointly determine the high-risk geometrical mutations or strong reflective primitives in the first 5% of the ambiguous indicator and the underconstrained primitives in the last 10% that lack photometric supervision as fuzzy primitives in the ambiguous range.

[0051] Specifically, for each optimized Gaussian primitive in the scene, extract all higher-order spherical harmonic coefficients stored in that primitive, assuming the higher-order spherical harmonic coefficient vector is... , where n is the highest order of spherical harmonics, according to the formula The magnitude S of the coefficient vector is calculated, which serves as the ambiguity indicator. Utilizing the inherent property that the variance of viewpoint-related color is proportional to the sum of squares of the coefficients of higher-order spherical harmonic functions, a larger sum of squares of spherical harmonic coefficients indicates stronger color fluctuations of the primitive as the viewing viewpoint changes, and a higher risk of geometric and photometric ambiguity. Thus, a mapping relationship is established between the ambiguity indicator and the degree of color change of the primitive at the viewpoint, achieving a quantitative characterization of the photometric ambiguity of each higher-order primitive, and providing quantitative indicators for subsequent screening of fuzzy primitives.

[0052] A dual-end indicator module was constructed. This module reads the polysemous indicator values ​​corresponding to all Gaussian primitives, sorts all values ​​from largest to smallest, calculates quantile thresholds based on the total sample size, extracts the thresholds corresponding to the top 5% intervals after sorting as the upper limit judgment thresholds, and extracts the thresholds corresponding to the top 10% intervals after sorting as the lower limit judgment thresholds. Iterates through each Gaussian primitive. When the polysemous indicator of a primitive is greater than or equal to the upper limit judgment threshold, the primitive is determined to be a primitive with high-risk geometrical mutation or strong reflectivity. When the polysemous indicator of a primitive is less than or equal to the lower limit judgment threshold, the primitive is determined to be an under-constrained primitive lacking photometric supervision. The above two types of primitives are uniformly marked as fuzzy primitives falling into the polysemous interval, completing the screening and identification of fuzzy primitives, and providing identification results for subsequent application of adaptive local regularization constraints to fuzzy primitives.

[0053] In one possible implementation, step S600 further includes: Step S610: Obtain the pixel-level depth map output by the pre-trained depth estimation network for rendering the current scene, perform partial derivative differentiation on the pixel-level depth map to generate the corresponding surface normal vector, and use it as the prior constraint for the pixel-level normal.

[0054] Step S620: For the fuzzy primitive, construct a parameter-level fine-grained regularization function with its three-dimensional center position and three-dimensional covariance matrix as input variables, and calculate the cosine angle penalty term between the model rendering normal of the fuzzy primitive and the surface normal vector.

[0055] Step S630: Introduce a preset weight coefficient to perform a weighted summation of the cosine angle penalty term, construct an adaptive local regularization loss function, and in the gradient backpropagation optimization iteration process of the Gaussian sputtering rendering scene, forcibly freeze the spatial scaling parameter and opacity parameter of the fuzzy primitive to cut off the invalid geometric drift caused by weak supervision.

[0056] Step S640: By iteratively minimizing the adaptive local regularization loss function, the surface artifacts and geometric deformations caused by photometric polysemy of the fuzzy primitives are corrected, thereby outputting the robust feature space.

[0057] Specifically, offline pre-training of the depth estimation network is completed in advance. A dataset of paired real images and ground truth depth maps is constructed, including low-altitude drone scenes, rooftop scenes, and complex outdoor lighting scenes. The network is iteratively trained using a depth regression loss function, and the network weights and bias parameters are continuously updated through backpropagation until the depth prediction error of the network on the validation set converges and reaches the preset accuracy index. The weight parameters are then saved to obtain a generalizable pre-trained depth estimation network. During real-time drone operation, the rendered image of the current scene is input into the pre-trained depth estimation network. After forward inference, a dense and continuous pixel-level depth map is output, in which the value of each pixel accurately corresponds to the spatial depth information of the corresponding location in the scene. Subsequently, two-dimensional discrete partial derivative operations are performed on the depth map to solve the first-order depth partial derivatives in the horizontal and vertical directions, respectively. Combined with the camera imaging geometry, the three-dimensional spatial surface normal vector corresponding to each pixel is solved to eliminate local depth noise disturbances. Finally, a pixel-by-pixel surface normal vector is generated, which serves as a pixel-level normal prior constraint for subsequent primitive optimization.

[0058] For the selected fuzzy primitives, the 3D center position and 3D covariance matrix of the fuzzy primitives are used as input variables to construct a parametric fine-grained regularization function. Eigenvalue decomposition is performed on the 3D covariance matrix of the fuzzy primitives to obtain the eigenvalues ​​and eigenvectors corresponding to the covariance matrix. The principal eigenvector corresponding to the largest eigenvalue is taken and normalized to obtain the model rendering normal of the fuzzy primitive. This characterizes the local surface orientation currently fitted by the Gaussian primitive; the primitive is projected onto the image plane, and the prior surface normal vector at the corresponding pixel location is obtained from the depth map. , Similarly, normalize to unit vectors; calculate the cosine values ​​of the two normal vectors using the dot product method, construct a cosine angle penalty term, and calculate the formula as follows: In the formula The dot product of two unit vectors is equal to the cosine of the angle between them; as the deviation between the model's rendered normal and the prior normal increases, the dot product becomes smaller, resulting in a penalty term. As the value increases, the penalty term is incorporated into the overall loss function for backpropagation optimization. During the iteration process, the covariance parameter of the fuzzy primitives is constrained, the spatial orientation of the primitives is corrected, the geometric fitting bias is reduced, and the geometric parameters of the fuzzy primitives are refined.

[0059] A set of adaptively adjustable preset weight coefficients are set up, and the normal cosine angle penalty terms corresponding to all the solved fuzzy primitives are weighted and calculated on a primitive-by-primitive basis. The weighted penalty terms are then summed globally, and an adaptive local regularization loss function for fuzzy primitive correction is constructed. In the gradient backpropagation iterative optimization process of the Gaussian sputtering rendering model, a parameter freezing constraint strategy is implemented on the identified fuzzy primitives. The spatial scaling parameter and opacity parameter of the fuzzy primitives are locked separately, and these two types of parameters are prohibited from participating in gradient updates and backpropagation iterative updates. Only the spherical harmonic color parameter and orientation normal parameter of the fuzzy primitives are allowed to participate in the optimization iteration. The parameter freezing mechanism completely blocks the invalid geometric offset, size distortion and abnormal transparency drift caused by the stochastic gradient update of parameters under weak supervision. Combined with the adaptive normal regularization loss, the surface orientation of the fuzzy primitives is continuously corrected. While ensuring the free iterative optimization of normal clear primitives, the geometric structure and transparency distribution of the fuzzy region are stabilized, and finally, the fine-grained and robust optimization of the global Gaussian primitives is completed.

[0060] In the overall iterative optimization framework of Gaussian sputtering rendering, gradient descent iterative updates are continuously performed with the optimization objective of minimizing the adaptive local regularization loss function. For normal primitives, the normal gradient update process retains all their parameters to ensure adaptive fitting of scene details. For the selected blurred primitives, the surface normal orientation is continuously constrained by the constructed regularization loss, constantly correcting surface artifacts and geometric deformations such as primitive orientation disorder, local geometric distortion, image ghosting, and edge blurring caused by photometric ambiguity, strong reflection, and weak supervision and under-constraint. At the same time, the constraint mechanism of freezing scaling parameters and opacity parameters is combined to prevent the blurred primitives from generating invalid geometric drift during the optimization process, so that the primitive geometry and optical parameters in the blurred area gradually converge to the physical features of the real scene. After multiple rounds of iteration until the loss function converges and stabilizes and no longer decreases significantly, the accurate correction and optimization of all Gaussian primitives in the field is finally completed, eliminating the modeling defects caused by ambiguity interference, and outputting a robust scene feature space with regular structure, stable features, and no artifact distortion, providing reliable feature support for subsequent high-precision feature matching and localization tasks.

[0061] In one possible implementation, step S700 further includes: Step S710: Using the set of key points and the set of feature descriptors as input, calculate the Euclidean distance score between the feature descriptors or perform bidirectional matching and filtering based on the mutual nearest neighbor relationship to form an initial candidate matching set containing feature point pairs.

[0062] Step S720: Input the initial candidate matching set into the RANSAC algorithm for iterative calculation. Combine the geometric prior constraints provided by the optimized primitive set and the robust feature space to eliminate outlier mismatch points, output the set of interior points that satisfy the spatial geometric constraints, and determine whether the current image matching state is valid based on the number of interior points, reprojection error and matching stability threshold.

[0063] Specifically, the keypoint set and feature descriptor set of the BEV top-view image, as well as the keypoint set and feature descriptor set of the candidate matching region image, are used as input. For each set of feature descriptor vectors to be compared in the two images, the Euclidean distance score between the descriptors is calculated. The smaller the Euclidean distance value, the higher the similarity between the two sets of features. Then, a mutual nearest neighbor bidirectional matching strategy is used for filtering. For any feature descriptor in the BEV image, the nearest descriptor in the candidate matching region image is found. Conversely, the nearest neighbor feature of the candidate descriptor in the BEV image is searched. Only when the two are each other's nearest neighbors is the set of feature points retained. One-way matching and incorrect matching with excessive distance are eliminated. Finally, an initial candidate matching set containing multiple sets of feature point pairs is formed, which provides original matching samples for subsequent elimination of incorrect matching points and solving the transformation matrix.

[0064] The initial candidate matching set is input into the RANSAC algorithm. A maximum number of iterations and a reprojection error threshold are set. In each iteration, the minimum number of sample points are randomly selected from each candidate matching point pair to solve the homography transformation matrix. During the model validation phase, optimization primitive sets and scene geometric prior constraints carried by the robust feature space are introduced. Local surface normal deviation checks and spatial distance boundary checks are added to the matching points. The reprojection coordinates of each matching feature point after transformation matrix mapping are calculated. The Euclidean distance between the reprojection coordinates and the corresponding feature point coordinates in the target image is taken as the reprojection error. Points with reprojection errors less than the preset error threshold and satisfying the geometric prior constraints are marked. The interior points are identified as inliers, and the remaining points are identified as outliers and removed. The sampling, modeling, and verification processes are continuously iterated until the number of iterations is exhausted or the optimal transformation model with the highest proportion of interior points is found, and the corresponding set of interior points is output. The total number of interior points under the optimal model is counted, and the average reprojection error of all interior points is calculated. Combined with the pre-set lower limit threshold for the number of interior points, the upper limit threshold for the average reprojection error, and the matching stability threshold, a comprehensive judgment is made. When the number of interior points is greater than or equal to the lower limit, the average reprojection error is less than the upper limit, and the stability index meets the threshold requirements, the current image matching state is determined to be valid; otherwise, the matching is determined to be invalid, providing reliable matching interior point data for subsequent coordinate calculation.

[0065] In one possible implementation, step S720 further includes: Step S721: Use the spatial transformation matrix to map the center point or target point of the BEV top view image to the target pixel coordinates of the candidate matching region.

[0066] Step S722: Combining the meter / pixel ratio recorded in the candidate matching region, convert the target pixel coordinates into actual geographical location coordinates represented by eastward and northward distances.

[0067] Step S723: Output includes error evaluation information and positioning results for eastward positioning error, northward positioning error, horizontal comprehensive error, heading angle error, altitude error, total number of candidate matching points, number of points in the RANSAC algorithm, and valid matching status flags.

[0068] Specifically, based on the optimal spatial transformation matrix obtained through iterative optimization using the RANSAC algorithm, the preset image center point or the user-specified target feature point to be matched in the BEV top-view image is determined. The original pixel coordinates of the key point in the BEV image coordinate system are extracted and constructed into homogeneous coordinates to adapt to matrix operation rules. The coordinate perspective mapping and geometric transformation calculation are completed by left multiplying by the optimal spatial transformation matrix, realizing the accurate projection transformation of the BEV image target point from the original image coordinate system to the candidate matching region image coordinate system. Finally, the matching target pixel coordinates of the target point in the candidate matching region image are calculated and output, providing accurate pixel mapping results for subsequent pixel coordinate to geographic real coordinate conversion and positioning calculation.

[0069] The meter / pixel ratio parameter, which represents the physical length of a single pixel, is retrieved from the candidate matching region. The horizontal and vertical differences between the target pixel coordinates and the image origin of the candidate matching region are multiplied by the meter / pixel ratio to obtain the eastward and northward distance components of the target point relative to the image origin. These components are then superimposed with the reference geographic eastward and northward coordinates corresponding to the image origin of the candidate matching region to complete the conversion from pixel coordinates to planar relative coordinates. Finally, the actual geographic location coordinates of the target point, characterized by eastward and northward distances, are output, completing the conversion from pixel space to geographic planar space and providing geographic coordinate data for the final positioning result output.

[0070] The difference between the calculated eastward and northward geographic coordinates of the target point and the pre-acquired true reference coordinates is used to obtain the eastward positioning error. Northward positioning error ,in The eastward positioning error represents the difference between the calculated eastward coordinates and the reference eastward coordinates. The northward positioning error represents the difference between the calculated northward coordinates and the reference northward coordinates, which is then calculated according to the formula. Calculate the overall horizontal error , Characterizes the overall positioning deviation of the planar position; the currently solved heading angle Height value and reference heading angle respectively Reference height Perform the difference calculation to determine the heading angle error. Height error ; Count the total number of candidate matching points generated in this feature matching. and the number of interior points obtained by the RANSAC algorithm. Based on the aforementioned matching validity determination results, a matching validity status flag (Flag) is generated. A Flag of 1 indicates a valid match, and a Flag of 0 indicates an invalid match. The eastward positioning error is then considered. Northward positioning error Horizontal comprehensive error Heading angle error Height error Total number of candidate matching points Number of interior points in RANSAC algorithm It also integrates and encapsulates the matching valid status flag bit, and synchronously outputs the positioning result and the corresponding error evaluation information to realize the accuracy quantification and validity identification of the positioning result.

[0071] To further illustrate the practical effectiveness of the method of this invention, two typical UAV operation scenarios were selected for comparative testing: a rooftop with strong reflectivity and an open field with weak texture. The test UAV flew at an altitude of 80m, the onboard camera had a resolution of 1920×1080, and the ground resolution of the geographic reference image was 0.1m / pixel. The method of this invention was compared with the traditional BEV-RANSAC matching method and the XFeat direct matching method. Five key indicators were statistically analyzed: eastward positioning error, northward positioning error, horizontal comprehensive error, proportion of matched interior points, and matching effectiveness. The experimental results are shown in Table 1. Table 1 Comparison of localization performance of different methods in the test scenario

[0072] As shown in Table 1, in typical scenarios prone to photometric ambiguity, such as strong roof reflection and weak texture in open fields, compared with the traditional BEV-RANSAC matching method and the XFeat direct matching method, this invention significantly improves the proportion of feature matching interior points and reduces the comprehensive positioning errors in the east, north, and horizontal directions by using Gaussian pixel photometric disambiguation, fuzzy primitive recognition, and adaptive local regularization constraints. This results in a significantly improved matching efficiency. In the strong roof reflection scenario, the horizontal comprehensive error of this invention is as low as 0.54m, with a matching interior point ratio of 67.2% and a matching efficiency of 96.0%. In the weak texture in open fields scenario, the horizontal comprehensive error is 0.71m, and the matching efficiency is 92.0%. This demonstrates that this invention can effectively suppress mismatch problems caused by photometric ambiguity in the scene and improve the robustness and positioning accuracy of UAV visual positioning.

[0073] Figure 2 This is a bar chart comparing the overall horizontal error of different algorithms provided in the embodiments of this application. The horizontal axis is divided into two groups: a rooftop with strong reflection and a wilderness with weak texture. Each group corresponds to traditional BEV-RANSAC matching, XFeat direct matching, and the method of this invention, respectively. The vertical axis represents the overall horizontal error in meters. By visually comparing the height of the bars, the differences in positioning errors of the three algorithms in the two test scenarios can be clearly seen. It can be seen that the height of the bars corresponding to the method of this invention is significantly lower than that of the two comparison algorithms, verifying the positioning accuracy advantage of this invention.

[0074] Example 2, based on the same inventive concept as the UAV visual positioning method based on geographic reference image matching in the previous examples, such as... Figure 3 As shown, this application provides a UAV visual positioning system based on geographic reference image matching. The system and method embodiments in this application are based on the same inventive concept. The system includes: The ground image acquisition module 10 is used to acquire ground images collected by the UAV in real time, construct a measurement coordinate system by combining flight status information and camera parameters, and convert the ground images into BEV top view images.

[0075] The candidate matching region acquisition module 20 is used to crop out candidate matching regions from the georeferenced image database based on prior location and field of view.

[0076] The multi-scale feature extraction module 30 is used to extract multi-scale features from the BEV top view image and the candidate matching region respectively, and generate a key point set and a feature descriptor subset.

[0077] The optimized primitive set generation module 40 is used to generate an optimized primitive set for the rendering scene composed of Gaussian primitives in the BEV top view image by directly discarding the edge regions of each Gaussian primitive and applying ray-color consistency weighted variance regularization to perform photometric disambiguation intervention.

[0078] The fuzzy primitive judgment module 50 is used to calculate the ambiguity indicator based on the magnitude of the coefficients of the higher-order spherical harmonic function, and to determine the primitives in the top percentile and bottom percentile as fuzzy primitives in the ambiguity interval.

[0079] The robust feature space output module 60 is used to apply adaptive local regularization constraints to the fuzzy primitives and output the corrected robust feature space.

[0080] The positioning result acquisition module 70 is used to perform candidate matching and filtering on the key point set and feature descriptor subset, and to use adaptive geometric consistency verification to eliminate mismatched points by combining the optimized primitive set and robust feature space. The image coordinates are mapped to the geographic reference image through a spatial transformation matrix, the actual geographic location coordinates are calculated, and error evaluation information and positioning results are output.

[0081] Furthermore, the system is also used to implement the following functions: The altitude, attitude angle, heading angle, and intrinsic and extrinsic parameters of the onboard camera are acquired simultaneously during the UAV's flight. The ground image is then processed sequentially with grayscale conversion, size normalization, and distortion correction based on distortion parameters. The homography transformation matrix is ​​calculated based on the altitude, attitude angle, heading angle, camera intrinsic and extrinsic parameters, and the original image pixels are mapped and converted into top view image pixels to obtain the BEV top view image.

[0082] Furthermore, the system is also used to implement the following functions: The prior position, current flight altitude, and field of view of the UAV are obtained as input parameters for the clipping space. The coverage boundary of the corresponding ground physical size is calculated with the prior position as the center and the current flight altitude and field of view. According to the map resolution, a local roof image covering the coverage boundary is extracted from the georeferenced image database as the candidate matching area, and the map pixel coordinates, real geographic coordinates, and meter / pixel ratio corresponding to the candidate matching area are recorded simultaneously.

[0083] Furthermore, the system is also used to implement the following functions: The BEV top-view image and the candidate matching region are input into the feature extraction network built on XFeat, with the BEV top-view image and the candidate matching region as the network input data; the TensorRT deep learning inference engine is used to accelerate the forward inference of the XFeat feature extraction network, and the corresponding key point set and feature descriptor set are output.

[0084] Furthermore, the system is also used to implement the following functions: The core boundaries of each Gaussian primitive are defined using the standard deviation. When calculating the rendering opacity, the edge regions are directly discarded, and only the contribution of the core regions is retained, thus cutting off the expansion deviation caused by weak gradients. The weighted variance regularization term of the emitted color along the ray relative to the desired blending color is calculated, which forces primitives at the intersection of the same physical surface to have highly similar optical properties, suppressing redundant artifacts, thereby generating the optimized primitive set.

[0085] Furthermore, the system is also used to implement the following functions: Utilizing the property that the variance of viewpoint-related color is proportional to the sum of squares of the coefficients of higher-order spherical harmonic functions, the modulus of the coefficients of higher-order spherical harmonic functions is defined as an ambiguity indicator. A dual-ended indicator module is designed to classify the high-risk geometrical abruptness or strong reflectivity primitives in the first 5% of the ambiguity indicator, as well as the underconstrained primitives in the last 10% that lack photometric supervision, as fuzzy primitives in the ambiguity interval.

[0086] Furthermore, the system is also used to implement the following functions: A pixel-level depth map is obtained from the rendering output of the current scene by a pre-trained depth estimation network. Partial derivatives are performed on the pixel-level depth map to generate corresponding surface normal vectors, which are used as prior constraints for pixel-level normals. For the fuzzy primitive, a parameter-level fine-grained regularization function is constructed with its 3D center position and 3D covariance matrix as input variables. A cosine angle penalty term is calculated between the model rendering normal of the fuzzy primitive and the surface normal vector. A weighted summation of the cosine angle penalty term is introduced using preset weight coefficients to construct an adaptive local regularization loss function. During the gradient backpropagation optimization iteration in the Gaussian sputtering rendering scene, the spatial scaling parameters and opacity parameters of the fuzzy primitive are forcibly frozen to cut off invalid geometric drift caused by weak supervision. Through the minimization iteration of the adaptive local regularization loss function, surface artifacts and geometric deformations caused by photometric ambiguity of the fuzzy primitive are corrected, thereby outputting the robust feature space.

[0087] Furthermore, the system is also used to implement the following functions: Using the set of key points and the set of feature descriptors as input, the Euclidean distance score between the feature descriptors is calculated or bidirectional matching and filtering is performed based on the mutual nearest neighbor relationship to form an initial candidate matching set containing feature point pairs. The initial candidate matching set is input into the RANSAC algorithm for iterative calculation. Combining the set of optimized primitives and the geometric prior constraints provided by the robust feature space, outlier mismatched points are eliminated, and the set of interior points that satisfy the spatial geometric constraints is output. The validity of the current image matching state is determined based on the number of interior points, reprojection error, and matching stability threshold.

[0088] Furthermore, the system is also used to implement the following functions: The spatial transformation matrix is ​​used to map the center point or target point of the BEV top view image to the target pixel coordinates of the candidate matching area; combined with the meter / pixel ratio recorded in the candidate matching area, the target pixel coordinates are converted into actual geographical location coordinates represented by eastward and northward distances; the output includes error evaluation information and positioning results including eastward positioning error, northward positioning error, horizontal comprehensive error, heading angle error, altitude error, total number of candidate matching points, number of points in the RANSAC algorithm, and valid matching status flag bits.

[0089] It should be noted that the order of the embodiments described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. Specific embodiments of this specification have been described above. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0090] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0091] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A UAV visual positioning method based on georeferenced image matching, characterized in that, The method includes: The system acquires real-time ground images collected by the UAV, constructs a measurement coordinate system by combining flight status information and camera parameters, and converts the ground images into a BEV top-view image. Candidate matching regions are cropped from the georeferenced image database based on prior location and field of view; Multi-scale feature extraction is performed on the BEV top view image and the candidate matching region to generate a key point set and a feature descriptor subset; For the rendering scene composed of Gaussian primitives in the BEV top-view image, photometric disambiguation intervention is performed by directly discarding the edge regions of each Gaussian primitive and applying ray-color consistency weighted variance regularization to generate an optimized primitive set. The ambiguity indicator is calculated based on the magnitude of the coefficients of higher-order spherical harmonic functions, and the primitives at the top percentile and bottom percentile are identified as fuzzy primitives in the ambiguity interval. Adaptive local regularization constraints are applied to the fuzzy primitives to output a modified robust feature space; Candidate matching and screening are performed on the key point set and feature descriptor subset. Adaptive geometric consistency verification is used to eliminate mismatched points in combination with the optimized primitive set and robust feature space. The image coordinates are mapped to the geographic reference image through the spatial transformation matrix, the actual geographic location coordinates are calculated, and error evaluation information and positioning results are output.

2. The UAV visual positioning method based on geographic reference image matching as described in claim 1, characterized in that, Acquire real-time ground images from the UAV, construct a measurement coordinate system by combining flight status information and camera parameters, and convert the ground images into a BEV top-view image, including: Simultaneously acquire the altitude, attitude angle, heading angle, and internal and external parameters of the onboard camera during the drone's flight. The ground image is sequentially processed by grayscale conversion, size normalization, and distortion correction based on distortion parameters; Based on the altitude, attitude angle, heading angle, camera intrinsic parameters and extrinsic parameters, the homography transformation matrix is ​​calculated, and the original image pixels are mapped and converted into top view image pixels to obtain the BEV top view image.

3. The UAV visual positioning method based on geographic reference image matching as described in claim 1, characterized in that, Candidate matching regions are cropped from the georeferenced image database based on prior location and field of view, including: The prior position, current flight altitude, and field of view of the drone are obtained as input parameters for the clipping space; Using the prior location as the center, the coverage boundary of the corresponding ground physical dimensions is calculated in combination with the current flight altitude and field of view. Based on the map resolution, a partial roof image covering the coverage boundary is extracted from the georeferenced image database as the candidate matching area, and the map pixel coordinates, real geographic coordinates, and meter / pixel ratio corresponding to the candidate matching area are recorded simultaneously.

4. The UAV visual positioning method based on geographic reference image matching as described in claim 1, characterized in that, Multi-scale feature extraction is performed on the BEV top-view image and the candidate matching region to generate a keypoint set and a feature descriptor subset, including: The BEV top-view image and the candidate matching region are input into a feature extraction network built based on XFeat, with the BEV top-view image and the candidate matching region as network input data; The TensorRT deep learning inference engine is used to accelerate the forward inference of the XFeat feature extraction network, and the corresponding key point set and feature descriptor subset are output.

5. The UAV visual positioning method based on geographic reference image matching as described in claim 1, characterized in that, For the rendered scene composed of Gaussian primitives in the BEV top-view image, photometric disambiguation intervention is performed by directly discarding the edge regions of each Gaussian primitive and applying ray-color consistency weighted variance regularization, generating an optimized primitive set, including: By using the standard deviation to define the core boundary of each Gaussian primitive, the edge region is directly discarded when calculating the rendering opacity, and only the contribution of the core region is retained, thus cutting off the dilation deviation caused by weak gradients. The weighted variance regularization term of the emitted color along the ray relative to the desired mixed color is calculated, which forces the primitives at the intersection of the same physical surface to have highly similar optical properties, suppressing redundant artifacts, thereby generating the optimized primitive set.

6. The UAV visual positioning method based on geographic reference image matching as described in claim 1, characterized in that, The ambiguity indicator is calculated based on the magnitude of the coefficients of higher-order spherical harmonic functions, and primitives at the top and bottom percentiles are identified as fuzzy primitives within the ambiguity interval, including: Taking advantage of the property that the variance of viewpoint-related color is proportional to the sum of squares of the coefficients of higher-order spherical harmonic functions, the modulus of the coefficients of higher-order spherical harmonic functions is defined as an ambiguous indicator. The dual-end indicator module is designed to classify the high-risk geometrical mutations or highly reflective primitives in the first 5% of the ambiguous indicator, as well as the underconstrained primitives in the last 10% that lack photometric supervision, as fuzzy primitives in the ambiguous range.

7. The UAV visual positioning method based on geographic reference image matching as described in claim 1, characterized in that, Adaptive local regularization constraints are applied to the fuzzy primitives to output a modified robust feature space, including: Obtain the pixel-level depth map rendered by the pre-trained depth estimation network for the current scene, perform partial derivative differentiation on the pixel-level depth map to generate the corresponding surface normal vector, and use it as a priori constraint for the pixel-level normal. For the fuzzy primitive, a parameter-level fine-grained regularization function is constructed with its three-dimensional center position and three-dimensional covariance matrix as input variables, and the cosine angle penalty term between the model rendering normal of the fuzzy primitive and the surface normal vector is calculated. A weighted summation of the cosine angle penalty term is introduced by introducing a preset weight coefficient to construct an adaptive local regularization loss function. In the gradient backpropagation optimization iteration process of the Gaussian sputtering rendering scene, the spatial scaling parameter and opacity parameter of the fuzzy primitive are forcibly frozen to cut off the invalid geometric drift caused by weak supervision. By iteratively minimizing the adaptive local regularization loss function, the surface artifacts and geometric deformations caused by photometric polysemy of the fuzzy primitives are corrected, thereby outputting the robust feature space.

8. The UAV visual positioning method based on geographic reference image matching as described in claim 1, characterized in that, Candidate matching is performed on the keypoint set and feature descriptor subset, and erroneous matching points are eliminated by adaptive geometric consistency verification in combination with the optimized primitive set and robust feature space, including: Using the set of key points and the set of feature descriptors as input, calculate the Euclidean distance score between the feature descriptors or perform bidirectional matching and filtering based on the mutual nearest neighbor relationship to form an initial candidate matching set containing feature point pairs; The initial candidate matching set is input into the RANSAC algorithm for iterative calculation. Combining the optimized primitive set with the geometric prior constraints provided by the robust feature space, outlier mismatch points are eliminated, and the set of interior points that satisfy the spatial geometric constraints is output. The validity of the current image matching state is determined based on the number of interior points, reprojection error, and matching stability threshold.

9. The UAV visual positioning method based on geographic reference image matching as described in claim 8, characterized in that, The spatial transformation matrix is ​​used to map the center point or target point of the BEV top view image to the target pixel coordinates of the candidate matching region. Based on the meter / pixel ratio recorded in the candidate matching region, the target pixel coordinates are converted into actual geographical location coordinates represented by eastward and northward distances; The output includes the eastward positioning error, northward positioning error, horizontal composite error, heading angle error, altitude error, total number of candidate matching points, number of points within the RANSAC algorithm, and error evaluation information and positioning results for the valid matching status flag.

10. A UAV visual positioning system based on georeferenced image matching, characterized in that, The system is used to implement the UAV visual positioning method based on georeferenced image matching as described in any one of claims 1-9, and the system comprises: The ground image acquisition module is used to acquire ground images collected by the UAV in real time, construct a measurement coordinate system by combining flight status information and camera parameters, and convert the ground images into BEV top view images; The candidate matching region acquisition module is used to crop out candidate matching regions from the georeferenced image database based on prior location and field of view. A multi-scale feature extraction module is used to extract multi-scale features from the BEV top view image and the candidate matching region respectively, and generate a key point set and a feature descriptor set. The optimized primitive set generation module is used to generate an optimized primitive set for the rendering scene composed of Gaussian primitives in the BEV top view image by directly discarding the edge regions of each Gaussian primitive and applying ray-color consistency weighted variance regularization to perform photometric disambiguation intervention. The fuzzy primitive judgment module is used to calculate the ambiguity indicator based on the magnitude of the coefficients of the higher-order spherical harmonic function, and to determine the primitives in the top percentile and bottom percentile as fuzzy primitives in the ambiguity interval; The robust feature space output module is used to apply adaptive local regularization constraints to the fuzzy primitives and output the corrected robust feature space. The positioning result acquisition module is used to perform candidate matching and filtering on the key point set and feature descriptor subset, and to use adaptive geometric consistency verification to eliminate mismatched points by combining the optimized primitive set and robust feature space. The image coordinates are mapped to the geographic reference image through a spatial transformation matrix, the actual geographic location coordinates are calculated, and error evaluation information and positioning results are output.

Images