Unmanned aerial vehicle GPS-free navigation and positioning method based on image AI matching

Abstract

This invention relates to the field of UAV navigation and positioning technology, specifically a GPS-free navigation and positioning method and system for UAVs based on image AI matching. Before flight, a priori map database with geographic coordinate references is established; during flight, a bottom-view camera acquires real-time ground images and performs distortion correction, scale normalization, and feature enhancement; the images to be matched are matched with the map through candidate matching and local registration to obtain a set of matching points, and the image positioning horizontal accuracy factor is calculated by combining the geometric observability of the matching points and the physical scale of camera imaging; based on continuous frame kinematic constraints and imaging geometric consistency, a matching motion consistency factor is calculated, and candidate results are filtered or weighted; finally, the candidate geographic locations are jointly optimized based on the above factors to output the absolute latitude and longitude information of the UAV.

Description

Technical Field

[0001] This invention relates to the field of drone navigation and positioning technology, specifically a GPS-free navigation and positioning method for drones based on image AI matching. Background Technology

[0002] When performing inspections, surveying, security, emergency rescue, and operations in complex environments, drones typically rely on global satellite navigation systems (such as GPS and BeiDou) to obtain their absolute geographical location and perform navigation and control functions such as hovering, flight along designated routes, and automatic return to home. However, in urban canyons, indoor spaces, underground facilities, forest cover areas, and scenarios with strong electromagnetic interference or limited satellite signals, satellite navigation signals are prone to becoming unusable, significantly reducing accuracy, or experiencing unstable positioning. This can prevent drones from obtaining reliable latitude and longitude information, thereby affecting their autonomous navigation capabilities and flight safety.

[0003] To address the aforementioned issues, existing technologies have proposed several alternative positioning and navigation schemes for conditions without or with weak GPS. Firstly, inertial navigation methods based on inertial measurement units (IMUs) calculate the UAV's pose by integrating acceleration and angular velocity. However, these methods inevitably generate cumulative errors, and the positioning deviation amplifies continuously with increasing flight time, making it difficult to consistently output absolute geographic coordinates over the long term. Secondly, techniques based on visual odometry or visual simultaneous localization and mapping (VIO / SLAM) estimate the UAV's trajectory by analyzing relative changes between consecutive images. These methods typically only obtain relative displacement or positional relationships in a local coordinate system, making it difficult to directly output absolute latitude and longitude information consistent with the geographic coordinate system. Furthermore, they are prone to tracking loss or scale drift in environments with repetitive textures, varying lighting, or significant changes in viewing angle. Thirdly, schemes based on ground base stations, beacons, or external positioning infrastructure, while providing positioning references to some extent, often rely on additional environmental deployments, resulting in high costs and limited applicability, failing to meet the needs of flexible UAV deployment and cross-regional operations.

[0004] In recent years, with the development of image processing and artificial intelligence technologies, some existing technologies have attempted to use the matching relationship between ground images and prior maps for visual localization. This involves matching ground images acquired in real-time by drones with satellite imagery or pre-built maps to infer the geographical area where the drone is located. However, existing solutions still have significant shortcomings in engineering applications: on the one hand, the matching results between images and maps are easily affected by repeated features, scale variations, perspective differences, and changes in lighting conditions, and a single match may produce a mismatch or a locally optimal solution; on the other hand, existing technologies typically lack a unified quantitative evaluation mechanism for the geometric reliability and kinematic rationality of the matching results, making it difficult to determine the physical credibility of the obtained latitude and longitude results, which may lead to incorrect localization results still being directly output.

[0005] Furthermore, in practical UAV systems, the flight control system typically relies on the latitude and longitude output by the GPS module and its corresponding positioning quality parameters (such as positioning accuracy indicators and status flags) to determine the reliability of the positioning and decide whether to enter the navigation control state. Most existing GPS-free visual positioning solutions fail to achieve effective compatibility with the existing positioning logic of the flight control system at the communication and protocol levels. This often requires modifications to the flight control algorithm or allows them to be used only as auxiliary reference information, thus limiting their engineering applicability and industrial promotion value. Summary of the Invention

[0006] The purpose of this invention is to provide a GPS-free navigation and positioning method for drones based on image AI matching, so as to solve the technical problems mentioned in the background art.

[0007] Based on the above ideas, the present invention provides the following technical solution:

[0008] A GPS-free navigation and positioning method for drones based on image AI matching includes the following steps:

[0009] S1. Acquire prior map data of the target area and establish a map database before flight, wherein the prior map data has a geographic coordinate reference.

[0010] S2. Real-time ground images are captured by the camera on the bottom of the drone during flight;

[0011] S3. Perform distortion correction, scale normalization and feature enhancement preprocessing on the real-time image to obtain the image to be matched;

[0012] S4. Perform candidate matching and local registration on the image to be matched and the map database to obtain a set of matching points, and calculate the image positioning horizontal accuracy factor based on the geometric observability of the set of matching points and the physical scale of camera imaging.

[0013] S5. Based on the continuous frame kinematic constraints and imaging geometric consistency of the UAV, calculate the matching motion consistency factor, and perform consistency screening or weight reduction on the candidate matching results.

[0014] S6. Based on the image positioning horizontal accuracy factor and the matching motion consistency factor, the candidate geographic location solutions are jointly optimized to obtain the final optimized geographic location set, and the absolute latitude and longitude information of the UAV used for output is determined from it.

[0015] A closed-loop process is established, encompassing prior maps, real-time images, matching and positioning quality assessment, joint optimization, and equivalent output. This enables UAVs to obtain absolute latitude and longitude coordinates, rather than just relative displacement, even in scenarios without or with weak GPS. This fundamentally supports latitude and longitude-based flight control functions such as point positioning, route planning, and return-to-home. By introducing positioning level accuracy factors and matching / motion consistency factors as quality constraints, the positioning results no longer rely on the randomness of a single match. This significantly reduces the risk of mismatches caused by repetitive textures, scale variations, and lighting differences, improving positioning stability and robustness during continuous flight. The latitude and longitude coordinates are output only after joint optimization, ensuring interpretable quality and filterable results. This facilitates the setting of threshold strategies in engineering applications, enabling a safe mechanism of outputting only usable coordinates and suppressing unusable ones.

[0016] Preferably, S4 further includes:

[0017] In the map database, a candidate region search is performed on the image to be matched to obtain at least one candidate map sub-map;

[0018] Feature points are extracted and matched between the image to be matched and the candidate map sub-image to obtain a set of matching points;

[0019] Based on the matching point set, the geometric transformation relationship used for the transformation from pixel coordinates to map plane coordinates is estimated;

[0020] By combining the drone's flight altitude, camera intrinsic parameters, and attitude angle, the pixel error is converted to ground error using physical scale.

[0021] The image positioning level accuracy factor is calculated based on the spatial distribution of the matching point set and the physical scale conversion result.

[0022] By combining candidate region retrieval with local registration, matching efficiency is improved and the computational burden of full-map matching is reduced, making the system more adaptable to the real-time requirements of airborne computing platforms. Through a chain of matching point sets, geometric transformation relationships, and pixel error to ground error conversion, positioning quality assessment is grounded in calculable and quantifiable steps, clarifying the sources of positioning error and facilitating engineering parameter tuning, acceptance testing, and calibration. Introducing flight altitude, camera intrinsic parameters, and attitude angles for physical scale conversion avoids biases caused by relying solely on image similarity to determine reliability, making the quality assessment more consistent with actual ground errors.

[0023] Preferably, the image positioning horizontal precision factor is determined according to the following principles:

[0024] Based on the geometric transformation relationship corresponding to the matching point set, an error propagation model for the geometric transformation parameters is established to calculate the geometric uncertainty determined by the spatial distribution of the matching points. The geometric uncertainty is combined with the feature point pixel positioning error, and a ground projection scale determined by flight altitude, camera focal length, pixel size, pitch angle, and roll angle is further introduced to convert the pixel-level error into a ground horizontal positioning error, thereby obtaining the image positioning horizontal accuracy factor. The more dispersed the matching points are, the smaller the pixel positioning error and the smaller the ground projection scale, the smaller the image positioning horizontal accuracy factor, which indicates a higher positioning reliability.

[0025] By modeling error propagation and geometric uncertainty, the positioning horizontal accuracy factor is no longer an empirical or fixed value, but rather adapts to the spatial distribution of matching points, pixel positioning errors, and projection scale, thus providing a more objective criterion for determining when a match is reliable and when it is not. Converting pixel-level errors to ground horizontal errors facilitates direct alignment with actual task indicators (meter-level, decimeter-level, etc.), making it easier to formulate unified engineering thresholds and performance indicators. The principle that the more dispersed the matching points, the smaller the positioning error is explicitly utilized, guiding the algorithm to select matching results with better geometric conditions and reducing erroneous positioning caused by local texture similarities but poor geometric conditions.

[0026] Preferably, S5 further includes:

[0027] Acquire inertial measurement information of the UAV between two adjacent frames, and calculate short-time displacement prediction or short-time velocity prediction within the time interval;

[0028] The displacement of two adjacent frames in the map coordinate system is calculated from the candidate matching results;

[0029] The displacement is checked for consistency with the short-time displacement prediction or short-time velocity prediction. Combined with the maximum acceleration constraint or maximum turning rate constraint of the UAV, candidate matching results that do not meet the consistency are eliminated or downweighted.

[0030] The consistency factor of the matched motion is calculated based on the consistency test results.

[0031] Introducing kinematic constraints between adjacent frames enables the system to suppress anomalous position jumps, particularly effectively filtering mismatches in cases of repetitive urban textures or similar woodland textures. Using short-term displacement / velocity predictions obtained from inertial measurements as priors, and then performing consistency checks on matched displacements, a stable mechanism that is "controllable in the short term and drift-free in the long term" can be established without relying on GPS. Adding maximum acceleration or maximum turning rate constraints is equivalent to introducing a platform physical realizability boundary, preventing the acceptance of physically impossible jumps in matching results and improving flight safety.

[0032] Preferably, the matching motion consistency factor is determined according to the following principles:

[0033] Based on short-time displacement or short-time velocity predictions obtained from inertial measurement information, the predicted displacement within the time interval is calculated; based on the candidate matching results, the actual matching displacement in the map coordinate system is calculated; the deviation between the predicted displacement and the actual matching displacement is used as the consistency residual, and the allowable residual range corresponding to the maximum acceleration constraint of the UAV is used as the normalization scale; the consistency residual is mapped to a monotonically decreasing consistency score, thereby obtaining the matching motion consistency factor; wherein, the smaller the consistency residual, the larger the matching motion consistency factor, indicating that the candidate matching result is more consistent with the UAV kinematics.

[0034] By solidifying the chain of predicted displacement, actual matching displacement, consistency residual, normalization scale, and consistency score, the consistency factor exhibits a clear directionality: the smaller the residual, the higher the score, enabling the system to be stably used for candidate screening, ranking, and fusion. Using the allowable residual corresponding to the maximum acceleration constraint as the normalization scale ensures that the score naturally matches the flight platform performance, avoiding the problem of threshold non-transferability under different aircraft types and load conditions, thus enhancing versatility. The consistency factor can serve as an online quality monitoring indicator, used to trigger relocation, reduce the automatic flight level, or enter a safety strategy, improving the controllability of the engineering system.

[0035] Preferably, S6 further includes:

[0036] For each candidate matching result, a corresponding geographic location solution is generated, and the geographic location solution is assigned the image positioning level accuracy factor obtained from S4 and the matching motion consistency factor obtained from S4.

[0037] Based on the image positioning level accuracy factor, candidate geographic location solutions are screened by error boundary, and candidate geographic location solutions that exceed the preset error threshold are eliminated;

[0038] Based on the matching motion consistency factor, the candidate geographic location solutions are weighted or sorted according to consistency to determine the geographic location solution to be output;

[0039] Based on the image positioning level accuracy factor, positioning quality parameters for flight control reliability determination are constructed, and together with the output latitude and longitude, they form equivalent GPS output data.

[0040] Assigning two quality factors to candidate geographic locations before screening / ranking improves the decision-making process from single similarity to multi-constraint decision-making, significantly reducing the probability of erroneous latitude and longitude coordinates being output to the flight control system. By first eliminating candidates with large errors through error bounds and then ranking them using consistency factors, both static geometric reliability and dynamic motion rationality can be considered, improving the smoothness and reliability of continuous positioning output. Constructing positioning quality parameters for flight control reliability determination and outputting them along with latitude and longitude coordinates allows the flight control system to judge positioning availability based on its existing logic, thereby achieving safer closed-loop control without modifying the flight control algorithm.

[0041] Preferably, the final optimized set of geographical locations is determined according to the following principles:

[0042] For multiple candidate geographic location solutions, their matching similarity scores are obtained respectively. Combined with the prior position obtained by inertial prediction from the output geographic location at the previous moment, the deviation of each candidate geographic location solution from the prior position is calculated. Using the matching similarity score as a trend term, the reasonableness of the deviation under the constraint of the image positioning level accuracy factor as a penalty term, and the matching motion consistency factor as a consistency gain term, a comprehensive evaluation criterion is constructed. The candidate geographic location solutions that satisfy the image positioning level accuracy factor not exceeding a preset threshold and the matching motion consistency factor not lower than a preset threshold are determined as the final optimized geographic location set, and the candidate geographic location solution with the highest comprehensive evaluation is selected as the optimal solution corresponding to the output latitude and longitude.

[0043] The comprehensive evaluation criteria simultaneously consider matching similarity, deviation from the inertial prior position, and consistency factor, ensuring that the final solution selection is not dominated by a single indicator and significantly reducing the risk of misselection due to high similarity but positional jumps. By setting thresholds for positioning level accuracy and consistency factors to form the final optimized set, a dual-threshold safety mechanism is established, enabling the system to conservatively output or trigger strategies when quality is insufficient, thus improving task reliability. The strategy of first constructing the set and then selecting the optimal solution from it facilitates subsequent expansion to multi-solution fusion or multi-hypothesis tracking without altering the overall claim framework, which is beneficial for subsequent iterations and a series of patent layouts.

[0044] A method for GPS-free navigation and positioning of unmanned aerial vehicles (UAVs) based on image AI matching, comprising the following steps:

[0045] The system includes:

[0046] The map database module is used to store prior map data with geographic coordinate references;

[0047] The image acquisition module is installed on the drone body and is used to acquire real-time ground images during flight;

[0048] The image preprocessing module is used to perform distortion correction, scale normalization, and feature enhancement processing on the real-time ground image to generate an image to be matched;

[0049] The image matching and localization calculation module is used to perform candidate matching and local registration between the image to be matched and the prior map data to obtain a set of matching points, and to calculate the image localization horizontal accuracy factor based on the geometric observability of the set of matching points and the physical scale of camera imaging.

[0050] The kinematic consistency analysis module is used to calculate the matching motion consistency factor based on the kinematic constraints and imaging geometric consistency of continuous frames of UAVs, and to perform consistency screening or weight reduction on candidate matching results.

[0051] The joint optimization and location solution determination module is used to jointly optimize the candidate geographic location solutions based on the image positioning horizontal accuracy factor and the matching motion consistency factor, to obtain the final optimized geographic location set and determine the final absolute latitude and longitude information of the UAV.

[0052] An equivalent GPS output module is used to convert the final absolute latitude and longitude information into equivalent GPS positioning data and output it to the UAV flight control system.

[0053] By modularizing the system, the functional chain of map data, image acquisition, preprocessing, matching and positioning, kinematic consistency, joint optimization, and equivalent output is clearly defined. This approach covers both hardware and software integration solutions, as well as different computing platforms and deployment methods. The inclusion of an equivalent GPS output module allows the system to directly interface with mainstream flight control interfaces, lowering the integration threshold and enhancing engineering value. This is particularly suitable for scenarios with no or weak GPS coverage, such as urban canyons, indoor environments, woodland obstructions, and areas with strong electromagnetic interference.

[0054] The technical solution of the present invention may include the following beneficial effects:

[0055] This invention, in environments with no or weak GPS, directly inverts the absolute latitude and longitude information of a UAV by candidate matching and local registration of prior maps and real-time ground images, and continuously updates the output during flight. This enables the UAV to achieve autonomous navigation functions that rely on absolute coordinates, such as hovering, flight path, and return, even in scenarios where satellite signals are unavailable or significantly reduced in accuracy, such as urban canyons, indoors, woodlands, underground spaces, and strong electromagnetic interference. This significantly expands the application scenarios of UAVs and improves mission feasibility.

[0056] Two types of engineering-interpretable quality constraints are introduced into the positioning link: First, a positioning horizontal accuracy factor is constructed based on the spatial distribution of the matching point set, pixel positioning error, and the ground projection scale determined by flight altitude, camera focal length, pixel size, and attitude angle, realizing the physical dimension conversion from pixel error to ground horizontal error and positioning reliability assessment; Second, a matching motion consistency factor is constructed based on short-time displacement / velocity prediction formed by adjacent frame inertial measurements, combined with the maximum acceleration or maximum turning rate constraint of the UAV, to perform consistency screening and weight reduction on candidate matching results. This can effectively suppress mismatches and position jumps caused by repetitive textures, changes in viewpoint scale, and differences in illumination, reduce the risk of incorrect latitude and longitude output, and improve the positioning stability, robustness, and safety during long-endurance flight.

[0057] The candidate geographic location solutions are jointly optimized based on the positioning level accuracy factor and the matching motion consistency factor to form the final optimized geographic location set. The optimal solution is then selected for output. Simultaneously, positioning quality parameters matching the flight control reliability judgment logic can be constructed, generating equivalent GPS positioning data. This allows the system to achieve transparent access and compatible use without modifying the existing flight control algorithm. This achieves a closed-loop engineering system with quantifiable quality, controllable output, and compatible access, lowering the system integration threshold and facilitating deployment under different aircraft models, computing power platforms, and map sources. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the workflow of a GPS-free navigation and positioning method for unmanned aerial vehicles (UAVs) based on image AI matching and its control method, as described in this invention.

[0059] Figure 2 This is a system block diagram of a GPS-free navigation and positioning method for unmanned aerial vehicles based on image AI matching according to the present invention. Detailed Implementation

[0060] Example 1

[0061] A GPS-free navigation and positioning method for drones based on image AI matching includes the following steps:

[0062] S1. Acquire prior map data of the target area and establish a map database before flight, wherein the prior map data has a geographic coordinate reference.

[0063] S2. Real-time ground images are captured by the camera on the bottom of the drone during flight;

[0064] S3. Perform distortion correction, scale normalization and feature enhancement preprocessing on the real-time image to obtain the image to be matched;

[0065] S4. Perform candidate matching and local registration on the image to be matched and the map database to obtain a set of matching points, and calculate the image positioning horizontal accuracy factor based on the geometric observability of the set of matching points and the physical scale of camera imaging.

[0066] S5. Based on the continuous frame kinematic constraints and imaging geometric consistency of the UAV, calculate the matching motion consistency factor, and perform consistency screening or weight reduction on the candidate matching results.

[0067] S6. Based on the image positioning horizontal accuracy factor and the matching motion consistency factor, the candidate geographic location solutions are jointly optimized to obtain the final optimized geographic location set, and the absolute latitude and longitude information of the UAV used for output is determined from it.

[0068] In one embodiment, the map database for implementing the method is prepared and loaded before flight: prior map data of the target area is imported into the airborne computing platform and a map database is established. The prior map data has a geographic coordinate reference, preferably orthophotos or satellite imagery with geographic reference. The map resolution can be selected from 0.1 m / pixel to 1.0 m / pixel. The map is managed in a tile format, and the tile pixel size can be selected as 512×512 or 1024×1024. Geographic reference information is recorded for each tile, including the latitude and longitude of the upper left corner of the tile, map projection parameters, and pixel resolution. To facilitate online candidate retrieval, global image retrieval features can be pre-calculated for each tile and stored in the database. The global retrieval features can be vectors of dimension 128 to 1024, preferably vectors of dimension 256. Correspondingly, real-time images also calculate vectors of the same dimension for Top-K retrieval, where K can be selected from 5 to 20, preferably 10. If a local map is missing or lacks sufficient texture, the tile can be marked as a low-matchability area and its recall priority can be reduced during the candidate retrieval stage to decrease the probability of triggering a false match.

[0069] In one embodiment, real-time images are acquired by a camera on the bottom of the UAV body. The image resolution can be selected from 640×480 to 1920×1080, preferably 1280×720; the frame rate can be selected from 10 Hz to 30 Hz, preferably 15 Hz to 20 Hz; each frame is accompanied by a timestamp to calculate the time interval between two adjacent frames. The inertial measurement information described in claim 4 is output by the IMU, with a sampling frequency selected from 100 Hz to 400 Hz, preferably 200 Hz. The output fields include at least angular velocity, linear acceleration, and timestamp, and can optionally output attitude angles (pitch angle, roll angle, and yaw angle) or be provided by the attitude calculation module; the flight altitude can be provided by a barometer or ranging device, or by flight control fusion results. For handling anomalies and missing data, the real-time image can be optionally quality-assessed: when an image is overexposed or underexposed (e.g., the image's mean gray level is below 20 or above 235) or blurred (e.g., the Laplacian variance is below a preset threshold), the frame is marked as a low-quality frame and a more conservative selection is adopted in the subsequent candidate quantity and threshold strategy; when the IMU is missing less than 100 ms within a short time window, the most recent valid value can be retained; when the missing data continues to exceed the preset threshold, the consistency factor can be set to a low value and a conservative output can be triggered (e.g., outputting a positioning unavailable state or maintaining the output of the previous moment and reducing the positioning quality parameters) to avoid erroneous latitude and longitude updates.

[0070] In one embodiment, the preprocessing includes at least distortion correction, scale normalization, and feature enhancement. Distortion correction is performed based on camera calibration intrinsics; scale normalization scales the image to a ground sampling scale similar to the map tiles based on the current height and map resolution; feature enhancement may employ CLAHE or Gamma correction (Gamma parameter can be selected from 0.8 to 1.2) to reduce shadow and highlight differences. For subsequent matching and registration, the image to be matched needs to construct a feature representation, which may be: keypoints and descriptors based on ORB or SIFT; or keypoints and descriptors output using a learned local feature network; the upper limit of the number of keypoints may be selected from 1000 to 3000, preferably 2000; the matching distance threshold may be a Hamming distance threshold (e.g., less than 60) or a ratio test (e.g., 0.7 to 0.85) to eliminate false matches.

[0071] Specifically, S4 further includes:

[0072] In the map database, a candidate region search is performed on the image to be matched to obtain at least one candidate map sub-map;

[0073] Feature points are extracted and matched between the image to be matched and the candidate map sub-image to obtain a set of matching points;

[0074] Based on the matching point set, the geometric transformation relationship used for the transformation from pixel coordinates to map plane coordinates is estimated;

[0075] By combining the drone's flight altitude, camera intrinsic parameters, and attitude angle, the pixel error is converted to ground error using physical scale.

[0076] The image positioning level accuracy factor is calculated based on the spatial distribution of the matching point set and the physical scale conversion result.

[0077] The candidate matching and local registration in the S4 refinement step can be implemented as follows: First, candidate region retrieval is performed on the image to be matched in the map database to obtain at least one candidate map sub-image. Candidate retrieval can be performed using a global vector retrieval method, i.e., calculating the global embedding vector of the image to be matched and calculating the cosine similarity with the tile embedding vector to select Top-K candidate tiles; alternatively, a non-learning retrieval method can be used, employing low-dimensional features such as color / gradient histograms for coarse retrieval as candidates or downgrades. Then, feature points are extracted and matched between the image to be matched and the candidate map sub-images to obtain a matching point set; matching can be achieved by combining nearest neighbor matching with ratio testing and cross-validation. Based on the matching point set, the geometric transformation relationship from pixel coordinates to map plane coordinates is estimated, and the geometric transformation relationship can be homography transformation; preferably, RANSAC robust estimation is used to eliminate outliers, with the RANSAC reprojection error threshold selectable to be 2 to 5 pixels and the number of iterations selectable to be 500 to 2000; if the proportion of inliers is less than 0.3 or the number of inliers is less than 15, the candidate can be marked as low confidence and enter the subsequent downweighting or elimination process. After obtaining the geometric transformation relationship, the reference point or matching point in the image to be matched can be projected onto the candidate map sub-map, and the corresponding location can be converted into candidate geographic location solution (latitude and longitude) by combining the tile georeference, thus forming a set of candidate geographic location solutions.

[0078] Specifically, the image positioning horizontal precision factor satisfies the following relationship:

[0079] Let the set of matching points be ;

[0080] The parameter vector estimated by geometric transformation is q, its linearized observation Jacobian is J(P), and the standard deviation of pixel measurement noise is σ. px The matching geometric precision factor is defined as follows:

[0081]

[0082] Where J is the Jacobian matrix of the geometric transformation model, representing the sensitivity of the matching point pixel coordinates to the geometric transformation parameters.

[0083] J T为 The transpose of J.

[0084] (J T J) −1 It reflects the magnitude of the uncertainty of the geometric solution.

[0085] tr(⋅) is the trace of the matrix (the sum of the diagonal elements), used to compress the uncertainty of multidimensional parameters into a scalar quantification index.

[0086] Let the equivalent ground sampling distance of the camera be:

[0087]

[0088] Where h is the flight altitude, p is the pixel size or equivalent pixel scale, f is the focal length, and θ and Φ are the pitch and roll angles, respectively:

[0089]

[0090] The calculation of the image positioning horizontal accuracy factor X is implemented according to the following mechanism and sequence: An error propagation model is established based on the geometric transformation relationship corresponding to the matching point set. The geometric transformation parameters are linearized to obtain the Jacobian matrix J, where the number of rows in J is related to the number of interior points, and the number of columns is related to the dimension of the geometric transformation parameters (e.g., 8-dimensional homography parameters). Based on this, the geometric accuracy factor D is calculated. D is determined by taking the square root of the trace of the inverse matrix of the product of the transpose of J and J'. This D is used to quantify the influence of the spatial distribution of matching points on the uncertainty of the geometric solution; the more uniform and dispersed the matching points are, the smaller D is. Pixel measurement noise standard deviation σ px Used to characterize the pixel localization error of feature points, in pixels, σ px The value can be selected from 0.3 to 2.0 pixels; where σ is used when learning features or SIFT are employed. px The preferred size is 0.3 to 1.0 pixels, and when using ORB, σ px The pixel size can be preferably between 1.0 and 2.0 pixels. σ px This can be obtained through static calibration: in a scene with stable texture, continuously capture multiple frames at the same location and repeatedly match them, calculate the root mean square of the in-point reprojection error and take it as σ. px Or its upper bound. The ground sampling distance (GSD) is used to map the pixel scale to the ground scale, in meters per pixel. Its calculation incorporates flight altitude h (in meters), pixel size p (in meters), focal length f (in meters), and attitude angles (pitch and roll) to correct the projected scale. The attitude angles are used to compensate for the increase in the equivalent pixel ground size caused by tilted shooting. The GSD is obtained accordingly based on the aforementioned relationship. The image positioning horizontal accuracy factor X is determined by D and σ according to the aforementioned relationship. px Multiplying by GSD yields a value in meters, which characterizes the magnitude of the horizontal error of the candidate geographical location solution.

[0091] Specifically, S5 also includes:

[0092] Acquire inertial measurement information of the UAV between two adjacent frames, and calculate short-time displacement prediction or short-time velocity prediction within the time interval;

[0093] The displacement of two adjacent frames in the map coordinate system is calculated from the candidate matching results;

[0094] The displacement is checked for consistency with the short-time displacement prediction or short-time velocity prediction. Combined with the maximum acceleration constraint or maximum turning rate constraint of the UAV, candidate matching results that do not meet the consistency are eliminated or downweighted.

[0095] The consistency factor of the matched motion is calculated based on the consistency test results.

[0096] Specifically, the matching motion consistency factor Y satisfies the following relationship:

[0097] Let the velocity prediction obtained from inertial measurement be Δv. IMU The displacement obtained by the candidate matching in the map coordinate system is Δs. map If the time interval is Δt, then the displacement residual is:

[0098]

[0099] And let the residual metric given by the platform's maneuverability be . Assuming maximum acceleration is a constraint, then:

[0100]

[0101] σ v The allowable displacement residual scale.

[0102] a max The maximum acceleration constraint for the drone.

[0103] e T e is the square norm of the residuals, representing the degree of inconsistency.

[0104] The matching motion consistency factor Y is implemented according to the following mechanism and sequence: Between two adjacent frames, short-time displacement prediction or short-time velocity prediction is calculated based on inertial measurement information within a time interval Δt. Preferably, the predicted displacement is obtained by multiplying the velocity prediction by Δt. This prediction process may include attitude compensation and gravity compensation, that is, converting the machine system acceleration to the navigation system, subtracting gravity, integrating to obtain the velocity increment, and integrating the velocity to obtain the displacement prediction. The displacement Δs between adjacent frames in the map coordinate system is calculated from the candidate matching results. mapThen, the consistency residual e is calculated, where e, as described in claim 5, is the difference between "map displacement and inertial velocity prediction multiplied by the time interval" (either a two-dimensional or three-dimensional vector, in meters). To normalize the residuals and form an interpretable confidence score, an allowable residual scale σ is introduced. v , σ v According to the maximum acceleration constraint a max The result is obtained by multiplying the square of Δt, with the unit being meters. max Maneuverability achievable from the platform: This can be selectively acquired through ground or air testing, performing maximum acceleration maneuvers in windless conditions, recording the peak acceleration and taking the 95th percentile as 'a'. max Alternatively, the maximum horizontal acceleration given in the aircraft specifications can be used. This is determined by e and σ. v The consistency factor Y is obtained according to the exponential relationship. Y takes a value between 0 and 1. The smaller the residual, the closer Y is to 1.

[0105] Specifically, S6 further includes:

[0106] For each candidate matching result, a corresponding geographic location solution is generated, and the geographic location solution is assigned the image positioning level accuracy factor obtained from S4 and the matching motion consistency factor obtained from S4.

[0107] Based on the image positioning level accuracy factor, candidate geographic location solutions are screened by error boundary, and candidate geographic location solutions that exceed the preset error threshold are eliminated;

[0108] Based on the matching motion consistency factor, the candidate geographic location solutions are weighted or sorted according to consistency to determine the geographic location solution to be output;

[0109] Based on the image positioning level accuracy factor, positioning quality parameters for flight control reliability determination are constructed, and together with the output latitude and longitude, they form equivalent GPS output data.

[0110] For each candidate matching result, a corresponding candidate geographic location solution is generated, and this solution is assigned an image positioning level accuracy factor X and a matching motion consistency factor Y. Simultaneously, the matching similarity score S is recorded. Based on X, the candidate geographic location solutions are filtered according to an error boundary, eliminating those exceeding a preset error threshold X. th Candidate solutions, where X th The distance can be selected from 0.5 m to 10 m, preferably from 1 m to 3 m, and can be determined by statistically analyzing the correspondence between X and the true error using GPS calibration data, so that the pass rate of samples with a true error greater than a preset upper limit on the validation set is lower than a preset proportion (e.g., 1%). Candidate solutions are screened for consistency or weighted down based on Y, and a consistency threshold Y can be set. th Y thThe threshold can be selected from 0.2 to 0.8, preferably from 0.5 to 0.7, and is determined through statistical analysis of motion data to balance filtering out jumps and avoiding accidental deletions. The final optimized candidate set Ω is obtained after applying the above thresholds. * The system outputs latitude and longitude when Ω is not empty; when Ω is empty, it can choose to degrade: output a positioning unavailable status and pause updates, or briefly maintain the previous output and set the positioning quality parameters to low confidence, thereby avoiding the injection of erroneous latitude and longitude into the flight controller. When constructing positioning quality parameters based on the image positioning horizontal accuracy factor X and forming equivalent GPS output data, a monotonic mapping method can be selected: the smaller X is, the higher the positioning quality parameters; when X increases, the positioning confidence decreases and triggers the flight controller's own positioning judgment logic. The equivalent GPS output frequency can be selected from 5 Hz to 10 Hz and is sent to the flight controller via serial port or bus.

[0111] Specifically, the final optimized set of geographic locations is determined by the following objective function:

[0112] Let the set of candidate geographic locations be... , where ω k The corresponding matching similarity score is S k And corresponding image positioning horizontal precision factor X k Matching motion consistency factor Y k Then the optimal geographical location solution ω * satisfy;

[0113]

[0114] in The prior position is obtained from the output geographical location of the previous time step through inertial prediction; and the final optimized set is obtained based on this.

[0115]

[0116] Sk represents the similarity score obtained from image matching.

[0117] X k This is the positioning level accuracy factor corresponding to the candidate solution.

[0118] Y k This is the matching motion consistency factor corresponding to the candidate solution.

[0119] X th This is the threshold for the positioning accuracy factor.

[0120] Y th This is the threshold for the consistency factor.

[0121] Each candidate solution in the candidate geographic location set Ω has a matching similarity score S, an image localization level accuracy factor X, and a matching-motion consistency factor Y; the previous output geographic location is used to obtain the prior location through inertial prediction. The comprehensive evaluation comprises three mechanisms: First, lnS represents the contribution of matching similarity in the logarithmic domain; second, a deviation penalty term is formed by dividing the square of the deviation of the candidate solution from the prior position by 2 multiplied by the square of the positioning level accuracy factor. The coefficient form of this term is consistent with the Gaussian prior likelihood, ensuring that the smaller X is (indicating more precise positioning), the stronger the penalty for jumps; third, lnY reflects the motion consistency likelihood, and the smaller Y is, the more negative the logarithm and the stronger the penalty. Thus, the candidate with the highest comprehensive evaluation within Ω is selected as the optimal solution for output latitude and longitude. To ensure that the scale of S is comparable to other terms, S needs to be scaled: optionally, a calibration dataset is collected in a GPS environment, and the correct candidate is defined as having a true position error less than a threshold (e.g., 2 m). The original similarity (such as cosine similarity or interior point score) is subjected to temperature / scale transformation to maximize the Top-1 hit rate of the correct candidate in the validation set under the comprehensive evaluation; the stopping condition can be set as the validation set hit rate improvement being less than 0.2% for several rounds.

[0122] A method for GPS-free navigation and positioning of unmanned aerial vehicles (UAVs) based on image AI matching, comprising the following steps:

[0123] The system includes:

[0124] The map database module is used to store prior map data with geographic coordinate references;

[0125] The image acquisition module is installed on the drone body and is used to acquire real-time ground images during flight;

[0126] The image preprocessing module is used to perform distortion correction, scale normalization, and feature enhancement processing on the real-time ground image to generate an image to be matched;

[0127] The image matching and localization calculation module is used to perform candidate matching and local registration between the image to be matched and the prior map data to obtain a set of matching points, and to calculate the image localization horizontal accuracy factor based on the geometric observability of the set of matching points and the physical scale of camera imaging.

[0128] The kinematic consistency analysis module is used to calculate the matching motion consistency factor based on the kinematic constraints and imaging geometric consistency of continuous frames of UAVs, and to perform consistency screening or weight reduction on candidate matching results.

[0129] The joint optimization and location solution determination module is used to jointly optimize the candidate geographic location solutions based on the image positioning horizontal accuracy factor and the matching motion consistency factor, to obtain the final optimized geographic location set and determine the final absolute latitude and longitude information of the UAV.

[0130] An equivalent GPS output module is used to convert the final absolute latitude and longitude information into equivalent GPS positioning data and output it to the UAV flight control system.

[0131] By modularizing the system, the functional chain of map data, image acquisition, preprocessing, matching and positioning, kinematic consistency, joint optimization, and equivalent output is clearly defined. This approach covers both hardware and software integration solutions, as well as different computing platforms and deployment methods. The inclusion of an equivalent GPS output module allows the system to directly interface with mainstream flight control interfaces, lowering the integration threshold and enhancing engineering value. This is particularly suitable for scenarios with no or weak GPS coverage, such as urban canyons, indoor environments, woodland obstructions, and areas with strong electromagnetic interference.

[0132] The map database module stores tile images and georeferences and provides an index-based reading interface; the image acquisition module acquires real-time images and outputs timestamped frames; the image preprocessing module performs distortion correction, scale normalization, and enhancement on the frames and outputs the images to be matched and their features; the image matching and localization calculation module performs candidate retrieval and local registration and outputs a set of candidate geographic locations, a set of matching points, and similarity scores; the kinematic consistency analysis module obtains inertial measurement information from the IMU and calculates residuals and consistency factors, writes consistency scores for candidate solutions, and performs elimination / reduction; the joint optimization and location solution determination module calculates the positioning horizontal accuracy factor and constructs Ω. * The optimal latitude and longitude are selected based on a comprehensive evaluation relationship. The equivalent GPS output module encapsulates the latitude and longitude and positioning quality parameters into equivalent GPS positioning data and sends it to the flight controller via serial port or bus. The output frequency can be selected from 5 Hz to 10 Hz, and it outputs a low reliability or loss status when positioning is unavailable to cooperate with the flight controller's positioning judgment logic. During system deployment, the map database module and the algorithm module can run on the same airborne computing platform process or be connected via inter-process communication. The camera and IMU inputs can be timestamped to ensure consistent Δt. The flight controller output interface can be set to a fixed baud rate and uses message queue buffering to avoid jitter.

Claims

1. A GPS-free navigation and positioning method for drones based on image AI matching, characterized in that, Includes the following steps: S1. Acquire prior map data of the target area and establish a map database before flight, wherein the prior map data has a geographic coordinate reference. S2. Real-time ground images are captured by the camera on the bottom of the drone during flight; S3. Perform distortion correction, scale normalization, and feature enhancement preprocessing on the real-time image to obtain the image to be matched; S4. Perform candidate matching and local registration on the image to be matched and the map database to obtain a set of matching points, and calculate the image positioning horizontal accuracy factor based on the geometric observability of the set of matching points and the physical scale of camera imaging; S5. Calculate the matching motion consistency factor based on the continuous frame kinematic constraints and imaging geometric consistency of the UAV, and perform consistency screening or weight reduction on the candidate matching results; S6. Perform joint optimization on the candidate geographic location solutions based on the image positioning horizontal accuracy factor and the matching motion consistency factor to obtain the final optimized geographic location set, and determine the absolute latitude and longitude information of the UAV used for output from it.

2. The method for controlling a UAV without GPS navigation and positioning based on image AI matching according to claim 1, characterized in that, S4 further includes: In the map database, a candidate region search is performed on the image to be matched to obtain at least one candidate map sub-map; Feature points are extracted and matched between the image to be matched and the candidate map sub-image to obtain a set of matching points; Based on the matching point set, the geometric transformation relationship used for the transformation from pixel coordinates to map plane coordinates is estimated; By combining the drone's flight altitude, camera intrinsic parameters, and attitude angle, the pixel error is converted to ground error using physical scale. The image positioning level accuracy factor is calculated based on the spatial distribution of the matching point set and the physical scale conversion result.

3. The method for controlling a UAV without GPS navigation and positioning based on image AI matching according to claim 2, characterized in that, The image positioning horizontal precision factor is determined according to the following principles: Based on the geometric transformation relationship corresponding to the matching point set, an error propagation model for the geometric transformation parameters is established to calculate the geometric uncertainty determined by the spatial distribution of the matching points. The geometric uncertainty is combined with the feature point pixel positioning error, and a ground projection scale determined by flight altitude, camera focal length, pixel size, pitch angle, and roll angle is further introduced to convert the pixel-level error into a ground horizontal positioning error, thereby obtaining the image positioning horizontal accuracy factor. The more dispersed the matching points are, the smaller the pixel positioning error and the smaller the ground projection scale, the smaller the image positioning horizontal accuracy factor, which indicates a higher positioning reliability.

4. The method for controlling a UAV without GPS navigation and positioning based on image AI matching according to claim 3, characterized in that, The S5 also includes: Acquire inertial measurement information of the UAV between two adjacent frames, and calculate short-time displacement prediction or short-time velocity prediction within the time interval; The displacement of two adjacent frames in the map coordinate system is calculated from the candidate matching results; The displacement is checked for consistency with the short-time displacement prediction or short-time velocity prediction. Combined with the maximum acceleration constraint or maximum turning rate constraint of the UAV, candidate matching results that do not meet the consistency are eliminated or downweighted. The consistency factor of the matched motion is calculated based on the consistency test results.

5. The method for controlling a UAV without GPS navigation and positioning based on image AI matching according to claim 4, characterized in that, The matching motion consistency factor is determined according to the following principles: Based on short-time displacement or short-time velocity predictions obtained from inertial measurement information, the predicted displacement within the time interval is calculated; based on the candidate matching results, the actual matching displacement in the map coordinate system is calculated; the deviation between the predicted displacement and the actual matching displacement is used as the consistency residual, and the allowable residual range corresponding to the maximum acceleration constraint of the UAV is used as the normalization scale; the consistency residual is mapped to a monotonically decreasing consistency score, thereby obtaining the matching motion consistency factor; wherein, the smaller the consistency residual, the larger the matching motion consistency factor, indicating that the candidate matching result is more consistent with the UAV kinematics.

6. The method for controlling a UAV without GPS navigation and positioning based on image AI matching according to claim 5, characterized in that, S6 further includes: For each candidate matching result, a corresponding geographic location solution is generated, and the geographic location solution is assigned the image positioning level accuracy factor obtained from S4 and the matching motion consistency factor obtained from S4. Based on the image positioning level accuracy factor, candidate geographic location solutions are screened by error boundary, and candidate geographic location solutions that exceed the preset error threshold are eliminated; Based on the matching motion consistency factor, the candidate geographic location solutions are weighted or sorted according to consistency to determine the geographic location solution to be output; Based on the image positioning level accuracy factor, positioning quality parameters for flight control reliability determination are constructed, and together with the output latitude and longitude, they form equivalent GPS output data.

7. The method for controlling a UAV without GPS navigation and positioning based on image AI matching according to claim 6, characterized in that, The final optimized set of geographical locations is determined according to the following principles: For multiple candidate geographic location solutions, their matching similarity scores are obtained respectively, and combined with the prior position obtained by inertial prediction of the output geographic location at the previous time step, the deviation of each candidate geographic location solution from the prior position is calculated. Using the matching similarity score as a trend term, the reasonableness of the deviation degree under the constraint of the image positioning level accuracy factor as a penalty term, and superimposed with the matching motion consistency factor as a consistency gain term, a comprehensive evaluation criterion is constructed. Candidate geographical location solutions that satisfy the condition that the image positioning level accuracy factor does not exceed a preset threshold and the matching motion consistency factor is not lower than a preset threshold are determined as the final optimized geographical location set, and the candidate geographical location solution with the highest comprehensive evaluation is selected as the optimal solution corresponding to the output latitude and longitude.

8. A GPS-free navigation and positioning method for unmanned aerial vehicles (UAVs) based on image AI matching, characterized in that: The method for GPS-free navigation and positioning of unmanned aerial vehicles based on image AI matching as described in any one of claims 1-7 includes: The system includes: The map database module is used to store prior map data with geographic coordinate references; The image acquisition module is installed on the drone body and is used to acquire real-time ground images during flight; The image preprocessing module is used to perform distortion correction, scale normalization, and feature enhancement processing on the real-time ground image to generate an image to be matched; The image matching and localization calculation module is used to perform candidate matching and local registration between the image to be matched and the prior map data to obtain a set of matching points, and to calculate the image localization horizontal accuracy factor based on the geometric observability of the set of matching points and the physical scale of camera imaging. The kinematic consistency analysis module is used to calculate the matching motion consistency factor based on the kinematic constraints and imaging geometric consistency of continuous frames of UAVs, and to perform consistency screening or weight reduction on candidate matching results. The joint optimization and location solution determination module is used to jointly optimize the candidate geographic location solutions based on the image positioning horizontal accuracy factor and the matching motion consistency factor, to obtain the final optimized geographic location set and determine the final absolute latitude and longitude information of the UAV. An equivalent GPS output module is used to convert the final absolute latitude and longitude information into equivalent GPS positioning data and output it to the UAV flight control system.

Images