UAV localization methods, systems, equipment, and media based on prior model registration
By constructing a priori feature map of the bridge and combining it with inertial prediction results, the problem of high-precision autonomous positioning of UAVs in the environment of satellite signal obstruction at the bottom of the bridge was solved, and the drift-free absolute position and attitude output was achieved, which met the continuous positioning requirements of bridge inspection.
Patent Information
- Application Number
- CN202611375479.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-07
- Publication Date
- 2026-11-17
AI Technical Summary
Drones struggle to achieve high-precision, drift-free autonomous positioning in environments where satellite navigation signals are obstructed, such as at the bottom of bridges. Traditional visual-inertial odometry methods suffer from cumulative positioning error drift and local feature matching failures, failing to meet the requirements for continuous inspection of bridges under absolute spatial coordinates.
By acquiring the three-dimensional prior data of the bridge, a prior feature map of the bridge in a unified coordinate system is constructed. During the flight of the UAV, the three-dimensional depth of the real-time features is determined by using binocular parallax. The spatial constraints of the prior data are introduced to filter the structural features of the bridge. The mapping relationship between the real-time three-dimensional visual features and the prior feature map is established. The absolute visual pose of the UAV is calculated, and the unified pose is output by combining the inertial prediction results.
It effectively eliminates the cumulative drift error in the relative positioning process, ensuring that the UAV obtains high-precision, drift-free absolute position and attitude in the complex and obscured environment under the bridge, thus meeting the continuous inspection requirements of the bridge under absolute spatial coordinates.
Smart Images

Figure CN123118765A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of UAV positioning technology, and in particular to a UAV positioning method, system, device and medium based on prior model registration. Background Technology
[0002] With the widespread application of drones in bridge inspection, structural defect detection, and intelligent infrastructure operation and maintenance, drones are increasingly penetrating narrow and enclosed spaces such as the underside of bridges and inside beams to perform operations. However, the underside of bridges is a typical environment where satellite navigation signals (GNSS) are obscured, and some areas have weak wireless communication signals, drastic changes in lighting, and highly repetitive surface textures of components. This makes high-precision, drift-free autonomous positioning of drones under bridges a key bottleneck restricting the development of intelligent inspection.
[0003] In related technologies, UAVs typically rely on relative positioning techniques such as visual inertial odometry (VIO), pure visual SLAM, or inertial navigation in environments without GNSS signals. These methods mainly rely on the previous frame image or local mapping to recursively deduce the current position. As the flight distance and time increase, the positioning error inevitably accumulates and drifts. At the same time, the large number of similar concrete surfaces and repetitive structures under the bridge can easily cause traditional local feature matching to fail. In addition, these methods lack spatial constraints under an absolute coordinate system, making it difficult to accurately determine the reliability of the current visual positioning result and difficult to achieve autonomous repositioning after positioning is lost, thus failing to meet the continuous inspection requirements of bridges under absolute spatial coordinates. Summary of the Invention
[0004] In view of this, it is necessary to provide a UAV positioning method, system, device and medium based on prior model registration, which can at least overcome one of the above defects.
[0005] In a first aspect, embodiments of this application provide a UAV positioning method based on prior model registration, the method comprising:
[0006] Obtain the three-dimensional prior data of the bridge to be inspected, extract the bridge structural area features from the three-dimensional prior data, and construct the bridge prior feature map under a unified coordinate system.
[0007] During the flight of the UAV, synchronous images, motion acceleration, and angular velocity data of the UAV are collected;
[0008] Feature extraction and binocular parallax calculation are performed on the synchronized images to determine the spatial depth of feature points in the camera coordinate system, and real-time three-dimensional visual features belonging to the bridge structure area are selected based on the spatial constraints of the prior data.
[0009] The selected real-time 3D visual features are matched with the bridge prior feature map to establish a mapping correspondence between the 2D feature points of the bridge structure in the synchronized image and the 3D feature points in the bridge prior feature map.
[0010] Based on the mapping correspondence, the camera imaging projection model is solved, and the absolute visual pose of the UAV relative to the unified coordinate system is calculated.
[0011] Using the absolute visual pose as the observation measurement combined with the inertial prediction result, the unified pose of the UAV in the unified coordinate system is output.
[0012] In one embodiment, the step of filtering out real-time three-dimensional visual features belonging to the bridge structure region based on the spatial constraints of the prior data includes:
[0013] Based on the structural type of the bridge to be inspected, the three-dimensional prior data is divided into multiple structural component regions, and a spatial bounding volume for each structural component region is established in the unified coordinate system.
[0014] Based on the unified pose output from the previous moment and the inertial prediction result obtained by recursion at the current moment, the spatial depth of each feature point in the camera coordinate system is converted into real-time three-dimensional spatial coordinates in the unified coordinate system.
[0015] Determine whether the real-time three-dimensional spatial coordinates fall within the spatial enclosure of any of the structural component regions;
[0016] Background and environmental interference feature points that do not fall within any of the spatial enclosures are removed, while feature points that fall within the spatial enclosures are retained to obtain real-time three-dimensional visual features belonging to the bridge structure area.
[0017] In one embodiment, the step of matching the selected real-time 3D visual features with the bridge prior feature map to establish a mapping correspondence between the 2D feature points of the bridge structure in the synchronized image and the 3D feature points in the bridge prior feature map includes:
[0018] Calculate the descriptor similarity between the selected real-time 3D visual features and each feature point in the bridge prior feature map to obtain candidate matching point pairs;
[0019] Two matching point pairs are randomly selected from the candidate matching point pairs. The first spatial distance between the two corresponding three-dimensional feature points in the prior feature map of the bridge, and the second spatial distance between the two corresponding real-time three-dimensional visual features are calculated.
[0020] When the absolute value of the difference between the first spatial distance and the second spatial distance is greater than the preset geometric deviation threshold, it is determined that there is a local structural mismatch in the candidate matching point pair and it is removed, and the filtered mapping correspondence is output.
[0021] In one embodiment, the step of using the absolute visual pose as an observation and combining it with the inertial prediction result to output the unified pose of the UAV in the unified coordinate system includes:
[0022] The number of matching feature points in the filtered mapping correspondence, the average reprojection error obtained by solving the camera imaging projection model, and the position and attitude deviation between the absolute visual pose and the inertial prediction result are calculated.
[0023] The positioning reliability index at the current moment is calculated by weighting the number of matching feature points, the average reprojection error, and the position and attitude deviation.
[0024] When the positioning confidence index is greater than or equal to a preset threshold, the absolute visual pose is used as the observation of the extended Kalman filter to correct the inertial prediction result and update the unified pose.
[0025] When the positioning confidence index is less than the preset threshold and continues to exceed the preset number of frames, the update of the extended Kalman filter by the absolute visual pose is blocked, the pure inertial state recursion is maintained, and the UAV is controlled to adjust the observation angle to re-perform model registration.
[0026] In one embodiment, extracting bridge structural region features from the three-dimensional prior data and constructing a bridge prior feature map in a unified coordinate system includes:
[0027] Extract structural feature points at the intersection of geometric abrupt change locations and structural topological boundaries from the three-dimensional prior data;
[0028] By associating the position coordinates of each structural feature point with the structural semantic attributes, a priori feature map of the bridge corresponding to the unified coordinate system is generated.
[0029] In one embodiment, after constructing the bridge prior feature map in a unified coordinate system, the method further includes:
[0030] Based on the unified pose output from the previous moment and the current flight state, the estimated viewing area of the UAV in the unified coordinate system is determined.
[0031] Dynamically crop a local prior feature sub-map corresponding to the estimated viewpoint region from the bridge prior feature map;
[0032] The local prior feature sub-map is used as a candidate map reference for matching with the real-time 3D visual features.
[0033] In one embodiment, using the local prior feature sub-map as a candidate map reference for matching the real-time 3D visual features includes:
[0034] Based on the structural semantic attributes and spatial sparsity of each structural feature point, the three-dimensional feature points in the local prior feature sub-map are sorted by saliency and downsampled and removed.
[0035] Based on the preserved 3D feature points and their spatial coordinate relationships, a hierarchical spatial tree index structure is constructed.
[0036] When matching the real-time 3D visual features with the local prior feature sub-map, nearest neighbor feature node retrieval is performed based on the tree-like index structure.
[0037] Secondly, embodiments of this application provide a UAV positioning system based on prior model registration, applied to the method described in the first aspect, the system comprising:
[0038] The prior map construction module is used to acquire three-dimensional prior data of the bridge to be inspected, extract the bridge structural area features from the three-dimensional prior data, and construct a prior feature map of the bridge in a unified coordinate system.
[0039] The data acquisition module is used to acquire synchronous images, motion acceleration, and angular velocity data of the UAV during its flight.
[0040] The feature filtering module is used to extract features and calculate binocular parallax of the synchronized image, determine the spatial depth of feature points in the camera coordinate system, and filter out real-time three-dimensional visual features belonging to the bridge structure area based on the spatial constraints of the prior data.
[0041] The feature matching module is used to match the selected real-time three-dimensional visual features with the bridge prior feature map, and establish a mapping correspondence between the two-dimensional feature points of the bridge structure in the synchronous image and the three-dimensional feature points in the bridge prior feature map.
[0042] The pose calculation module is used to solve the camera imaging projection model based on the mapping correspondence and to calculate the absolute visual pose of the UAV relative to the unified coordinate system.
[0043] The pose fusion module is used to output the unified pose of the UAV in the unified coordinate system by combining the absolute visual pose as the observation measurement with the inertial prediction result.
[0044] Thirdly, embodiments of this application provide an electronic device, including:
[0045] Processor; and
[0046] A memory having computer-readable instructions stored thereon for controlling the processor to perform the method as described in the first aspect.
[0047] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the method described in the first aspect at runtime.
[0048] This application provides a UAV positioning method, system, device, and medium based on prior model registration. It constructs a prior feature map of the bridge in a unified coordinate system by acquiring 3D prior data of the bridge to be inspected. During UAV flight, it uses binocular parallax to determine the 3D depth of real-time features and introduces spatial constraints from the prior data to filter bridge structural features. Subsequently, the filtered real-time 3D features are registered with the prior feature map to establish a 2D-3D mapping relationship to calculate the absolute visual pose. This pose is then used as the observation measurement combined with inertial prediction results to output a unified pose in a unified coordinate system. Compared to existing technologies that rely on local mapping or pure visual-inertial odometry for relative displacement recursion, this application transforms the positioning problem in satellite-free environments into a solution based on absolute geometric constraints of an existing structural model. This fundamentally eliminates the cumulative drift error in the relative positioning process, ensuring that the UAV can continuously acquire high-precision, drift-free absolute position and attitude in complex, obstructed environments under bridges. This effectively solves the technical problem of significant decrease in positioning accuracy with increasing inspection distance in traditional methods. Attached Figure Description
[0049] Figure 1 This is a flowchart illustrating a UAV positioning method based on prior model registration provided in an embodiment of this application.
[0050] Figure 2 This is a schematic diagram of a module of an unmanned aerial vehicle (UAV) positioning system provided in an embodiment of this application.
[0051] Figure 3 This is a schematic diagram of the modules of an electronic device provided in an embodiment of this application. Detailed Implementation
[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.
[0053] It should be noted that, in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.
[0054] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features specified as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0055] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0056] With the widespread application of drones in bridge inspection, structural defect detection, and intelligent infrastructure operation and maintenance, drones are increasingly penetrating narrow and enclosed spaces such as the underside of bridges and inside beams to perform operations. However, the underside of bridges is a typical environment where satellite navigation signals (GNSS) are obscured, and some areas have weak wireless communication signals, drastic changes in lighting, and highly repetitive surface textures of components. This makes high-precision, drift-free autonomous positioning of drones under bridges a key bottleneck restricting the development of intelligent inspection.
[0057] In related technologies, UAVs typically rely on relative positioning techniques such as visual inertial odometry (VIO), pure visual SLAM, or inertial navigation in environments without GNSS signals. These methods mainly rely on the previous frame image or local mapping to recursively deduce the current position. As the flight distance and time increase, the positioning error inevitably accumulates and drifts. At the same time, the large number of similar concrete surfaces and repetitive structures under the bridge can easily cause traditional local feature matching to fail. In addition, these methods lack spatial constraints under an absolute coordinate system, making it difficult to accurately determine the reliability of the current visual positioning result and difficult to achieve autonomous repositioning after positioning is lost, thus failing to meet the continuous inspection requirements of bridges under absolute spatial coordinates.
[0058] In view of this, the UAV positioning method, system, device, and medium based on prior model registration provided in this application construct a prior feature map of the bridge in a unified coordinate system by acquiring 3D prior data of the bridge to be inspected. During the UAV flight, the 3D depth of the real-time features is determined using binocular parallax, and spatial constraints of the prior data are introduced to filter bridge structural features. Subsequently, the filtered real-time 3D features are registered with the prior feature map to establish a 2D-3D mapping relationship to calculate the absolute visual pose, and this is used as the observation measurement combined with inertial prediction results to output a unified pose in a unified coordinate system. Compared with the existing technology that relies on local mapping or pure visual inertial odometry for relative displacement recursion, this application transforms the positioning problem in the absence of satellite signals into a solution based on the absolute geometric constraints of the existing structural model, fundamentally eliminating the cumulative drift error in the relative positioning process. This ensures that the UAV can continuously acquire high-precision, drift-free absolute position and attitude in the complex and obscured environment under the bridge, effectively solving the technical problem that the positioning accuracy of traditional methods decreases significantly with increasing inspection distance.
[0059] This application further combines feature selection with the spatial bounding volume of structural components and geometric topology verification based on dual spatial distances. This can accurately eliminate background environmental interference such as the sky, water surface, and distant view, as well as mismatches caused by repeated textures on the concrete surface, greatly improving the robustness of solving 2D-3D mapping relationships. At the same time, it dynamically crops local prior sub-maps based on the UAV's estimated viewpoint and combines a tree-like index structure to achieve efficient nearest neighbor retrieval, significantly reducing the computational complexity of feature matching for massive maps and meeting the real-time requirements of airborne computing platforms. In addition, by constructing a positioning reliability evaluation mechanism that integrates the number of matching features, reprojection error, and pose deviation, it blocks erroneous updates and actively triggers attitude adjustment and re-registration when there are visual anomalies or strong occlusions, achieving autonomous fault self-healing and continuous high-reliability positioning in the absence of signal under the bridge.
[0060] Figure 1 This is a flowchart illustrating a UAV localization method based on prior model registration, provided in an embodiment of this application. Figure 1The UAV localization method based on prior model registration shown includes at least the following steps: S100: Acquire 3D prior data of the bridge to be inspected, extract bridge structural area features from the 3D prior data, and construct a bridge prior feature map in a unified coordinate system; S200: During UAV flight, collect synchronous images, motion acceleration, and angular velocity data of the UAV; S300: Perform feature extraction and binocular parallax calculation on the synchronous images, determine the spatial depth of feature points in the camera coordinate system, and filter out real-time 3D visual features belonging to the bridge structural area based on the spatial constraints of the prior data; S400: Match the filtered real-time 3D visual features with the bridge prior feature map to establish a mapping correspondence between the 2D feature points of the bridge structure in the synchronous images and the 3D feature points in the bridge prior feature map; S500: Solve the camera imaging projection model based on the mapping correspondence to calculate the absolute visual pose of the UAV relative to the unified coordinate system; S600: Use the absolute visual pose as the observation and combine it with the inertial prediction results to output the unified pose of the UAV in the unified coordinate system.
[0061] S100: Obtain the three-dimensional prior data of the bridge to be inspected, extract the structural features of the bridge from the three-dimensional prior data, and construct a prior feature map of the bridge in a unified coordinate system.
[0062] In this embodiment of the application, the UAV positioning method based on prior model registration includes, in step S100, acquiring three-dimensional prior data of the bridge to be inspected, extracting the bridge structural area features from the three-dimensional prior data, and constructing a bridge prior feature map under a unified coordinate system.
[0063] Specifically, the three-dimensional prior data can originate from the BIM (Building Information Modeling) design model of the bridge to be inspected, the historical UAV inspection reconstruction model, or the three-dimensional point cloud data of LiDAR. First, a unified coordinate system (i.e., the bridge prior coordinate system) is established, where the longitudinal direction of the bridge is set as the X-axis, the transverse direction as the Y-axis, and the vertical direction as the Z-axis. The algorithm extracts geometrical abrupt changes and topological nodes of physical structures such as the edges of main beams, the ends of crossbeams, the corners of piers, and the bearing pads from the three-dimensional prior data, and assigns them corresponding three-dimensional coordinates. Furthermore, the system assigns semantic labels to the bridge structure (such as "main beam" and "pier") based on the bridge structure category. In addition, to alleviate the computational burden of matching massive point clouds during drone flight, the system predicts the drone's estimated viewpoint for the next moment based on the pose and flight speed of the previous moment. It then slices and crops "local prior feature sub-maps" in the global map according to a preset perception radius (e.g., 20 meters) and stores them using a KD-Tree tree-like index structure, greatly accelerating the backend retrieval efficiency.
[0064] Understandably, traditional visual positioning often requires drones to perform real-time mapping (SLAM) while in flight. This is prone to mapping failures or cumulative drift when dealing with large areas of repetitive textures such as concrete. This step utilizes the existing 3D geometry of the bridge as an "offline map," pre-structuring the positional attributes of bridge components. This not only transforms the subsequent online estimation into a "map registration" problem with absolute coordinate references, eliminating cumulative errors, but also avoids full-map searches through dynamic cropping of local submaps and tree-like indexing, significantly reducing the consumption of onboard computing resources.
[0065] S200: During the flight of the UAV, it collects synchronous images, motion acceleration and angular velocity data of the UAV.
[0066] In this embodiment of the application, the UAV positioning method based on prior model registration includes, in step S200, acquiring synchronous images, motion acceleration and angular velocity data of the UAV during its flight.
[0067] Specifically, the binocular vision sensor (including a left-eye camera and a right-eye camera) triggers exposure through hard or soft synchronization at a fixed frame rate (e.g., 20Hz or 30Hz) to acquire the left-eye image at the same time. With right eye image Simultaneously, the inertial measurement unit (IMU) outputs the current acceleration measurement value of the UAV in its own body coordinate system at a higher sampling frequency (e.g., 200Hz). With angular velocity measurement value The two sets of sensor data are precisely synchronized and aligned using hardware timestamps.
[0068] Understandably, binocular vision sensors can physically recover depth information from spatial baselines, providing high-precision 3D geometric observation; while inertial measurement units are unaffected by harsh visual environments such as drastic changes in lighting, lens occlusion, or low texture, and can provide continuous high-frequency motion estimation in a short period of time. The synchronous acquisition of both provides complete physical perception data for subsequent sensor fusion positioning of "high-frequency inertial estimation + low-frequency visual absolute pose correction".
[0069] S300: Performs feature extraction and binocular parallax calculation on synchronized images, determines the spatial depth of feature points in the camera coordinate system, and filters out real-time 3D visual features belonging to the bridge structure area based on the spatial constraints of prior data.
[0070] In this embodiment of the application, the UAV positioning method based on prior model registration includes step S300, which involves extracting features and calculating binocular parallax on the synchronized image, determining the spatial depth of the feature points in the camera coordinate system, and filtering out real-time three-dimensional visual features belonging to the bridge structure area based on the spatial constraints of prior data.
[0071] Specifically, feature extraction is performed on the acquired left and right eye images, and pixel disparity is obtained through a binocular matching algorithm. According to the camera focal length With baseline length Calculate the spatial depth of feature points in the camera coordinate system. This allows for the recovery of the three-dimensional coordinates of the feature points in the camera coordinate system. Subsequently, using the pose output by the system at the previous moment and the IMU recursive results, a transformation matrix is constructed to convert the current camera coordinate system to a unified coordinate system, mapping the feature point coordinates to real-time 3D spatial coordinates in the unified coordinate system. The system uses the "spatial bounding box" that was previously created in the 3D prior model for components such as main beams, crossbeams, and piers as a spatial mask. If the calculated coordinates fall within the bounding box, the feature point is retained; if they fall outside the bounding box (such as distant ground, water reflections, sky, or passing ships), they are directly removed as background noise.
[0072] Understandably, in real-world bridge inspection scenarios, the drone's view is often filled with background interference from non-bridge structures such as water flow, sky, and distant trees. These background features do not exist in the prior map. Direct matching without filtering would lead to an extremely high mismatch rate or even solution scattering. This step achieves "clutter suppression" and "adaptive purification" of features at the physical space level through "prior spatial bounding volume," ensuring that the subsequent registration stage focuses only on the actual bridge structure itself, greatly improving the system's anti-interference capability.
[0073] S400: Match the selected real-time 3D visual features with the bridge prior feature map to establish a mapping correspondence between the 2D feature points of the bridge structure in the synchronous image and the 3D feature points in the bridge prior feature map.
[0074] In this embodiment of the application, the UAV localization method based on prior model registration includes, in step S400, matching the selected real-time three-dimensional visual features with the prior feature map of the bridge, and establishing a mapping correspondence between the two-dimensional feature points of the bridge structure in the synchronous image and the three-dimensional feature points in the prior feature map of the bridge.
[0075] Specifically, the system first retrieves initial "2D image point - 3D prior map point" candidate matching pairs based on the Euclidean distance or Hamming distance between real-time feature descriptors (such as Super Point or traditional feature descriptors) and feature descriptors in the prior map. Then, a geometrically rigid distance invariant constraint is introduced: Let two 3D feature points in the prior map... and The absolute spatial distance is In current binocular observation, the corresponding two real-time 3D feature points and The spatial distance is Because bridges are rigid structures, and Theoretically, they should be equal. If the difference between the two... ( If a preset geometric deviation threshold (e.g., 0.05 meters) is set, it indicates the presence of incorrect matches due to the repetitive height of the concrete surface structure. The system will then remove these abnormal point pairs and ultimately output a high-purity set of 2D-3D mapping correspondences. .
[0076] Understandably, the bridge base contains numerous concrete surfaces, flat beam bottoms, and repetitive components with highly similar appearances, making it highly susceptible to mismatches when relying solely on image appearance descriptors. This step addresses this by combining "geometric rigid topological constraints" with physical spatial distance verification instead of pure appearance comparison, eliminating false matches and ensuring that the established 2D-3D mapping relationship has extremely high geometric consistency and physical reliability.
[0077] S500: Based on the mapping correspondence, solve the camera imaging projection model and calculate the absolute visual pose of the UAV relative to a unified coordinate system.
[0078] In this embodiment of the application, the UAV localization method based on prior model registration includes, in step S500, solving the camera imaging projection model based on the mapping correspondence, and calculating the absolute visual pose of the UAV relative to a unified coordinate system.
[0079] Specifically, the geometric relationship formula for camera imaging perspective projection is as follows:
[0080]
[0081] in, For the camera intrinsic parameter matrix, These are the coordinates of two-dimensional feature points in the image. These are the absolute coordinates of the corresponding 3D feature points in the prior map. The scaling factor is used. The solution process employs EPnP or P3P in conjunction with RANSAC iterative elimination of outliers to minimize the reprojection error.
[0082]
[0083] The solution obtained This refers to the absolute position of the UAV in a unified coordinate system, and the rotation matrix. Convert to quaternion That is, the absolute posture.
[0084] Understandably, traditional visual odometry only calculates the relative displacement "from the previous frame to the current frame," and the error accumulates infinitely over time. This step, however, uses the PnP algorithm to directly project and register real-time image features with a global prior 3D model, calculating the UAV's "absolute coordinates" in the bridge's unified coordinate system. This allows the UAV to obtain a drift-free absolute position at any time, even when flying at great distances in areas without signal coverage.
[0085] S600: Using absolute visual pose as the observation and combining it with inertial prediction results, it outputs the unified pose of the UAV in a unified coordinate system.
[0086] In this embodiment of the application, the UAV localization method based on prior model registration includes step S600, which uses the absolute visual pose as the observation and combines it with the inertial prediction result to output the unified pose of the UAV in a unified coordinate system.
[0087] Specifically, the system maintains a state vector containing position, velocity, quaternion attitude, accelerometer bias, and gyroscope bias. Between adjacent visual frames, continuous inertial state recursion is performed using high-frequency IMU data to obtain the predicted state. When the absolute visual pose calculated in step S500 is received First, calculate the credibility index:
[0088]
[0089] in To match the point factor, This represents the average reprojection error. The Mahalanobis distance between the visual pose and the IMU-predicted pose.
[0090] like (Indicating that the current visual registration is extremely reliable), then the EKF observation update is triggered to correct the accumulated zero bias generated by the IMU and output the filtered unified pose. ;like This indicates that the current vision is affected by strong occlusion or drastic changes in lighting. The system automatically "isolates" this visual observation and maintains flight using only pure inertial recursion. If the reliability is low for multiple consecutive frames, the system triggers active self-healing, controlling the UAV to fine-tune its attitude or reduce its speed to re-find a region with significant features and re-register.
[0091] Understandably, this step constructs a closed-loop self-healing fuse fusion mechanism. On the one hand, it organically combines the smoothness of high-frequency inertial navigation with the absolute accuracy of low-frequency vision; on the other hand, it uses a positioning reliability evaluation mechanism as a "safety valve" to avoid the impact of erroneous visual registration results on the navigation system, ensuring the continuity, robustness, and safety of UAV positioning in complex, degraded signal-free environments under bridges.
[0092] In this embodiment, real-time 3D visual features belonging to the bridge structure area are selected based on the spatial constraints of prior data. This includes: dividing the 3D prior data into multiple structural component regions according to the structural type of the bridge to be inspected, and establishing the spatial bounding volume of each structural component region in a unified coordinate system. Based on the unified pose output from the previous time step and the inertial prediction result recursively obtained at the current time step, the spatial depth of each feature point in the camera coordinate system is converted into real-time 3D spatial coordinates in the unified coordinate system. It is then determined whether the real-time 3D spatial coordinates fall within the spatial bounding volume of any structural component region. Background and environmental interference feature points that do not fall within any spatial bounding volume are removed, while feature points that fall within the spatial bounding volume are retained, thus obtaining the real-time 3D visual features belonging to the bridge structure area.
[0093] Specifically, the system first analyzes the prior BIM or point cloud data of the bridge to be inspected. Based on the geometric outlines of components such as main beams, crossbeams, piers, and supports, it establishes an Oriented Bounding Box (OBB) or Axis-Aligned Bounding Box (AABB) as a spatial bounding volume for each component in a unified coordinate system. During the real-time perception phase, the three-dimensional coordinates of feature points in the camera coordinate system are obtained using a binocular parallax algorithm. Subsequently, the uniform pose output from the previous time step is used. Construct the pose transformation matrix from the current camera coordinate system to a unified coordinate system using the relative pose change derived from the high-frequency integration of the IMU at the current moment. Mapping feature point coordinates to real-time 3D spatial coordinates in a unified coordinate system The system determines collisions using a fast collision detection algorithm. Whether it is contained within the spatial enclosure of any component; if it is determined to be outside the enclosure, the feature point is determined to belong to the background and dynamic interference points such as the sky, water reflection, distant trees or ships sailing under the bridge, and is directly removed.
[0094] Understandably, when drones inspect under bridges, their binocular field of view often contains a large amount of background noise that is not related to the bridge itself. Since these background features do not exist in the prior map, directly involving them in subsequent registration would lead to an extremely high mismatch rate and a waste of computational resources. This step utilizes short-time prior predictions derived from the previous pose and inertial recursion to construct a spatial geometric filter. This performs "physical space noise reduction" before registration calculation, ensuring that subsequent feature matching focuses only on the actual bridge structure itself, greatly improving the ability to resist environmental interference and the efficiency of solution.
[0095] In this embodiment, the selected real-time 3D visual features are matched with a priori feature map of the bridge to establish a mapping correspondence between 2D feature points of the bridge structure in the synchronized image and 3D feature points in the priori feature map. This includes: calculating the descriptor similarity between the selected real-time 3D visual features and each feature point in the priori feature map of the bridge to obtain candidate matching point pairs. Two matching point pairs are randomly selected from the candidate matching point pairs, and the first spatial distance between the two corresponding 3D feature points in the priori feature map of the bridge, and the second spatial distance between the two corresponding real-time 3D visual features are calculated. When the absolute value of the difference between the first spatial distance and the second spatial distance is greater than a preset geometric deviation threshold, it is determined that there is a local structural mismatch in the candidate matching point pair and it is removed. The filtered mapping correspondence is then output.
[0096] Specifically, the system first describes cosine similarity based on features such as Super Point, and then uses nearest neighbor search (such as FLANN) to retrieve candidate matching point pairs in the local prior feature sub-map. Let a candidate matching point pair 1 be a two-dimensional point in the image. With land Figure 3 Dimension Candidate matching point pair 2 is a two-dimensional point in the image. With land Figure 3 Dimension The system calculates the prior map. and Euclidean distance Simultaneously, based on the intrinsic parameters and parallax of the binocular camera, the following calculations were performed: and Calculate the three-dimensional coordinates in the real-time camera coordinate system and the measured spatial distance between the two. Considering that the bridge is a rigid structure, if the point-to-point matching is error-free, and Theoretically, they should remain consistent. The system sets a geometric deviation threshold. (For example, 0.03 meters), if If at least one mismatch exists between point pair 1 and point pair 2, outlier matching point pairs are eliminated iteratively using a consistency graph or maximum clique algorithm, while retaining the matching set with consistent topological relationships, and outputting the final 2D-3D mapping correspondence.
[0097] Understandably, the bridge bottoms contain large areas of similar concrete surfaces, flat beam bottoms, and repetitive components. Relying solely on appearance descriptors (appearance similarity) is highly susceptible to severe mismatches due to texture duplication. This step introduces a "three-dimensional geometric rigid distance invariant" as a physical constraint, using real three-dimensional spatial distance verification to replace pure image appearance comparison. This accurately identifies and eliminates false candidate points with highly similar surfaces but mismatched spatial topology, ensuring that the mapping correspondence passed to the PnP solver has extremely high absolute geometric reliability.
[0098] In this embodiment, the absolute visual pose is used as the observation and combined with the inertial prediction result to output the unified pose of the UAV in a unified coordinate system. This includes calculating the number of matching feature points in the filtered mapping correspondence, the average reprojection error obtained from the camera imaging projection model, and the position and attitude deviation between the absolute visual pose and the inertial prediction result. The positioning confidence index at the current moment is calculated based on a weighted average of the number of matching feature points, the average reprojection error, and the position and attitude deviation. When the positioning confidence index is greater than or equal to a preset threshold, the absolute visual pose is used as the observation of the extended Kalman filter to correct the inertial prediction result and update the unified pose. When the positioning confidence index is less than the preset threshold and continues to exceed a preset number of frames, the update of the extended Kalman filter by the absolute visual pose is blocked, the pure inertial state is maintained, and the UAV is controlled to adjust the observation angle to re-perform model registration.
[0099] Specifically, let the absolute visual pose calculated by PnP at the current time be... The predicted pose obtained by IMU integration is The system calculates the location reliability index using the following weighted formula. :
[0100]
[0101] in, The number of valid 2D-3D matching points after filtering. The average reprojection error obtained by solving for PnP. The Mahalanobis distance between the visual pose and the inertial predicted pose. These are the weighting coefficients. When At that time, the system will As the measurement input to the EKF filter, it updates the position, attitude, and zero bias of the IMU accelerometer and gyroscope in the state vector, and outputs the filtered and corrected unified pose; when If the number of matching points decreases sharply or the residual is too large due to direct strong light or partial occlusion, and the number of consecutive frames exceeds the preset threshold (e.g., 5 frames), the system cuts off the EKF visual update channel and relies solely on the IMU to maintain high-frequency dead reckoning. At the same time, it sends instructions to the flight control system to control the UAV to fine-tune the nose yaw angle or gimbal view and turn to the highly salient structural area marked in the prior map (e.g., crossbeam node) to reacquire rich features to restore registration.
[0102] Understandably, in complex bridge-under environments, visual perception may experience temporary failures or computational anomalies due to sudden shadow changes, lens contamination, or structural occlusion. This step establishes a multi-dimensional weighted positioning reliability assessment mechanism, effectively installing a "safety circuit breaker" for the navigation system to prevent positioning drift caused by out-of-field visual pose contamination filtering. Simultaneously, combined with a self-healing re-registration strategy of "blocking updates + actively fine-tuning the viewpoint," it ensures that the UAV maintains navigation continuity and a high degree of system autonomy even under complex lock-down risks.
[0103] In this embodiment, bridge structural region features are extracted from 3D prior data to construct a bridge prior feature map in a unified coordinate system. This includes: extracting structural feature points at the intersection of geometric abrupt changes and structural topological boundaries in the 3D prior data; associating the position coordinates of each structural feature point with its structural semantic attributes to generate a bridge prior feature map corresponding to the unified coordinate system.
[0104] Specifically, the system first imports the bridge's 3D point cloud, BIM model, or 3D design mesh data, and aligns them uniformly to a unified coordinate system set with the bridge's axis (e.g., longitudinal as the X-axis, transverse as the Y-axis, and vertical as the Z-axis). The algorithm performs curvature analysis and normal vector mutation calculation on the 3D prior data, automatically detecting and extracting geometric mutation locations and structural topology nodes such as the edges of the main beam, the ends of the crossbeams, the corners of the piers, the bearing pads, and the junctions between the beams and crossbeams; simultaneously, it assigns precise spatial coordinates to each extracted feature point in the unified coordinate system. By combining the bridge structural design information with the corresponding structural semantic attributes (such as "main beam edge", "crossbeam node", "pier top" or "support area"), a structured and semantic bridge prior feature map is generated.
[0105] Understandably, the locations of geometrical abrupt changes and topological boundaries are among the most physically significant and viewpoint-invariant regions in a bridge structure, and can still be stably observed under complex conditions such as changes in illumination and occlusion. This step extracts the disordered and redundant raw 3D point cloud data into a high-order prior map containing "absolute spatial coordinates + structural semantic attributes," which not only significantly reduces the map storage volume but also lays a rich data foundation for subsequent semantically constrained adaptive matching and feature selection.
[0106] In this embodiment, after constructing the bridge prior feature map in a unified coordinate system, the method further includes: determining the estimated viewpoint region of the UAV in the unified coordinate system based on the unified pose output from the previous time step and the current flight state; dynamically cropping a local prior feature sub-map corresponding to the estimated viewpoint region from the bridge prior feature map; and using the local prior feature sub-map as a candidate map reference for matching with real-time 3D visual features.
[0107] Specifically, during the UAV inspection process, the system acquires the unified pose of the EKF fusion filter output from the previous moment. Based on the flight velocity vector and attitude change rate collected by the onboard inertial measurement unit (IMU), the predicted pose of the UAV in the unified coordinate system at the current moment is calculated. Subsequently, by combining the camera's field of view (FOV), depth of field, and installation tilt angle, a 3D spatial frustum or a preset perception radius (such as a local sphere with a radius of 15 meters) is constructed as the estimated viewing angle region for the current camera. The system utilizes a spatial bounding volume clipping algorithm to quickly remove useless features outside the viewing angle and at extremely long distances from the global bridge prior feature map, retaining only feature points falling within the estimated viewing angle region, and dynamically generating a "local prior feature sub-map" dedicated to the current frame.
[0108] Understandably, the prior feature map of the entire bridge typically contains a massive number of 3D points. If a full match were performed with the global map every frame during flight, it would consume enormous onboard computing power and cause matching lag. This step, through flight state prediction and spatial viewpoint clipping, narrows the matching search space from the global map to a small local area visible in the current field of view (the number of feature points can be drastically reduced from hundreds of thousands to hundreds). This significantly reduces computational complexity while maintaining registration accuracy, ensuring the algorithm runs in real time on the onboard embedded platform.
[0109] In this embodiment, a local prior feature sub-map is used as a candidate map benchmark for matching with real-time 3D visual features. This includes: ranking and downsampling 3D feature points in the local prior feature sub-map based on their saliency and spatial sparsity, according to the structural semantic attributes and spatial sparsity of each structural feature point. A hierarchical spatial tree-like index structure is constructed based on the retained 3D feature points and their spatial coordinate relationships. When matching real-time 3D visual features with the local prior feature sub-map, nearest neighbor feature node retrieval is performed based on the tree-like index structure.
[0110] Specifically, the system first assigns different weights to feature points based on their structural semantic attributes (e.g., high-significance weights are given to highly identifiable support nodes and beam intersections, while straight main beam edges are given secondary weights). It then uses a voxel grid downsampling algorithm to calculate the spatial sparsity of the feature points. Based on the combined score of saliency weights and sparsity, the feature points in the sub-map are sorted, eliminating redundant and overly dense low-saliency feature points, and retaining high-quality feature points with uniform spatial distribution and significant structure. Next, the system utilizes the absolute coordinates of the retained 3D feature points... Construct hierarchical spatial tree-like index structures such as 3D KD-Tree (k-dimensional tree) or octree; in subsequent matching with real-time 3D visual features, the tree-like index is used for nearest neighbor (NN) or... Neighbors ( -NN) retrieval quickly identifies candidate feature pairs with the closest geometric and descriptor distances.
[0111] Understandably, if feature points are excessively clustered or unevenly distributed in a local submap, the registration results can easily shift towards denser areas and increase invalid retrieval time. This step homogenizes the feature distribution and simplifies the map through "saliency downsampling," and then, combined with a "hierarchical spatial tree index structure," reduces the time complexity of nearest neighbor retrieval from linear to linear. Reduced to logarithmic level It achieves microsecond-level feature pair retrieval, providing core computing power support for high-frequency and accurate absolute visual pose calculation.
[0112] The complete workflow of the UAV localization method based on prior model registration provided in this application is described below with an exemplary embodiment.
[0113] In the offline mapping and preprocessing stage, firstly, 3D prior data such as 3D point clouds or design models of the bridge to be inspected are acquired and uniformly transformed into a bridge prior coordinate system established in the longitudinal, transverse, and vertical directions of the bridge. The algorithm performs curvature and geometric topology analysis on the 3D prior data, extracting feature points at significant locations such as the edges of the main beams, the ends of the crossbeams, the tops of the piers, and the support nodes, and assigning them absolute spatial coordinates and structural semantic labels. Simultaneously, a spatial bounding volume is established based on the geometric outlines of components such as the main beams, crossbeams, piers, and supports, thereby constructing a bridge prior feature map containing absolute geometric information and semantic constraints.
[0114] When the drone flies from an open area into the area under the bridge where there is no satellite navigation signal, the system uses the positioning data before entering the obscured area to establish a correspondence between the drone's initial state and the bridge's prior coordinate system. Subsequently, the drone conducts high-frequency synchronous data acquisition through its onboard binocular vision sensor and airborne inertial measurement unit (IMU). The binocular vision sensor continuously acquires synchronized images from both eyes, while the IMU acquires the drone's acceleration and angular velocity data in real time. The two sets of sensors achieve precise millisecond-level time synchronization alignment through hardware timestamps.
[0115] In the real-time perception and feature selection phase, the system predicts the UAV's estimated pose and field of view region at the current moment based on the unified pose output from the previous moment and the motion data recursively derived from the IMU at the current moment. Using this estimated field of view region, the system dynamically crops a local prior feature sub-map from the global bridge prior feature map and constructs a 3D tree index. Simultaneously, the system performs feature extraction and disparity calculation on the currently acquired binocular images, recovers the spatial depth and 3D coordinates of feature points in the camera coordinate system, and transforms them to the bridge prior coordinate system in conjunction with the estimated pose. By determining whether the real-time feature points fall within the pre-established spatial enclosure of the components, background and dynamic interference points such as the sky, water reflections, and distant ground are removed, retaining only the real-time 3D visual features belonging to the bridge structure itself.
[0116] In the feature matching and absolute pose determination stages, the system performs descriptor similarity retrieval on the selected clean real-time 3D visual features and 3D feature points in the local prior feature sub-map to obtain initial candidate matching point pairs. To eliminate mismatches caused by repeated textures in the concrete under the bridge, the system introduces a geometrically rigid distance invariant constraint, calculating the absolute spatial distance between candidate point pairs in the prior map and the measured spatial distance observed by the binoculars. When the absolute value of the difference between the two distances exceeds a preset geometric deviation threshold, a mismatch is determined and eliminated, outputting a high-purity 2D-to-3D mapping correspondence. Subsequently, based on this mapping relationship, the PnP algorithm is used to solve the camera imaging perspective projection model, calculating the absolute rotation matrix and absolute translation vector of the UAV relative to the bridge's prior coordinate system to obtain the absolute visual pose.
[0117] In the fusion calculation and fault-tolerant control stage, the system comprehensively considers the number of effective matching feature points after screening, the average reprojection error obtained from PnP solution, and the deviation between absolute visual pose and IMU predicted pose, and calculates the positioning confidence index at the current moment using a weighted average. When the positioning confidence index is greater than or equal to a preset threshold, the system uses the absolute visual pose as the observation input to extend the Kalman filter, corrects the inertial state recursion result and updates the sensor zero bias, and fuses the output to obtain the final unified pose of the UAV in the bridge prior coordinate system. When the positioning confidence index is less than the preset threshold, the system cuts off visual observation updates and relies on pure inertial dead reckoning to maintain high-frequency positioning; if the confidence index is low for multiple consecutive frames, the system automatically triggers the re-registration mechanism, controls the UAV to decelerate or adjust the observation angle, turns to a highly salient structural region to rescan and restore model registration, thereby achieving autonomous positioning in a signal-free environment.
[0118] Figure 2 This is a schematic diagram of a drone positioning system module provided in one embodiment of this application. Figure 2 The UAV positioning system 10 shown includes at least the following components: a priori map construction module 11, a data acquisition module 12, a feature filtering module 13, a feature matching module 14, a pose calculation module 15, and a pose fusion module 16.
[0119] In this embodiment, the prior map construction module 11 is used to acquire three-dimensional prior data of the bridge to be inspected, extract the bridge structural area features from the three-dimensional prior data, and construct a prior feature map of the bridge in a unified coordinate system. Please refer to the following for details. Figure 1 The details and their corresponding descriptions are not repeated here.
[0120] In this embodiment, the data acquisition module 12 is used to acquire synchronous images, motion acceleration, and angular velocity data of the UAV during its flight. Please refer to the following for details. Figure 1 The details and their corresponding descriptions are not repeated here.
[0121] In this embodiment, the feature filtering module 13 is used to extract features and calculate binocular parallax on the synchronized image, determine the spatial depth of feature points in the camera coordinate system, and filter out real-time three-dimensional visual features belonging to the bridge structure area based on the spatial constraints of the prior data. Please refer to the following for details. Figure 1 The details and their corresponding descriptions are not repeated here.
[0122] In this embodiment, the feature matching module 14 is used to match the selected real-time 3D visual features with the bridge prior feature map, establishing a mapping correspondence between the 2D feature points of the bridge structure in the synchronized image and the 3D feature points in the bridge prior feature map. Please refer to the following for details. Figure 1 The details and their corresponding descriptions are not repeated here.
[0123] In this embodiment, the pose calculation module 15 is used to solve the camera imaging projection model based on the mapping correspondence, and calculate the absolute visual pose of the UAV relative to the unified coordinate system. Please refer to the following for details. Figure 1 The details and their corresponding descriptions are not repeated here.
[0124] In this embodiment, the pose fusion module 16 is used to combine the absolute visual pose as the observation measurement with the inertial prediction result to output the unified pose of the UAV in the unified coordinate system. Please refer to the following for details. Figure 1 The details and their corresponding descriptions are not repeated here.
[0125] Figure 3 This is an electronic device 20 provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device 20 includes at least the following components: a processor 21 and a memory 22.
[0126] In this embodiment, the memory 22 is used to store executable instructions of the processor 21, which, when configured to execute instructions, implement... Figure 1 The method shown in the figure.
[0127] In one embodiment of this application, the program operating in the electronic device 20 may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). The information processed by these devices is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (Flash ROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.
[0128] In this embodiment of the application, the electronic device 20 may be the drone itself or an onboard processing device (such as an onboard computing chip, flight control motherboard, edge computing terminal or embedded AI computing module, etc.) mounted on the drone body.
[0129] When the electronic device 20 is a drone or an airborne processing device:
[0130] The processor 21 may include at least one or a combination of a central processing unit (CPU), a microcontroller (MCU), a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), or a system-on-a-chip (SoC). The processor 21 is specifically designed for airborne operation of neural network feature extraction algorithms, binocular depth recovery algorithms, PnP pose solving algorithms, extended Kalman filter (EKF) fusion algorithms, and flight control command generation programs, etc.
[0131] The memory 22 is used to store the aforementioned pre-built bridge prior feature map (and dynamically cropped local sub-maps), deep learning network weight parameters, IMU calibration parameters, binocular camera intrinsic parameters and baseline parameters, and program instructions executable by the processor 21.
[0132] In addition, the electronic device 20 also includes a sensor interface and a communication bus connected to the processor 21, for electrically connecting to the airborne binocular vision sensor and airborne inertial measurement unit (IMU) to receive binocular synchronized images and inertial measurement data in real time; and can communicate with the UAV's electronic speed controller (ESC), motor actuators and flight control system to feed back the unified attitude calculated by fusion to the flight control system in real time, so as to realize UAV attitude fine adjustment, active re-registration and closed-loop navigation control.
[0133] Understandably, by embodying the electronic device 20 as the UAV itself or an onboard processing device, the aforementioned set of algorithms—including prior map cropping, real-time depth recovery, spatial bounding volume noise reduction, geometric rigidity verification, absolute pose calculation, and reliability-based circuit breaker fusion—can run directly and locally on the UAV end (onboard end) in real time, without relying on the high-bandwidth wireless communication of a remote ground station or cloud computing resources. In complex environments with severely attenuated or even completely disconnected wireless signals, such as under bridges, the onboard electronic device 20 can achieve fully autonomous, zero-latency end-side closed-loop positioning and navigation, fundamentally ensuring the continuity of UAV operations and flight safety during inspections under bridges.
[0134] It should be noted that a portion of the electronic device 20 described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.
[0135] It should be noted that the term "computer" as used here refers to a computer built into electronic device 20, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard drives built into the computer.
[0136] Furthermore, a "computer-readable recording medium" can include: a medium that dynamically stores a program for a short period of time, such as a communication line used when transmitting a program via a network such as the Internet or a communication line such as a telephone line; or a medium that stores a program for a fixed period of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer.
[0137] It is understood that the UAV positioning method, system, device, and medium based on prior model registration provided in this application construct a prior feature map of the bridge in a unified coordinate system by acquiring three-dimensional prior data of the bridge to be inspected. During the UAV flight, the three-dimensional depth of the real-time features is determined using binocular parallax, and spatial constraints of the prior data are introduced to filter bridge structural features. Subsequently, the filtered real-time three-dimensional features are registered with the prior feature map to establish a 2D-3D mapping relationship to calculate the absolute visual pose, and this is used as the observation measurement combined with the inertial prediction result to output a unified pose in a unified coordinate system. Compared with the existing technology that relies on local mapping or pure visual inertial odometry for relative displacement recursion, this application transforms the positioning problem in the absence of satellite signals into a solution based on the absolute geometric constraints of the existing structural model, fundamentally eliminating the cumulative drift error in the relative positioning process. This ensures that the UAV can continuously acquire high-precision, drift-free absolute position and attitude in the complex and obscured environment under the bridge, effectively solving the technical problem that the positioning accuracy of traditional methods decreases significantly with increasing inspection distance.
[0138] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.
Claims
1. A UAV localization method based on prior model registration, characterized in that, The method includes: Obtain the three-dimensional prior data of the bridge to be inspected, extract the bridge structural area features from the three-dimensional prior data, and construct the bridge prior feature map under a unified coordinate system. During the flight of the UAV, synchronous images, motion acceleration, and angular velocity data of the UAV are collected; Feature extraction and binocular parallax calculation are performed on the synchronized images to determine the spatial depth of feature points in the camera coordinate system, and real-time three-dimensional visual features belonging to the bridge structure area are selected based on the spatial constraints of the prior data. The selected real-time 3D visual features are matched with the bridge prior feature map to establish a mapping correspondence between the 2D feature points of the bridge structure in the synchronized image and the 3D feature points in the bridge prior feature map. Based on the mapping correspondence, the camera imaging projection model is solved, and the absolute visual pose of the UAV relative to the unified coordinate system is calculated. Using the absolute visual pose as the observation measurement combined with the inertial prediction result, the unified pose of the UAV in the unified coordinate system is output.
2. The UAV localization method based on prior model registration according to claim 1, characterized in that, The real-time 3D visual features belonging to the bridge structure region are selected based on the spatial constraints of the prior data, including: Based on the structural type of the bridge to be inspected, the three-dimensional prior data is divided into multiple structural component regions, and a spatial bounding volume for each structural component region is established in the unified coordinate system. Based on the unified pose output from the previous moment and the inertial prediction result obtained by recursion at the current moment, the spatial depth of each feature point in the camera coordinate system is converted into real-time three-dimensional spatial coordinates in the unified coordinate system. Determine whether the real-time three-dimensional spatial coordinates fall within the spatial enclosure of any of the structural component regions; Background and environmental interference feature points that do not fall within any of the spatial enclosures are removed, while feature points that fall within the spatial enclosures are retained to obtain real-time three-dimensional visual features belonging to the bridge structure area.
3. The UAV localization method based on prior model registration according to claim 2, characterized in that, The step of matching the selected real-time 3D visual features with the bridge prior feature map to establish a mapping correspondence between the 2D feature points of the bridge structure in the synchronized image and the 3D feature points in the bridge prior feature map includes: Calculate the descriptor similarity between the selected real-time 3D visual features and each feature point in the bridge prior feature map to obtain candidate matching point pairs; Two matching point pairs are randomly selected from the candidate matching point pairs. The first spatial distance between the two corresponding three-dimensional feature points in the prior feature map of the bridge, and the second spatial distance between the two corresponding real-time three-dimensional visual features are calculated. When the absolute value of the difference between the first spatial distance and the second spatial distance is greater than the preset geometric deviation threshold, it is determined that there is a local structural mismatch in the candidate matching point pair and it is removed, and the filtered mapping correspondence is output.
4. The UAV localization method based on prior model registration according to claim 3, characterized in that, The step of using the absolute visual pose as an observation and combining it with the inertial prediction result to output the unified pose of the UAV in the unified coordinate system includes: The number of matching feature points in the filtered mapping correspondence, the average reprojection error obtained by solving the camera imaging projection model, and the position and attitude deviation between the absolute visual pose and the inertial prediction result are calculated. The positioning reliability index at the current moment is calculated by weighting the number of matching feature points, the average reprojection error, and the position and attitude deviation. When the positioning confidence index is greater than or equal to a preset threshold, the absolute visual pose is used as the observation of the extended Kalman filter to correct the inertial prediction result and update the unified pose. When the positioning confidence index is less than the preset threshold and continues to exceed the preset number of frames, the update of the extended Kalman filter by the absolute visual pose is blocked, the pure inertial state recursion is maintained, and the UAV is controlled to adjust the observation angle to re-perform model registration.
5. The UAV localization method based on prior model registration according to claim 1, characterized in that, The step of extracting bridge structural region features from the three-dimensional prior data and constructing a bridge prior feature map in a unified coordinate system includes: Extract structural feature points at the intersection of geometric abrupt change locations and structural topological boundaries from the three-dimensional prior data; By associating the position coordinates of each structural feature point with the structural semantic attributes, a priori feature map of the bridge corresponding to the unified coordinate system is generated.
6. The UAV localization method based on prior model registration according to claim 5, characterized in that, After constructing the bridge prior feature map in a unified coordinate system, the following is also included: Based on the unified pose output from the previous moment and the current flight state, the estimated viewing area of the UAV in the unified coordinate system is determined. Dynamically crop a local prior feature sub-map corresponding to the estimated viewpoint region from the bridge prior feature map; The local prior feature sub-map is used as a candidate map reference for matching with the real-time 3D visual features.
7. The UAV localization method based on prior model registration according to claim 6, characterized in that, The step of using the local prior feature sub-map as a candidate map reference for matching with the real-time 3D visual features includes: Based on the structural semantic attributes and spatial sparsity of each structural feature point, the three-dimensional feature points in the local prior feature sub-map are sorted by saliency and downsampled and removed. Based on the preserved 3D feature points and their spatial coordinate relationships, a hierarchical spatial tree index structure is constructed. When matching the real-time 3D visual features with the local prior feature sub-map, nearest neighbor feature node retrieval is performed based on the tree-like index structure.
8. A UAV positioning system based on prior model registration, characterized in that, The system, applicable to the method as described in any one of claims 1 to 7, comprises: The prior map construction module is used to acquire three-dimensional prior data of the bridge to be inspected, extract the bridge structural area features from the three-dimensional prior data, and construct a prior feature map of the bridge in a unified coordinate system. The data acquisition module is used to acquire synchronous images, motion acceleration, and angular velocity data of the UAV during its flight. The feature filtering module is used to extract features and calculate binocular parallax of the synchronized image, determine the spatial depth of feature points in the camera coordinate system, and filter out real-time three-dimensional visual features belonging to the bridge structure area based on the spatial constraints of the prior data. The feature matching module is used to match the selected real-time three-dimensional visual features with the bridge prior feature map, and establish a mapping correspondence between the two-dimensional feature points of the bridge structure in the synchronous image and the three-dimensional feature points in the bridge prior feature map. The pose calculation module is used to solve the camera imaging projection model based on the mapping correspondence and to calculate the absolute visual pose of the UAV relative to the unified coordinate system. The pose fusion module is used to output the unified pose of the UAV in the unified coordinate system by combining the absolute visual pose as the observation measurement with the inertial prediction result.
9. An electronic device, characterized in that, include: processor; as well as A memory having computer-readable instructions stored thereon for controlling the processor to perform the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to perform the method described in any one of claims 1 to 7 when executed.
