A dynamic industrial photogrammetry method and system based on a drone
By mounting an industrial camera on a drone and combining visual SLAM and local beam adjustment techniques, the problems of low efficiency, poor accessibility, and insufficient real-time performance of traditional industrial photogrammetry methods are solved, achieving efficient and reliable dynamic measurement.
Patent Information
- Application Number
- CN202611237809.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-03
AI Technical Summary
Traditional industrial photogrammetry methods are inefficient, have poor accessibility, lack real-time performance, and drone measurement solutions rely on the drone's own positioning, making them prone to failure when positioning information is lacking.
By using an industrial camera mounted on a drone, combined with visual SLAM and local bundle adjustment techniques, local optimization is performed through tracking and pose estimation, and scale correction is performed after global adjustment to achieve dynamic measurement.
It improves measurement efficiency and reliability, ensures the stability and accuracy of the calculation when the UAV lacks positioning information, and realizes high-precision dynamic measurement.
Smart Images
Figure CN123028500A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a dynamic industrial photogrammetry method and system based on unmanned aerial vehicles (UAVs), belonging to the field of industrial measurement technology. Background Technology
[0002] Industrial photogrammetry is a non-contact measurement technique that uses a camera to capture two-dimensional images and then employs perspective geometry principles and mathematical calculations to accurately obtain the three-dimensional spatial coordinates of an object. Traditional industrial photogrammetry typically operates offline: coded markers are manually placed, high-resolution SLR cameras are used to take pictures at different stations, and the images are then exported to a computer for offline calculations. This approach suffers from the following drawbacks: 1. Low efficiency: Manual site setup, point marking, and photography are time-consuming and cannot meet the rapid inspection needs of large workpieces (such as wind turbine blades, aircraft skin, and large antennas).
[0003] 2. Poor accessibility: It is difficult to set up cameras or targets manually in high-altitude, confined spaces or hazardous environments (such as hot chambers in nuclear industries or inside large storage tanks).
[0004] 3. Lack of real-time capability: Traditional methods cannot provide real-time feedback on the three-dimensional coordinates of the measured point during the measurement process, making it difficult to guide on-site assembly or dynamic adjustment.
[0005] Although there have been attempts to combine drones with photogrammetry in recent years, most of them follow the "fly-stop-shoot" model, such as "Research on High-Precision Canal Slope Deformation Monitoring Technology Based on UAVs" (Luo Guangguang, Yang Xuhui, Wang Dangqiang, et al., Proceedings of the 2019 Academic Annual Meeting of the China Water Resources Society, Volume 5). In essence, it is still a static measurement. Moreover, the pose calculation of the UAV itself depends on GPS or visual guidance. However, in the industrial internal environment without GNSS, relying solely on visual feature matching is prone to failure due to texture loss. Summary of the Invention
[0006] The purpose of this invention is to provide a dynamic industrial photogrammetry method and system based on unmanned aerial vehicles (UAVs) to solve the problem that current UAV-based industrial measurement calculations rely on the UAV's own positioning, which leads to calculation failure when positioning information is lacking.
[0007] To address the aforementioned technical problems, the present invention provides a dynamic industrial photogrammetry method based on unmanned aerial vehicles (UAVs), which includes the following steps: 1) Select an initial point from the target being measured, and use the initial point as the origin of the world coordinate system. Use the initial point image taken by the industrial camera mounted on the UAV to perform initial value calculation to obtain the initial pose of the industrial camera mounted on the UAV. 2) During the flight of the UAV from the initial point, the industrial camera acquires dynamic images of the target at a set frame rate and processes them to obtain the coded ID corresponding to the coded marker and the center coordinates of the target point; the coded marker and the target point are both markers set on the target. 3) Based on tracking, pose estimation, and forward intersection in visual SLAM, the industrial camera pose and preliminary 3D coordinates of the target point corresponding to each frame of the image are calculated; and local bundle adjustment is used to locally optimize the calculated industrial camera pose and preliminary 3D coordinates of the target point. 4) After the drone completes one lap, the global adjustment principle is used to globally optimize the poses of all industrial cameras and the coordinates of the target points.
[0008] In one possible implementation, the method also includes using a reference scale or control points with known coordinates to perform precise scale correction on the calculated coordinates of all target points.
[0009] In one possible implementation, the initial point is a coded marker point or a control point with known coordinates placed in the target being measured.
[0010] In one possible implementation, the coded markers and target points are both markers with a certain shape and made of reflective material affixed to the target being measured.
[0011] In one possible implementation, the drone is also equipped with an active light source module to provide a reliable light source for industrial camera shooting.
[0012] In one possible implementation, local bundle adjustment is timed. Local bundle adjustment uses the coordinates of all historical poses of the industrial camera and all observed 3D points as optimization variables, and minimizes the pixel error of reprojecting all observed 3D points onto the 2D image as the cost function. By solving this cost function, the poses of each frame and the coordinates of the observed 3D points are optimized and updated. The observed 3D points include the observed coded marker points and target points.
[0013] In one possible implementation, the drone completes a flight based on the initial point on the target being measured, which is re-acquired by an industrial camera.
[0014] In one possible implementation, if the drone flies more than two laps, the average of the results after each correction is taken as the final optimized result.
[0015] In one possible implementation, when multiple scale corrections are used, the average of the multiple scale corrections is taken as the final correction result.
[0016] To address the aforementioned problems, the present invention provides a second aspect of a dynamic industrial photogrammetry system based on a drone, comprising an industrial camera and a computer mounted on the drone, wherein the dynamic industrial photogrammetry method based on a drone employed by the system includes the following steps: 1) Select an initial point from the target being measured, and use the initial point as the origin of the world coordinate system. Use the initial point image taken by the industrial camera mounted on the UAV to perform initial value calculation to obtain the initial pose of the industrial camera mounted on the UAV. 2) During the flight of the UAV from the initial point, the industrial camera acquires dynamic images of the target at a set frame rate and processes them to obtain the coded ID corresponding to the coded marker and the center coordinates of the target point; the coded marker and the target point are both markers set on the target. 3) Based on tracking, pose estimation, and forward intersection in visual SLAM, the industrial camera pose and preliminary 3D coordinates of the target point corresponding to each frame of the image are calculated; and local bundle adjustment is used to locally optimize the calculated industrial camera pose and preliminary 3D coordinates of the target point. 4) After the drone completes one lap, the global adjustment principle is used to globally optimize the poses of all industrial cameras and the coordinates of the target points.
[0017] In one possible implementation, the method also includes using a reference scale or control points with known coordinates to perform precise scale correction on the calculated coordinates of all target points.
[0018] In one possible implementation, the initial point is a coded marker point or a control point with known coordinates placed in the target being measured.
[0019] In one possible implementation, the coded markers and target points are both markers with a certain shape and made of reflective material affixed to the target being measured.
[0020] In one possible implementation, the drone is also equipped with an active light source module to provide a reliable light source for industrial camera shooting.
[0021] In one possible implementation, local bundle adjustment is timed. Local bundle adjustment uses the coordinates of all historical poses of the industrial camera and all observed 3D points as optimization variables, and minimizes the pixel error of reprojecting all observed 3D points onto the 2D image as the cost function. By solving this cost function, the poses of each frame and the coordinates of the observed 3D points are optimized and updated. The observed 3D points include the observed coded marker points and target points.
[0022] In one possible implementation, the drone completes a flight based on the initial point on the target being measured, which is re-acquired by an industrial camera.
[0023] In one possible implementation, if the drone flies more than two laps, the average of the results after each correction is taken as the final optimized result.
[0024] In one possible implementation, when multiple scale corrections are used, the average of the multiple scale corrections is taken as the final correction result.
[0025] The beneficial effects of this invention are as follows: This invention equips an industrial camera on a drone, improving the measurement efficiency of the industrial camera through autonomous drone flight. During the calculation process, it borrows from visual SLAM, no longer relying on the drone's own positioning data. Through tracking and pose estimation, it can estimate the coarse pose of the current frame based on the drone's pose from the previous frame and the feature points extracted in the current frame. Local bundle adjustment is used to locally optimize the calculated industrial camera pose and the preliminary 3D coordinates of the target point. After the drone completes one lap, global adjustment is used to globally optimize all calculated industrial camera poses and target point coordinates. This invention, through a combination of local (adjustment) and global optimization strategies, ensures high accuracy and stability in the dynamic measurement process, avoiding the problem of calculation failure when the drone lacks its own positioning information, thus improving the reliability of the measurement.
[0026] To avoid the problem of inaccurate scale in measurement results caused by scale drift risk in monocular vision, this invention also monitors the reference scale or control points with known coordinates at known distances in the scene in real time. Once this prior information is observed, it is immediately added as a strong constraint to the optimization equation, and the scale of all the calculated target point coordinates is accurately corrected. Attached Figure Description
[0027] Figure 1 This is a schematic diagram of the hardware configuration of the dynamic industrial photogrammetry system based on UAV of the present invention; 1 is the airborne communication module, 2 is the UAV, 3 is the industrial camera, 4 is the active light source module, 5 is the target under test, 6 is the ground station, and 7 is the ground communication module. Detailed Implementation
[0028] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0029] This invention utilizes a drone equipped with a high-brightness active light source (to solve the problems of uneven lighting and feature loss in industrial environments), mimicking the "transfer" and "intersection" logic in traditional industrial photogrammetry, and employs an improved SLAM (Simultaneous Localization and Mapping) real-time decoupled algorithm for computation. The loop closure detection used triggers global loop closure optimization, eliminating accumulated errors; real-time scale correction accurately adjusts the scale of all calculated target point coordinates. Tracking and pose estimation estimate a coarse pose for the current frame based on the drone's pose from the previous frame and feature points extracted in the current frame. Local map construction and association maintain a dynamic "3D point cloud map" containing all target points to be measured.
[0030] Implementation of a Drone-Based Dynamic Industrial Photogrammetry System The UAV-based dynamic industrial photogrammetry system of the present invention includes an industrial camera 3 and a computer (which may be referred to as an airborne computer) mounted on a UAV 2. To ensure uniform lighting conditions during shooting, the UAV is also equipped with an active light source module 4 to provide a reliable light source for the industrial camera. The industrial camera 3 captures images of the target 5 and transmits the captured images to the airborne computer. The airborne computer performs relevant processing based on the received images to obtain measurement results and transmits them to a ground station 6. The UAV 2 and the ground station 6 communicate via a wireless data transmission link, which includes an airborne communication module 1 (hereinafter referred to as the airborne end) and a ground communication module 7 (hereinafter referred to as the ground end).
[0031] In industrial photogrammetry, coded markers are artificial markers with unique numerical codes. They are typically composed of ordinary artificial markers arranged according to specific rules and can be automatically identified and measured by computers. Their main characteristics and functions include: Uniqueness: Each coded marker has a unique ID, facilitating differentiation and matching among a large number of images; Automation: They can be automatically identified and decoded by computers, forming the basis for automated measurement; Dual Use: In measurement, they can serve as both target points and control points for image point matching or stitching between different images. Common types of coded markers include ring-shaped (such as binary coded rings) and point-distributed types.
[0032] The target being tested is a relatively large piece of equipment; the antenna panel is used as the target in the following example. Figure 1 As shown, the coded markers are distributed as evenly as possible across different locations on the target. Apart from the coded markers, the other markers are simply target points; therefore, these markers can be simply referred to as target points. Both the number of coded markers and the number of target points can be adaptively adjusted according to the size of the target, ensuring complete coverage.
[0033] The working process of this system is explained below.
[0034] Step 1: Establish a global baseline.
[0035] 1. Determine the initial point for drone photography and establish the corresponding coordinate system. The initial point of this invention is selected from the deployed coded marker points. For example, a coded marker with ID is selected as the orientation target, and the orientation target is used as the initial point, that is, the origin of the world coordinate system.
[0036] As another implementation method, a control point with known coordinates can be used as the initial point.
[0037] 2. After the drone takes off, it flies to the initial point and uses an industrial camera to collect images containing a sufficient number of coded markers.
[0038] After the drone takes off, the invention first controls it to fly to the vicinity of the initial point, that is, near the directional target. In this way, the industrial camera on the drone can take pictures of the directional target. The drone can be controlled to stop at that location to take multiple pictures, providing more data for subsequent calculations.
[0039] 3. Use images taken by the UAV near the initial point to perform initial value calculations to obtain the initial pose of the UAV near the initial point.
[0040] The core principle of the PNP (Perspective-n-Point) algorithm is to use the geometric relative relationship between known 3D spatial points and their 2D projection points on the image to calculate the position and orientation of the camera when it was taken by using a perspective projection model. In simple terms, it uses several photos taken from different angles with known marker points to infer the camera's pose.
[0041] Therefore, this invention also draws on the PNP algorithm. In addition to an industrial camera, the UAV also carries a computer, referred to as an onboard computer. The industrial camera sends images captured near the initial point to the onboard computer. The onboard computer identifies coded markers in the images in real time and uses the PNP (Perspective-n-Point) algorithm to calculate the initial pose (position and attitude) of the UAV (camera coordinate system) relative to the directional target (world coordinate system). The purpose of the directional target is to define the origin and direction of the local coordinate system; generally, it is not necessary to determine the position of the directional target. If communication with the world coordinate system is required, multiple transformation fixtures are typically measured using instruments such as a total station, and the transformation relationship with the world coordinate system is obtained through common point transformation.
[0042] 4. By using a reference scale or control point coordinates with known distances, the initial point cloud model is given a realistic physical scale, thus solving the scale ambiguity problem of monocular vision.
[0043] Step 2: Dynamic image acquisition and feature extraction.
[0044] 1. During the flight of the drone, an industrial camera is used to continuously collect data on the target to obtain the corresponding video stream.
[0045] The drone flies from the initial point along a preset path or a self-planned path, and collects data on the target at a set frame rate during the flight. The frame rate can be adaptively set according to actual needs, and is generally set to be higher. By collecting 60 frames per second, the relevant video stream can be obtained.
[0046] To avoid shooting errors caused by inconsistent lighting conditions during the shooting process, this invention also installs a high-brightness light source on the drone. During flight, the high-brightness light source is used to actively illuminate the target being tested, ensuring consistent lighting in the industrial environment and enabling the coded marker points and target points to present high contrast in the image.
[0047] 2. Process the frame images in the obtained video stream to obtain the encoding ID corresponding to the encoded marker and the center coordinates of the target point.
[0048] Specifically, the present invention processes each frame of the video stream to extract coded marker points and target points. For the extracted coded marker points, the coded marker points are identified and decoded to obtain their corresponding unique IDs. For the extracted target points (reflection markers or natural feature points of objects), it is necessary to identify their sub-pixel center coordinates.
[0049] Step 3: Real-time dynamic calculation and map construction.
[0050] This step draws on and improves upon the backend optimization ideas of visual SLAM, constructing a solution loop specifically for industrial measurement.
[0051] Visual SLAM, or visual simultaneous localization and mapping, refers to a technology that uses only a camera as the core sensor, enabling a robot to simultaneously estimate its own motion and build an environmental map in an unknown environment. To better facilitate computation in industrial measurement processes, this invention applies visual SLAM to industrial measurement. The following is a detailed description of the application of visual SLAM in UAV-based industrial measurement.
[0052] 1. Tracking and pose estimation: Based on the UAV pose of the previous frame and the feature points extracted in the current frame, estimate the coarse pose of the current frame.
[0053] Step two yields the extracted feature points for each frame, including coded markers, their corresponding unique IDs, and the target point. The pose of the first frame is directly provided by the initial value calculation in step one. For subsequent frames (second, third, etc.), the pose of the previous frame is used as the initial value, combined with the feature points extracted in the current frame (matched with points in the previous frame or on the map), and the PnP algorithm is used to estimate the coarse pose of the current frame, thus achieving iterative tracking. Iterative calculations in this manner yield the pose for each frame.
[0054] Because industrial cameras have a high frame rate and minimal displacement between adjacent frames, triangulation errors are large. Therefore, this invention discards redundant frames and avoids processing every frame. Only when the UAV's displacement or attitude changes significantly relative to the previous keyframe (e.g., displacement exceeds a set threshold, or rotation angle exceeds a set threshold) is the current frame selected as a keyframe for subsequent forward intersection and local bundle adjustment. This ensures both computational efficiency and baseline accuracy in 3D reconstruction.
[0055] 2. Local map construction and association: Maintain a dynamic "3D point cloud map" that contains all target points to be measured.
[0056] In visual SLAM, a local map can be understood as a "dynamic working area" centered on the current camera position. It stores partial information observed at the current moment and provides local constraints for real-time attitude tracking by associating data with the current frame.
[0057] A local map consists of keyframes and map points. Keyframes are a select few representative frames chosen from an image sequence to efficiently represent the environment. Map points are 3D spatial points in the environment, generated by triangulation from keyframes, representing the geometric structure of the environment. In this invention, a keyframe refers to a representative image frame selected from a continuous video stream based on a spatial motion baseline (e.g., the drone's displacement or rotation angle exceeding a preset threshold). The core purpose of selecting keyframes is to ensure triangulation accuracy (the longer the baseline, the more accurate the depth calculation) and control the computational load of backend optimization. In this invention, map points refer broadly to all three-dimensional spatial points calculated through multi-view forward intersection.
[0058] 3. Real-time forward intersection: When the same physical point is observed in multiple consecutive frames at different locations, forward intersection is performed to calculate the preliminary three-dimensional coordinates of the point and insert them into the map.
[0059] In visual SLAM, forward intersection usually refers to triangulation, which is a fundamental and core step in visual SLAM. Its purpose is to calculate the 3D coordinates of the same spatial point from different observations. This process can be broken down into the following steps: 1) Establish correspondence: In images taken at different times, find the pixels that represent the same physical point in space, thus completing feature matching; 2) Forming an observation ray: For each pair of matching feature points, connect the pixel to the position (optical center) of the camera that captured the image, forming a "line of sight" or "observation ray" that starts from the camera, passes through the pixel, and extends into the three-dimensional space. Theoretically, this spatial point is on this ray. 3) Solving for the 3D intersection point: When the same point is observed from multiple locations, multiple observation rays are obtained. Ideally, these rays will precisely correspond to the same point in space. This intersection point is the 3D point to be solved for. The coordinates of the 3D point can be obtained by minimizing the 3D distance or minimizing the reprojection error. Minimizing the 3D distance means finding a 3D point that minimizes the sum of the perpendicular distances to all observation rays. Minimizing the reprojection error involves reprojecting the candidate 3D point onto various image planes and adjusting its position so that the sum of the distances between the reprojected position and the original observation pixel is minimized.
[0060] In this invention, the matching of identical target points across multiple image frames is achieved in the following way: 1. Image feature matching: The system extracts target points (including non-coded marker points) from adjacent frames and performs similarity measurement based on their local image grayscale neighborhood or feature descriptors, or uses optical flow tracing to establish inter-frame correspondence, thereby determining whether pixels in different frames correspond to the same physical spatial point.
[0061] 2. Epipolar geometric constraint verification: Using the known camera pose and fundamental matrix, the preliminary matching results are screened using epipolar constraints to remove mismatched point pairs that do not conform to the epipolar geometric relationship.
[0062] 3. Spatial Proximity Consistency Check: For target points whose matching is blurred due to similar textures, stable coded markers in their neighboring regions can be used as reference benchmarks. By judging whether the relative distribution structure of the target point and its neighboring coded markers remains similar under different viewpoints, the correctness of the matching can be further confirmed. This strategy is only an auxiliary verification method and does not replace the main matching process.
[0063] When the same physical target point is successfully matched in multiple consecutive image frames at different locations, the system calculates the preliminary three-dimensional coordinates of the point through multi-view forward intersection (triangulation) and inserts them into the local three-dimensional map.
[0064] 4. Local bundle adjustment: Local optimization is triggered at regular intervals.
[0065] Local bundle adjustment is a core technique in visual SLAM used for real-time and efficient optimization of camera poses and 3D map points within local areas. It performs fine-tuning only on the latest, local data to keep computational load within acceptable limits for real-time systems while maintaining map accuracy. Its core objective is to combat the accumulation of local errors. In visual SLAM, as the camera moves, newly added poses and map points always contain minor errors. This algorithm aims to suppress the accumulation of errors through repeated local corrections before they spread to the entire map.
[0066] For this invention, the coordinates of the same point in different frames measured by the industrial camera should also be at the same point when projected onto the 2D image. However, due to accumulated errors, the 3D coordinates of the same point measured are not at the same point when projected onto the 2D image. Based on this, this invention uses the coordinates of all historical poses of the UAV (equivalent to the exterior orientation elements in photogrammetry) and all 3D points (including observed coded marker points and target points) as optimization variables, and minimizes the pixel error of all observed 3D points (including observed coded marker points and target points) reprojected onto the 2D image as the cost function. By solving this cost function, the pose of each frame and the coordinates of the observed 3D points can be optimized and updated.
[0067] This process utilizes the graph optimization framework of visual SLAM to implement a dynamic version of "bundle adjustment," ensuring measurement accuracy in local areas.
[0068] Step 4: Global optimization and scale correction.
[0069] 1. Closed-loop detection: When the drone returns to the vicinity of its initial position after flying a circle, the system recognizes the previously observed coded flags, triggering global closed-loop optimization to eliminate accumulated errors.
[0070] If the target reappears in the image measured by the industrial camera during unmanned flight, it indicates that the drone has returned to its initial position, meaning it has completed one lap. At this point, global optimization is required, employing the adjustment principle. The target is first observed and added to the map in the first frame (initial frame), and its three-dimensional spatial coordinates are defined as the origin of the world coordinate system, denoted as A0. After the drone completes one lap, in the... NWhen the target is observed again in the loopback frame, the system recalculates the target's spatial coordinates, denoted as A1, using forward intersection based on the currently accumulated estimated camera pose. Theoretically, A1 should coincide with A0, but due to the accumulated drift error in inter-frame pose estimation, there will be a deviation between A1 and A0. Therefore, this invention initiates global optimization. Specifically, the system treats the target observation in the loopback frame as a strong constraint term, incorporating it into the objective function of Global Bundle Adjustment. This optimization equation uses the camera poses of all keyframes and the 3D coordinates of all map points (including coded markers and ordinary target points) as optimization variables, and minimizes the sum of squared reprojection errors of all observed map points on their corresponding image frames as the cost function. Simultaneously, it mandates that the reprojection position of the target observed in the loopback frame must be consistent with its observation in the initial frame.
[0071] In actual measurement, the number of times the drone flies can be multiple, such as three times. In this case, when making corrections, the average of the results after each correction can be used as the final optimized result, thereby obtaining the coordinates of each target point in global optimization, which realizes the measurement of the target.
[0072] 2. Real-time scale correction: Performs precise scale correction on the coordinates of all calculated target points.
[0073] Because monocular vision is susceptible to scale drift, leading to scale distortion in the measured target, this invention requires scale correction of the measurement results. This implementation uses a reference ruler or control points with known coordinates for scale correction. The core principle is to provide a known, high-precision physical length as a reference for reconstructing a "unitless" 3D model using monocular SLAM. The system monitors the reference ruler or control points with known coordinates at known distances in the scene in real time. Once this prior information is observed, it is immediately added as a strong constraint to the optimization equation. For example, the distance between the two endpoints of the reference ruler must be equal to its known physical length as a strong constraint during adjustment. This distance constraint must be satisfied while optimizing the camera pose and 3D point positions, thus ensuring the correct scale value of the measured target is obtained. If scale correction is performed multiple times, the average of the multiple corrections is used as the final correction result.
[0074] As another implementation, the scale correction of the present invention can also be used when establishing a global reference in step one, by using a reference ruler or control point coordinates at a known distance to give the directional target a true physical scale, thereby solving the scale ambiguity problem of monocular vision.
[0075] Step 5: Results Output and Visualization.
[0076] The drone sends the calculated 3D coordinates of the target point to the ground station, which receives and displays the calculated 3D coordinates, accuracy indicators, and point cloud model of the target point in real time, making it easy for staff to observe intuitively.
[0077] As described above, the UAV-based dynamic industrial photogrammetry system of this invention mounts an industrial camera onto a UAV, which can fly autonomously, eliminating the need for manual camera setup at each station. This reduces measurement time by more than 50%, significantly improving measurement efficiency. The UAV can easily reach high-altitude and hazardous areas, expanding the application scenarios of industrial photogrammetry. The measurement process and result calculation are performed simultaneously, enabling online inspection and dynamic guided assembly with strong real-time performance. Through a combined strategy of local BA (adjustment), global optimization, and scale correction, this invention ensures high precision and stability in the dynamic measurement process, achieving sub-millimeter accuracy.
[0078] Implementation of Dynamic Industrial Photogrammetry Method Based on UAVs The UAV-based dynamic industrial photogrammetry method of this invention first selects an initial point on the target, using it as the origin of the world coordinate system. Initial values are calculated using images of this initial point captured by an industrial camera mounted on the UAV to obtain the initial pose of the camera. Then, during the UAV's flight from the initial point, the industrial camera acquires dynamic images of the target at a set frame rate, which are then processed to obtain the coded IDs corresponding to coded marker points and the center coordinates of the target points. Both the coded marker points and the target points are marker points set on the target. Next, based on tracking, pose estimation, and forward intersection in visual SLAM, the industrial camera pose and preliminary 3D coordinates of the target points corresponding to each frame are calculated. Local bundle adjustment is then used to locally optimize the calculated industrial camera pose and preliminary 3D coordinates of the target points. Finally, after the UAV completes one lap, global adjustment is used to globally optimize all calculated industrial camera poses and target point coordinates. Simultaneously, a reference ruler or control points with known coordinates can be used to precisely correct the scale of all calculated target point coordinates. The specific implementation process of this method has been described in detail in the implementation of the UAV-based dynamic industrial photogrammetry system, and will not be repeated here.
Claims
1. A dynamic industrial photogrammetry method based on unmanned aerial vehicles (UAVs), characterized in that, The method includes the following steps: 1) Select an initial point from the target being measured, and use the initial point as the origin of the world coordinate system. Use the initial point image taken by the industrial camera mounted on the UAV to perform initial value calculation to obtain the initial pose of the industrial camera mounted on the UAV. 2) During the flight of the UAV from the initial point, the industrial camera acquires dynamic images of the target at a set frame rate and processes them to obtain the coded ID corresponding to the coded marker and the center coordinates of the target point; the coded marker and the target point are both markers set on the target. 3) Based on tracking, pose estimation, and forward intersection in visual SLAM, the industrial camera pose and preliminary 3D coordinates of the target point corresponding to each frame of the image are calculated; and local bundle adjustment is used to locally optimize the calculated industrial camera pose and preliminary 3D coordinates of the target point. 4) After the drone completes one lap, the global adjustment principle is used to globally optimize the poses of all industrial cameras and the coordinates of the target points.
2. The dynamic industrial photogrammetry method based on UAV according to claim 1, characterized in that, The method also includes using a reference ruler or control points with known coordinates to perform precise scale correction on the calculated coordinates of all target points.
3. The dynamic industrial photogrammetry method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The initial point is a coded marker point or a control point with known coordinates placed in the target being measured.
4. The dynamic industrial photogrammetry method based on UAV according to claim 1, characterized in that, The coded markers and target points mentioned are markers with a certain shape and made of reflective material that are affixed to the target being measured.
5. The dynamic industrial photogrammetry method based on UAV according to claim 1, characterized in that, The drone is also equipped with an active light source module, providing a reliable light source for industrial camera shooting.
6. The dynamic industrial photogrammetry method based on UAV according to claim 1, characterized in that, Local bundle adjustment is triggered at a time. It uses the coordinates of all historical poses of the industrial camera and all observed 3D points as optimization variables, and minimizes the pixel error of reprojecting all observed 3D points onto the 2D image as the cost function. By solving this cost function, the poses of each frame and the coordinates of the observed 3D points are optimized and updated. The observed 3D points include the observed coded marker points and target points.
7. The dynamic industrial photogrammetry method based on UAV according to claim 1, characterized in that, The drone completes a flight based on the initial point on the target being measured, which is re-collected by the industrial camera.
8. The dynamic industrial photogrammetry method based on UAV according to claim 7, characterized in that, If the drone flies more than two laps, the average of the results after each correction will be used as the final optimized result.
9. The dynamic industrial photogrammetry method based on unmanned aerial vehicles according to claim 2, characterized in that, When multiple scale corrections are applied, the average of the multiple scale corrections is used as the final correction result.
10. A dynamic industrial photogrammetry system based on a drone, comprising an industrial camera and a computer mounted on the drone, characterized in that, The system uses the UAV-based dynamic industrial photogrammetry method as described in any one of claims 1-9 to measure the product under test.