An unmanned aerial vehicle image supplementing flight path planning method and device

Abstract

The application belongs to the technical field of data processing, and particularly relates to a UAV image supplementary acquisition flight path planning method and device. The method first acquires initial acquisition data of a to-be-measured region and UAV task constraints, establishes an initial three-dimensional model based on the initial acquisition data, and divides the model into multiple three-dimensional model units; calculates quality state data for each unit, and generates a three-dimensional model cavity quality graph; clusters cavity and low-quality regions into supplementary acquisition target blocks, reversely generates a candidate supplementary acquisition viewpoint set according to supplementary acquisition requirements and task constraints of each target block; scores the candidate viewpoints by comprehensively considering visible area gain, point cloud density gain, texture quality gain, viewpoint complementary degree and flight cost and the like, selects supplementary acquisition viewpoints by optimization, and performs path planning to generate an executable supplementary acquisition flight path. The application realizes full-automatic closed-loop supplementary acquisition from defect identification, viewpoint generation to flight path planning.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a method and apparatus for planning flight routes for UAV image acquisition. Background Technology

[0002] Currently, UAV point cloud acquisition and oblique photogrammetry are widely used in surveying, mining, urban modeling, building facade inspection, disaster emergency response, digital twins, and engineering acceptance. In typical operations, UAVs are equipped with cameras, LiDAR, inertial navigation equipment, and positioning equipment to collect data on the area to be surveyed along a flight path. Then, a 3D model is generated through processes such as point cloud registration, image matching, aerial triangulation, mesh reconstruction, and texture mapping.

[0003] In actual projects, initial flight paths are usually planned in advance based on area boundaries, heading overlap rate, lateral overlap rate, flight altitude, and load parameters. However, due to factors such as terrain or building obstruction, facade reflection, inappropriate flight path spacing, changes in flight attitude, changes in lighting, weak textures, excessively large point cloud incident angles, or local acquisition failures, the initial 3D model often exhibits problems such as model holes, sparse point clouds, blurred textures, missing facades, broken boundaries, and unstable local reconstruction.

[0004] The common practice is to manually review the model quality after it is generated, determine which areas need re-flying, and then manually plan the re-flying routes. This approach has several problems: First, model defects are usually scattered, making manual inspection time-consuming and prone to missing localized issues; second, it's difficult for humans to accurately determine from which angle to re-fly holes or low-quality areas, potentially leading to duplicate re-flying or unsatisfactory results after re-flying; third, re-flying routes are often planned primarily based on spatial location, with insufficient consideration for point cloud density, viewpoint complementarity, texture quality, occlusion relationships, and energy constraints; and fourth, there is a lack of automatic quality assessment after re-flying, requiring manual verification of the re-flying effect. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method and apparatus for planning flight routes for UAV image acquisition supplementation.

[0006] The first aspect of this application provides a method for planning flight paths for UAV image acquisition, mainly including:

[0007] Step S1: Obtain the initial data collection for the area to be tested and the UAV mission constraints;

[0008] Step S2: Based on the initial collected data, establish an initial three-dimensional model and divide the initial three-dimensional model into multiple three-dimensional model units. Each three-dimensional model unit is represented by a voxel, a mesh patch, a triangular patch, a point cloud cluster, or a combination thereof.

[0009] Step S3: Determine the quality status data of each three-dimensional model unit to generate a three-dimensional model cavity quality map. The quality status data includes quality level, cavity marker, low quality marker, missing observation direction, re-sampling requirement type, and re-sampling priority.

[0010] Step S4: Cluster the cavity regions and low-quality regions of the cavity quality map of the three-dimensional model into at least one re-mining target block, and determine the re-mining requirements of each re-mining target block;

[0011] Step S5: Generate a set of candidate replenishment viewpoints based on the replenishment requirements of each replenishment target block and the constraints of the UAV mission;

[0012] Step S6: Calculate the viewpoint score for each candidate supplementary viewpoint in the candidate supplementary viewpoint set;

[0013] Step S7: Based on the viewpoint score, select supplementary viewpoints from the candidate supplementary viewpoint set, and perform path planning on the selected supplementary viewpoints to generate supplementary sampling routes.

[0014] Preferably, in step S1, the initial acquisition data includes one or more of the following: initial point cloud, oblique image, orthophoto, UAV pose data, camera intrinsic parameters, camera extrinsic parameters, lidar extrinsic parameters, and initial 3D model; the UAV mission constraints include one or more of the following: no-fly zone, altitude-restricted zone, obstacles, terrain height, remaining battery power, maximum range, maximum flight time, sensor operating parameters, desired ground sampling distance, and desired point cloud density.

[0015] Preferably, in step S2, the size of the three-dimensional model unit is determined according to the type and importance of the object in the area to be measured.

[0016] Preferably, in step S3, the quality level is obtained by weighted calculation and mapping of one or more of the following: point cloud density, number of image coverage directions, viewing angle, ground sampling distance, texture sharpness, reprojection residual, normal uncertainty, occlusion probability, point cloud height variance, model hole area, and hole boundary confidence; the low quality mark refers to a mark whose quality level is lower than a set threshold.

[0017] Preferably, in step S4, the type of supplementary data acquisition requirement includes one or more of the following: point cloud densification, texture supplementary shooting, multi-view supplementary shooting, occlusion removal supplementary shooting, facade supplementary shooting, boundary supplementary shooting, and key area verification.

[0018] Preferably, in step S6, the viewpoint score consists of positive gain indicators and negative gain indicators. The positive gain indicators include visible area gain, point cloud density gain, texture quality gain, and viewpoint complementarity. The negative gain indicators include flight cost, safety cost, energy cost, and repeated acquisition penalty parameters.

[0019] Preferably, in step S7, one or more viewpoints with viewpoint scores from high to low are selected from the candidate supplementary viewpoint set as supplementary viewpoints.

[0020] Preferably, the method further includes:

[0021] Step S8: Control the UAV to perform the supplementary mining task according to the supplementary mining route, acquire supplementary mining point cloud or supplementary mining image, and fuse the supplementary mining point cloud or supplementary mining image with the initial acquisition data to update the three-dimensional model and the three-dimensional model cavity quality map.

[0022] Step S9: Determine whether the updated 3D model cavity quality map meets the preset quality stop condition; if not, re-execute the target block clustering based on the updated 3D model cavity quality map until a replacement mining route is generated; if it meets the condition, output the replacement mining task result.

[0023] The second aspect of this application provides a UAV image acquisition and flight path planning device, mainly comprising:

[0024] The initial data acquisition module is used to acquire the initial data of the area to be tested and the UAV mission constraints;

[0025] The three-dimensional model unit division module is used to establish an initial three-dimensional model based on the initial acquired data and divide the initial three-dimensional model into multiple three-dimensional model units. Each three-dimensional model unit is represented by a voxel, a mesh patch, a triangular patch, a point cloud cluster, or a combination thereof.

[0026] The 3D model cavity quality map generation module is used to determine the quality status data of each 3D model unit to generate a 3D model cavity quality map. The quality status data includes quality level, cavity marker, low quality marker, missing observation direction, re-sampling requirement type, and re-sampling priority.

[0027] The target block determination module is used to cluster the cavity and low-quality regions of the cavity quality map of the three-dimensional model into at least one target block for replenishment, and determine the replenishment requirements of each target block.

[0028] The candidate supplementary mining viewpoint set generation module is used to generate a candidate supplementary mining viewpoint set based on the supplementary mining requirements of each supplementary mining target block and the constraints of the UAV mission.

[0029] The viewpoint scoring calculation module is used to calculate a viewpoint score for each candidate supplementary viewpoint in the candidate supplementary viewpoint set.

[0030] The supplementary mining route generation module is used to select supplementary mining viewpoints from the candidate supplementary mining viewpoint set based on the viewpoint score, and perform path planning on the selected supplementary mining viewpoints to generate supplementary mining routes.

[0031] This application uses a 3D model cavity quality map as a driver to automatically identify defective areas, reverse-generate the optimal replenishment route, and form a closed-loop iterative mechanism for acquisition, evaluation and replanning. Attached Figure Description

[0032] Figure 1 This is a flowchart of a preferred embodiment of the UAV image acquisition route planning method of this application.

[0033] Figure 2 This is a schematic diagram of the mass composition of a three-dimensional model unit.

[0034] Figure 3 This is a schematic diagram of the candidate viewpoint locations.

[0035] Figure 4 This is a flowchart of viewpoint selection and path planning. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of this application, not all of them. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0037] The first aspect of this application provides a method for planning flight paths for UAV image acquisition, such as... Figure 1 As shown, it mainly includes:

[0038] Step S1: Obtain the initial data collection for the area to be tested and the UAV mission constraints;

[0039] Step S2: Based on the initial collected data, establish an initial three-dimensional model and divide the initial three-dimensional model into multiple three-dimensional model units. Each three-dimensional model unit is represented by a voxel, a mesh patch, a triangular patch, a point cloud cluster, or a combination thereof.

[0040] Step S3: Determine the quality status data of each three-dimensional model unit to generate a three-dimensional model cavity quality map. The quality status data includes quality level, cavity marker, low quality marker, missing observation direction, re-sampling requirement type, and re-sampling priority.

[0041] Step S4: Cluster the cavity regions and low-quality regions of the cavity quality map of the three-dimensional model into at least one re-mining target block, and determine the re-mining requirements of each re-mining target block;

[0042] Step S5: Generate a set of candidate replenishment viewpoints based on the replenishment requirements of each replenishment target block and the constraints of the UAV mission;

[0043] Step S6: Calculate the viewpoint score for each candidate supplementary viewpoint in the candidate supplementary viewpoint set;

[0044] Step S7: Based on the viewpoint score, select supplementary viewpoints from the candidate supplementary viewpoint set, and perform path planning on the selected supplementary viewpoints to generate supplementary sampling routes.

[0045] This application transforms the quality defects of a 3D model into a structured void quality map, and uses this as the core driving signal to complete a fully automated closed-loop process from defect identification to viewpoint generation to route planning.

[0046] The initial data acquired in step S1 includes point clouds, oblique images, orthophotos, and various sensor intrinsic and extrinsic parameters and pose data. This data is not only used to construct the initial 3D model but also provides the raw input for subsequent quality index calculations. UAV mission constraints are integrated throughout the entire data acquisition planning process, ensuring that the generated flight paths are physically executable. In step S2, the 3D model unit partitioning method is highly flexible. Voxelization is suitable for uniform quality assessment under regular meshes, while mesh patches and triangular patches are more suitable for feature extraction based on surface geometry. Point cloud clusters can be operated directly in the point cloud space. The combination of multiple representation methods can adapt to the structural characteristics of different measured objects such as buildings, mine slopes, and bridges. Step S3 assigns quality state data to each 3D model unit, thus overlaying a layer of quality semantic information onto the 3D model, which originally only contained geometric and texture information. The determination of missing observation directions can accurately indicate which angle of observation data is missing in a certain area, and the data acquisition priority provides a decision-making basis for subsequent viewpoint selection and flight path sequencing. The clustering operation in step S4 aggregates fragmented low-quality units into spatially continuous re-collection target blocks. This aggregation not only reduces the complexity of subsequent viewpoint generation but also improves the coverage of a single re-collection by merging adjacent units with similar re-collection needs. Steps S5 to S7 transform quality requirements into executable routes. The generation of candidate viewpoints is based on the reverse derivation of the geometric features of the re-collection target blocks and the missing observation directions, rather than simply adding waypoints above the defective areas. This reverse derivation mechanism ensures that each generated viewpoint has a clear intention to improve quality. The subsequent scoring and selection process is further optimized based on this, and finally, through path sorting and route construction, directly executable re-collection task instructions are output.

[0047] In some optional implementations, in step S1, the initial acquisition data includes one or more of the following: initial point cloud, oblique image, orthophoto, UAV pose data, camera intrinsic parameters, camera extrinsic parameters, lidar extrinsic parameters, and initial 3D model; the UAV mission constraints include one or more of the following: no-fly zone, altitude-restricted zone, obstacles, terrain altitude, remaining battery power, maximum range, maximum flight time, sensor operating parameters, desired ground sampling distance, and desired point cloud density.

[0048] In this embodiment, the initial point cloud provides the three-dimensional geometric structure information of the scene in the initial acquisition data, the oblique image and orthophoto provide visual texture information from multiple perspectives, the UAV pose data records the spatial position and attitude of each frame of data acquisition, and the camera intrinsic and extrinsic parameters and the lidar extrinsic parameters provide the necessary calibration benchmarks for subsequent reprojection error calculation, point cloud registration and quality index calculation. It is worth noting that the initial three-dimensional model itself can also be used as one of the input data, which means that this application can either completely perform modeling and supplementary acquisition planning from the original acquisition data, or directly perform supplementary acquisition analysis based on the existing three-dimensional model, which has good forward compatibility.

[0049] Among the various mission constraints of the aforementioned UAVs, no-fly zones and altitude-restricted zones are hard constraints at the legal and regulatory level, obstacle and terrain height are physical constraints for flight safety, remaining battery power, maximum range, and maximum flight time are soft constraints in the energy dimension, and sensor operating parameters, expected ground sampling distance, and expected point cloud density are constraints proposed from the perspective of acquisition quality. These constraints play a role in all stages of candidate viewpoint generation, scoring, and path planning, ensuring that the final output supplementary sampling route can not only theoretically improve model quality, but is also safe and feasible in actual operation and does not exceed the capability boundaries of the UAV.

[0050] In some alternative implementations, in step S2, the size of the three-dimensional model unit is determined based on the type and importance of the object within the area to be measured.

[0051] The 3D model elements in this application are dynamically adjusted according to the type and importance of the object being measured, thereby achieving a balance between computational accuracy and efficiency. For areas with complex geometry and rich details, such as key parts like architectural decorative components, bridge nodes, and equipment instruments, using smaller element sizes enables more refined quality assessments, ensuring that even minor defects are not overlooked. Conversely, for large, flat areas, such as floors, main walls, and mining platforms, using larger element sizes can significantly reduce the amount of data processing while maintaining the reliability of the quality assessment. This adaptive partitioning mechanism can be configured in conjunction with prior knowledge during implementation. For example, fine-grained partitioning can be used for pre-defined facade areas of buildings, while coarse-grained partitioning can be used for roof planes. Different unit sizes can be set according to slope changes for mine slopes, and partitioning parameters can be set separately for different component types such as main beams, piers, and cables for bridge structures. In addition, the adjustment of unit granularity can be linked with subsequent quality index calculations. For example, fine-grained units can be used to calculate detailed indices such as normal uncertainty and reprojection residuals, while coarse-grained units can be used to examine macroscopic indices such as point cloud density and number of coverage directions, thereby realizing a hierarchical quality assessment system.

[0052] In some alternative implementations, in step S3, such as Figure 2 As shown, the quality level is obtained by weighted calculation and mapping of one or more of the following: point cloud density, number of image coverage directions, viewing angle, ground sampling distance, texture sharpness, reprojection residual, normal uncertainty, occlusion probability, point cloud height variance, model hole area, and hole boundary confidence; the low quality label refers to a label whose quality level is lower than a set threshold.

[0053] In this embodiment, the quality level is a comprehensive evaluation result obtained by weighted fusion and mapping of multiple underlying quality indicators. Point cloud density reflects the richness of the three-dimensional geometric information of the region. The number of image coverage directions reveals how many cameras at different angles have observed the region. The viewing angle characterizes the uniformity of the distribution among existing observation directions and is used to determine whether existing observations can support stable three-dimensional reconstruction. Too small an angle means insufficient multi-view geometric constraints. Ground sampling distance determines the spatial resolution that the model can achieve. Texture clarity directly affects the visual effect and usability of the final model and is used to indicate whether the texture meets the modeling or acceptance requirements. Reprojection residual measures the degree of consistency between the image and the model. Excessive residual can lead to texture mapping errors or model distortion. Normal uncertainty describes the reliability of the estimation of surface geometric directions. This indicator will increase significantly, especially in steep slope areas when the point cloud incident angle is too large. Occlusion probability quantifies the possibility that the region is occluded by terrain or ground features. Point cloud height variance reflects the local geometric complexity of the surface. Model hole area directly identifies the size of the geometric missing range. Hole boundary confidence is used to distinguish between real object boundaries and pseudo-holes caused by insufficient acquisition. These indicators, after weighted calculation, are mapped to discrete quality levels, such as excellent, good, average, and poor, or continuous numerical scores, enabling the system to compare and rank the quality status of different areas on a uniform scale. Low-quality marking explicitly identifies units with quality levels below a preset threshold. This threshold can be flexibly set according to project acceptance standards or user needs. For example, urban modeling projects may have high requirements for texture clarity, while mine monitoring projects may focus more on point cloud density and normal stability. Through differentiated threshold configurations, this application can adapt to the quality requirements of different application scenarios.

[0054] In some optional implementations, in step S4, the type of supplementary acquisition requirement includes one or more of the following: point cloud densification, texture supplementation, multi-view supplementation, occlusion removal supplementation, facade supplementation, boundary supplementation, and key area verification.

[0055] In this embodiment, each type of replenishment demand corresponds to a specific quality defect mode and a corresponding replenishment strategy. Point cloud densification primarily targets areas with insufficient point cloud density or high normal uncertainty. During densification, priority is given to observation angles that can reduce the point cloud incident angle and increase point cloud coverage density. Texture re-imaging targets areas with blurred textures, excessive reprojection residuals, or missing textures. During re-imaging, viewpoints that provide higher resolution, better lighting conditions, or more direct observation angles should be selected. Multi-view re-imaging is suitable for areas with insufficient image coverage directions. By adding new observation directions, the robustness of multi-view stereo matching is improved. Occlusion removal re-imaging is specifically designed for blind spots caused by terrain or ground features. Observations need to be taken from special angles that can bypass the obstructions. Facade re-imaging mainly targets the vertical or near-vertical surfaces of buildings or man-made structures. These surfaces are often insufficiently covered in traditional top-down routes, requiring specially designed lateral or oblique observation routes. Boundary re-imaging addresses the problem of fragmented or incomplete model edge areas. By adding observations along the outer edge of the boundary, the clarity and integrity of the boundary are ensured. Critical area verification targets areas with high importance or large quality uncertainty. By repeatedly observing from multiple angles, the reliability of the reconstruction results is improved. In practice, a target block for supplementary sampling may have multiple supplementary sampling demand types at the same time. The system will determine the final supplementary sampling strategy and viewpoint generation parameters based on the dominant demand type or a combination of multiple demand types. For example, for a region that has both point cloud sparsity and texture blurring problems, the generation of its candidate viewpoints needs to take into account both the point cloud incident angle optimization and texture resolution improvement.

[0056] Then, in step S5, based on the target block to be supplemented, the system generates candidate supplementary viewpoints in reverse. Candidate supplementary viewpoints include the UAV's spatial position, heading angle, gimbal attitude, sensor operating mode, and acquisition parameters. When generating candidate supplementary viewpoints, an initial viewpoint can be generated first along the opposite direction of the target surface normal or the missing observation direction. Then, the viewpoint distance is determined based on the camera's field of view angle, lidar scanning angle, desired ground sampling distance, desired point cloud incident angle, and safety buffer range. Simultaneously, the system eliminates viewpoints that are practically inaccessible or of high risk based on no-fly zones, obstacles, height restrictions, terrain height, communication coverage, and UAV motion constraints, such as... Figure 3 As shown.

[0057] Step S6 is used for viewpoint score calculation, such as Figure 4 As shown, in some optional embodiments, the viewpoint score consists of positive gain metrics and negative gain metrics. Positive gain metrics include visible area gain, point cloud density gain, texture quality gain, and viewpoint complementarity. Negative gain metrics include flight cost, safety cost, energy cost, and repeated acquisition penalty parameters.

[0058] In this embodiment, visible area gain measures the size of the area to be supplemented that the viewpoint can cover, and is the most intuitive indicator for evaluating the value of the viewpoint; point cloud density gain and texture quality gain are the extent of quality improvement after supplementation based on sensor models and geometric relationships. These two indicators make the viewpoint score no longer a pure spatial coverage measure, but an expected benefit assessment directly linked to quality improvement; viewpoint complementarity measures the difference between the new viewpoint and the existing observation direction. The higher the complementarity, the more independent geometric information the new viewpoint can provide, and the greater its contribution to multi-view reconstruction.

[0059] Among the negative gain indicators, flight cost and safety cost are negative indicators introduced from an engineering implementation perspective, reflecting the distance cost between viewpoints and the degree of risk of approaching obstacles or no-fly zones. Energy consumption cost mainly considers the remaining battery power to incorporate the UAV's energy status into the real-time decision-making process, ensuring sufficient return margin while completing as many resampling tasks as possible. In addition, communication quality cost can also be included, reflecting the degree of communication reliability degradation when far from the base station. The introduction of these cost items ensures that viewpoint selection does not ignore the safety and feasibility of actual operation in pursuit of maximum quality gain. The above indicators are combined by weighting to form the final viewpoint score. The weight coefficients can be dynamically adjusted according to task priority, environmental conditions, or user preferences. For example, when battery power is sufficient, the weight of quality gain indicators can be increased, while when battery power is tight, the influence of flight cost and energy consumption constraints can be increased.

[0060] In some alternative implementations, in step S7, one or more viewpoints with viewpoint scores from high to low are selected from the candidate supplementary viewpoint set as supplementary viewpoints.

[0061] This embodiment selects several top-scoring viewpoints from the candidate viewpoint set as the final supplementary acquisition viewpoints. The core assumption of this strategy is that the scoring function has fully reflected the comprehensive value of each viewpoint, so the viewpoint with the highest score is the optimal choice. At the same time, allowing the selection of more than one viewpoint is to deal with situations where a single viewpoint cannot fully cover the supplementary acquisition target block or where a single viewpoint poses a risk.

[0062] In practice, the number of viewpoints selected can be determined comprehensively based on the area and shape complexity of the target block, the type of re-acquisition requirement, and the drone's endurance. For target blocks with a small area and a single requirement, only one optimal viewpoint may be needed. However, for large-area target blocks or those with multiple re-acquisition requirements, multiple high-resolution viewpoints may need to be selected to form a viewpoint combination. In addition, the selection of viewpoints can also introduce diversity constraints. That is, when the spatial positions or observation directions of multiple high-resolution viewpoints are too close, the system can actively discard some redundant viewpoints and instead select viewpoints with slightly lower scores but that can provide more independent information. This avoids wasting re-acquisition resources and improves the overall robustness of the observation geometry.

[0063] In step S7, the route construction process is expressed as a constrained multi-objective optimization problem. Constraints include remaining battery power, maximum range, maximum flight time, minimum turning radius, maximum climb rate, maximum descent rate, image overlap, point cloud overlap, no-fly zone safety distance, and obstacle safety distance. Remaining battery power and maximum flight time ensure the mission can be completed safely and return safely; maximum range limits the total flight distance; minimum turning radius, maximum climb rate, and maximum descent rate are physical limitations determined by the UAV's own dynamic characteristics; image overlap and point cloud overlap ensure good integration between supplementary data and existing data; and no-fly zone safety distance and obstacle safety distance provide the minimum guarantee for flight safety. In the actual path sequencing process, the system can use heuristic or exact algorithms to solve for the viewpoint access order that satisfies all constraints and achieves the highest comprehensive score. When necessary, the path can be smoothed to reduce UAV attitude changes and energy consumption.

[0064] In some alternative implementations, the method further includes:

[0065] Step S8: Control the UAV to perform the supplementary mining task according to the supplementary mining route, acquire supplementary mining point cloud or supplementary mining image, and fuse the supplementary mining point cloud or supplementary mining image with the initial acquisition data to update the three-dimensional model and the three-dimensional model cavity quality map.

[0066] Step S9: Determine whether the updated 3D model cavity quality map meets the preset quality stop condition; if not, re-execute the target block clustering based on the updated 3D model cavity quality map until a replacement mining route is generated; if it meets the condition, output the replacement mining task result.

[0067] Unlike one-time re-flight solutions, in step S8 of this application, the UAV performs the actual data acquisition task according to the generated re-acquisition flight path. The acquired re-acquisition point cloud or imagery is not used independently but is fused with the initial acquisition data. This fusion involves not only simple data overlay but also point cloud registration, image alignment, and aerial triangulation to ensure consistency between the old and new data in coordinate system and scale. Based on the fused data, the system reconstructs or updates the 3D model and simultaneously updates the 3D model hole quality map, so that the quality assessment results reflect the latest acquisition status. The quality stop condition judgment in step S9 constitutes the decision node of the closed-loop iteration. If the updated quality map shows that there are still areas that do not meet the standards, the system automatically enters the next round of re-acquisition planning loop and re-executes the entire process from clustering of re-acquisition target blocks to flight path generation. If all areas have met the quality requirements, the re-acquisition task results are output and the process ends. The advantage of this iterative mechanism is that it can adaptively allocate supplementary resources: for areas that can be met with one supplementary sampling, no additional flight time is wasted; for areas that cannot be fully improved with one supplementary sampling, subsequent supplementary sampling is automatically arranged until the preset quality target is reached or the task budget limit is reached, such as the maximum number of iterations, the minimum remaining power, or the longest operation time.

[0068] In addition, this closed-loop mechanism can be combined with an online skipping mechanism, which monitors the quality improvement of each target block in real time during the supplementary mining process. If a target block has already met the standard after the first few viewpoints, the remaining viewpoints for that block can be skipped, further saving flight resources.

[0069] This application transforms quality issues such as scattered holes, sparse point clouds, blurred textures, and missing views in 3D models into structured 3D model hole quality maps. By assigning multi-dimensional attributes such as quality level, hole marker, low quality marker, missing observation direction, re-sampling demand type, and re-sampling priority to each 3D model unit, the re-sampling demand is transformed from subjective experience judgment to objective decision-making based on quality data, significantly reducing the rate of missed detections and misjudgments by humans.

[0070] This application integrates multiple quality indicators, including point cloud density, number of image coverage directions, viewing angle, texture clarity, reprojection residual, normal uncertainty, occlusion probability, point cloud height variance, model hole area, and hole boundary confidence. It can distinguish between real object boundaries and pseudo-holes caused by insufficient acquisition, and can accurately identify specific quality shortcomings in each region, such as insufficient point cloud, missing texture, or poor multi-view geometry. This provides accurate data support for subsequent targeted acquisition and avoids the waste of resources caused by indiscriminate repeated acquisition.

[0071] This application does not simply add waypoints along a preset route. Instead, it solves for reachable viewpoints in reverse by combining the surface normal of the target block and the missing observation direction with the sensor field of view, the expected ground sampling distance, and safety constraints. This ensures that each candidate viewpoint has a clear geometric meaning and expected quality improvement. At the same time, it eliminates unreachable viewpoints by using no-fly zones, obstacles, height restrictions, and terrain constraints, thus ensuring the feasibility of the supplementary sampling scheme from the source.

[0072] In the viewpoint selection stage, this application comprehensively considers visible area gain, point cloud density gain, texture quality gain, view complementarity, overlap satisfaction, flight cost, safety cost, and energy consumption constraints. Through weighted scoring, it selects the viewpoint combination with the highest quality benefit and the lowest flight cost. In the route generation stage, it further introduces engineering constraints such as minimum turning radius, maximum climb rate, descent rate, and safe distance for path sorting and smoothing. While ensuring the quality of supplementary data collection, it effectively reduces flight energy consumption and time costs, and reduces invalid supplementary data collection and redundant flight segments.

[0073] This application integrates the new data with the initial data after each supplementary data collection, dynamically updates the 3D model and the cavity quality map, and determines whether the next round of supplementary data collection is needed based on the updated quality status. This forms an iterative optimization mechanism of collection, evaluation, updating, and replanning, realizing the adaptive allocation of supplementary data collection resources until the model quality meets the preset requirements or reaches the upper limit of the mission budget. This fundamentally solves the problem that traditional one-time supplementary flights cannot guarantee the final quality.

[0074] The method framework of this application does not depend on a specific type of test object or load configuration. By adjusting the granularity of the three-dimensional model unit, the weight of the quality index and the type of re-mining requirement, it can be flexibly adapted to various application scenarios such as oblique photography re-capture of building facades, point cloud densification of mine slopes, and re-mining of voids at the bottom of bridges. It has strong engineering practicality and promotional value.

[0075] The second aspect of this application provides a UAV image acquisition and flight path planning device corresponding to the above method, mainly comprising:

[0076] The initial data acquisition module is used to acquire the initial data of the area to be tested and the UAV mission constraints;

[0077] The three-dimensional model unit division module is used to establish an initial three-dimensional model based on the initial acquired data and divide the initial three-dimensional model into multiple three-dimensional model units. Each three-dimensional model unit is represented by a voxel, a mesh patch, a triangular patch, a point cloud cluster, or a combination thereof.

[0078] The 3D model cavity quality map generation module is used to determine the quality status data of each 3D model unit to generate a 3D model cavity quality map. The quality status data includes quality level, cavity marker, low quality marker, missing observation direction, re-sampling requirement type, and re-sampling priority.

[0079] The target block determination module is used to cluster the cavity and low-quality regions of the cavity quality map of the three-dimensional model into at least one target block for replenishment, and determine the replenishment requirements of each target block.

[0080] The candidate supplementary mining viewpoint set generation module is used to generate a candidate supplementary mining viewpoint set based on the supplementary mining requirements of each supplementary mining target block and the constraints of the UAV mission.

[0081] The viewpoint scoring calculation module is used to calculate a viewpoint score for each candidate supplementary viewpoint in the candidate supplementary viewpoint set.

[0082] The supplementary mining route generation module is used to select supplementary mining viewpoints from the candidate supplementary mining viewpoint set based on the viewpoint score, and perform path planning on the selected supplementary mining viewpoints to generate supplementary mining routes.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for planning flight paths for UAV image acquisition, characterized in that, include: Step S1: Obtain the initial data collection for the area to be tested and the UAV mission constraints; Step S2: Based on the initial collected data, establish an initial three-dimensional model and divide the initial three-dimensional model into multiple three-dimensional model units. Each three-dimensional model unit is represented by a voxel, a mesh patch, a triangular patch, a point cloud cluster, or a combination thereof. Step S3: Determine the quality status data of each three-dimensional model unit to generate a three-dimensional model cavity quality map. The quality status data includes quality level, cavity marker, low quality marker, missing observation direction, re-sampling requirement type, and re-sampling priority. Step S4: Cluster the cavity regions and low-quality regions of the cavity quality map of the three-dimensional model into at least one re-mining target block, and determine the re-mining requirements of each re-mining target block; Step S5: Generate a set of candidate replenishment viewpoints based on the replenishment requirements of each replenishment target block and the constraints of the UAV mission; Step S6: Calculate the viewpoint score for each candidate supplementary viewpoint in the candidate supplementary viewpoint set; Step S7: Based on the viewpoint score, select supplementary viewpoints from the candidate supplementary viewpoint set, and perform path planning on the selected supplementary viewpoints to generate supplementary sampling routes.

2. The UAV image acquisition and flight path planning method according to claim 1, characterized in that, In step S1, the initial acquisition data includes one or more of the following: initial point cloud, oblique image, orthophoto, UAV pose data, camera intrinsic parameters, camera extrinsic parameters, lidar extrinsic parameters, and initial 3D model; the UAV mission constraints include one or more of the following: no-fly zone, altitude-restricted zone, obstacles, terrain altitude, remaining battery power, maximum range, maximum flight time, sensor operating parameters, desired ground sampling distance, and desired point cloud density.

3. The UAV image acquisition route planning method according to claim 1, characterized in that, In step S2, the size of the three-dimensional model unit is determined according to the type and importance of the object in the area to be measured.

4. The UAV image acquisition route planning method according to claim 1, characterized in that, In step S3, the quality level is obtained by weighted calculation and mapping of one or more of the following: point cloud density, number of image coverage directions, viewing angle, ground sampling distance, texture sharpness, reprojection residual, normal uncertainty, occlusion probability, point cloud height variance, model hole area, and hole boundary confidence; the low quality label refers to a label whose quality level is lower than a set threshold.

5. The UAV image acquisition route planning method according to claim 4, characterized in that, In step S4, the types of supplementary data collection requirements include one or more of the following: point cloud densification, texture supplementation, multi-view supplementation, occlusion removal supplementation, facade supplementation, boundary supplementation, and key area verification.

6. The UAV image acquisition route planning method according to claim 1, characterized in that, In step S6, the viewpoint score consists of positive gain indicators and negative gain indicators. The positive gain indicators include visible area gain, point cloud density gain, texture quality gain, and viewpoint complementarity. The negative gain indicators include flight cost, safety cost, energy cost, and repeated acquisition penalty parameters.

7. The UAV image acquisition route planning method according to claim 1, characterized in that, In step S7, one or more viewpoints with viewpoint scores from high to low are selected from the candidate supplementary viewpoint set as supplementary viewpoints.

8. The UAV image acquisition route planning method according to claim 1, characterized in that, The method further includes: Step S8: Control the UAV to perform the supplementary mining task according to the supplementary mining route, acquire supplementary mining point cloud or supplementary mining image, and fuse the supplementary mining point cloud or supplementary mining image with the initial acquisition data to update the three-dimensional model and the three-dimensional model cavity quality map. Step S9: Determine whether the updated 3D model cavity quality map meets the preset quality stop condition; if not, re-execute the target block clustering based on the updated 3D model cavity quality map until a replacement mining route is generated; if it meets the condition, output the replacement mining task result.

9. A UAV image acquisition and flight path planning device, characterized in that, include: The initial data acquisition module is used to acquire the initial data of the area to be tested and the UAV mission constraints; The three-dimensional model unit division module is used to establish an initial three-dimensional model based on the initial acquired data and divide the initial three-dimensional model into multiple three-dimensional model units. Each three-dimensional model unit is represented by a voxel, a mesh patch, a triangular patch, a point cloud cluster, or a combination thereof. The 3D model cavity quality map generation module is used to determine the quality status data of each 3D model unit to generate a 3D model cavity quality map. The quality status data includes quality level, cavity marker, low quality marker, missing observation direction, re-sampling requirement type, and re-sampling priority. The target block determination module is used to cluster the cavity and low-quality regions of the cavity quality map of the three-dimensional model into at least one target block for replenishment, and determine the replenishment requirements of each target block. The candidate supplementary mining viewpoint set generation module is used to generate a candidate supplementary mining viewpoint set based on the supplementary mining requirements of each supplementary mining target block and the constraints of the UAV mission. The viewpoint scoring calculation module is used to calculate a viewpoint score for each candidate supplementary viewpoint in the candidate supplementary viewpoint set. The supplementary mining route generation module is used to select supplementary mining viewpoints from the candidate supplementary mining viewpoint set based on the viewpoint score, and perform path planning on the selected supplementary mining viewpoints to generate supplementary mining routes.

Images