A channel slope deformation monitoring method based on close-range photogrammetry of unmanned aerial vehicle

Abstract

This invention provides a method for monitoring canal slope deformation based on close-up photogrammetry using unmanned aerial vehicles (UAVs), comprising the following steps: automatically generating a globally optimized flight path; adjusting the next waypoint position and UAV attitude online; inputting the multi-view images and the three-dimensional coordinates of image control points deployed in the area to be measured into an aerial triangulation system to reconstruct a dense point cloud at the current moment; adaptively adjusting the sampling density according to the geometric changes of the neighborhood of each point in the point cloud; inputting the adaptively downsampled point cloud at the current moment and the point cloud at a reference moment into an iterative nearest-point registration algorithm with dynamic neighborhood search and feature weighting mechanisms to achieve accurate alignment of the two temporal point clouds; calculating the three-dimensional displacement vectors of corresponding points in the aligned two temporal point clouds, and generating a continuous displacement field through multi-scale interpolation; and conducting a risk assessment of canal slope deformation based on the displacement magnitude and spatial gradient of the displacement field. This invention improves the accuracy and efficiency of canal slope deformation monitoring.

Description

Technical Field

[0001] This invention belongs to the field of UAV remote sensing monitoring technology, specifically relating to a method for monitoring canal slope deformation based on close-up photogrammetry using UAVs. Background Technology

[0002] Monitoring canal slope deformation is a crucial foundation for the safe operation and maintenance of water conservancy projects and for early warning of geological disasters. Traditional methods for monitoring canal slope deformation mainly rely on ground measurement technologies such as total stations and GNSS (Global Navigation Satellite System). While these methods offer high accuracy, they suffer from drawbacks such as low measurement efficiency and high labor and economic costs. In particular, they are difficult to implement effectively in complex terrain areas such as steep slopes and cracks, and their terrain coverage is limited.

[0003] In recent years, UAV aerial photogrammetry technology has been widely used in monitoring canal slope deformation, significantly improving monitoring efficiency. However, existing UAV aerial surveying technology also has many shortcomings, mainly manifested in:

[0004] (1) The route planning method with fixed flight altitude and speed is difficult to effectively adapt to the drastic topographic features of the canal slope, resulting in insufficient image overlap at the top of the canal slope and image resolution at the bottom of the canal slope that does not meet the monitoring requirements due to the relatively high flight altitude.

[0005] (2) Static flight path planning cannot respond in real time to dynamic obstacles that occur during flight, such as birds and temporary construction equipment, which leads to a greater risk of UAV crashes. In addition, existing technologies usually rely on threshold-triggered hovering or return-to-home to avoid obstacles, which frequently interrupts the mission, resulting in low efficiency of image data acquisition and a significant decrease in image overlap consistency of the re-flight data.

[0006] (3) The classic Iterative Closest Point (ICP) algorithm is generally used in the data processing stage to register point clouds at different times. However, this algorithm is sensitive to the initial position of the point cloud. Under the conditions of vegetation shading or seasonal changes (such as snow cover), the registration error accumulates significantly. Especially in high curvature areas, the detailed features are severely lost, making it difficult to meet the required high-precision monitoring requirements.

[0007] (4) Existing displacement field construction technology usually adopts the moving least squares method with fixed node spacing, which is difficult to effectively capture high-frequency deformation signals in steep slopes and cracked areas. This results in an overly smooth displacement field that cannot accurately express millimeter-level micro-deformation information, which seriously restricts the accuracy and reliability of canal slope deformation monitoring.

[0008] The aforementioned existing technical problems greatly limit the application effect of canal slope deformation monitoring technology in actual engineering. Therefore, there is an urgent need for a more efficient, accurate and adaptable canal slope deformation monitoring method to meet the actual needs of safe operation and maintenance of water conservancy projects and early warning of geological disasters. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of the aforementioned background technology and provide a method for monitoring canal slope deformation based on UAV close-up photogrammetry. This method enables terrain-adaptive UAV close-up photogrammetry flight path planning and dynamic obstacle avoidance, significantly improving the accuracy and efficiency of canal slope deformation monitoring.

[0010] The technical solution adopted in this invention is: a method for monitoring canal slope deformation based on close-up photogrammetry using unmanned aerial vehicles (UAVs), comprising the following steps:

[0011] Based on a rough model of the area to be measured and historical manually planned data, a globally optimized route that covers the area and maintains a preset safe distance in any flight segment is automatically generated.

[0012] The drone is controlled to fly close to the globally optimized flight path and collect multi-view images. During the flight, the position of the next waypoint and the attitude of the drone are adjusted online based on the drone's real-time status data and continuous aerial images to ensure image coverage and avoid obstacles in real time.

[0013] The multi-view images and the three-dimensional coordinates of the image control points deployed in the area to be measured are input into the aerial triangulation system to reconstruct the dense point cloud at the current moment; and the sampling density is adaptively adjusted according to the geometric changes of the neighborhood of each point in the point cloud to maintain a high point cloud density in areas with abrupt terrain changes.

[0014] The current point cloud after adaptive downsampling and the point cloud at the reference time are input into an iterative nearest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to achieve accurate alignment of the two temporal point clouds.

[0015] Based on the aligned two time-phase points, the three-dimensional displacement vectors of corresponding points are calculated, and a continuous displacement field is generated through multi-scale interpolation.

[0016] By combining the displacement amount and spatial gradient of the displacement field, a risk assessment of the canal slope deformation is conducted, and deformation monitoring results and early warning information are output.

[0017] The process of obtaining the globally optimized flight route in the above technical solution includes:

[0018] Offline training phase:

[0019] A rough model of the area to be measured and historical manually planned flight route data are simultaneously input into a generative adversarial network (GAN) flight route planning model. The GAN includes:

[0020] A generator is used to generate a probability map of candidate routes based on a probabilistic model.

[0021] The discriminator judges the quality of the candidate route probability map based on historical manual route data and feeds the judgment result back to the generator.

[0022] Through multiple rounds of adversarial training between the generator and the discriminator until they reach stable convergence, the final network parameters of the generator are determined.

[0023] Online generation stage:

[0024] The probabilistic model is input into the generator that has been trained above to generate a probability map of flight routes covering the region.

[0025] Non-maximum suppression is first applied to the route probability map, and then connectivity extraction is performed based on the probability threshold to obtain a continuous globally optimized route.

[0026] This globally optimized flight path will be used as the initial flight path output for subsequent UAV close-up photogrammetry flights.

[0027] In the above technical solution, the discriminator, during the offline training phase, evaluates the degree to which the image overlap of the route probability map output by the generator meets the requirements and the terrain adaptability based on historical manually planned route data, and feeds the evaluation results back to the generator.

[0028] In the above technical solution, during the adversarial training phase, the loss function of the generator includes:

[0029] The adversarial loss term is used to constrain the game balance between the generator and the discriminator.

[0030] The route area intersection and union ratio loss term is used to constrain the regional overlap between generated routes and historical manual routes.

[0031] The total variation regularization term is used to constrain the spatial smoothness of the route probability graph.

[0032] Each loss item is summed by weighting it according to a predetermined weighting coefficient.

[0033] In the above technical solution, during flight, an attitude control strategy network based on deep reinforcement learning is constructed;

[0034] The input state vector of the policy network includes: the pose and motion stacking information of the UAV at the current and several historical moments; the relative distance to environmental obstacles; and the terrain texture variance features extracted from the general model.

[0035] The strategy network outputs three-dimensional continuous control commands in real time, which are used to fine-tune the roll angle, pitch angle and throttle thrust of the UAV online.

[0036] The reward function of the policy network comprehensively evaluates three indicators: image coverage quality, obstacle safety distance, and remaining battery power. Accordingly, improved image coverage, increased obstacle distance, and improved battery life are set as positive rewards. Using this reward function, the policy network parameters are iteratively updated using a near-end policy optimization algorithm.

[0037] In the above technical solution, during flight, convolutional feature extraction is performed on multiple frames of aerial images continuously collected in front of the drone, and the threat weight of each obstacle pixel is obtained through spatiotemporal self-attention calculation.

[0038] Based on the threat weights, calculate the risk weights at several future waypoints;

[0039] When the risk weight of any predicted waypoint exceeds a preset threshold, obstacle avoidance correction commands are generated in real time to adjust the drone's flight path or to avoid dynamic obstacles online by climbing or hovering.

[0040] In the above technical solution, the execution process of the iterative nearest point registration algorithm includes:

[0041] Based on the ISS algorithm, key points with curvature greater than a preset threshold are extracted from the point cloud at the current time, and FPFH feature descriptors are calculated for each key point;

[0042] The FPFH descriptors are matched using the RANSAC algorithm to filter matching pairs where the distance between the FPFH feature descriptors is less than a preset threshold and the proportion of points in RANSAC is not less than a preset threshold. The initial rigid transformation matrix is ​​then estimated based on the matching pair.

[0043] The initial rigid transformation matrix is ​​applied to the point cloud at the current time step as the initial value for the fine registration iteration to obtain the pre-transformed point cloud;

[0044] Perform iterative nearest-point fine registration on the pre-transformed point cloud:

[0045] In each iteration, the neighborhood search radius is gradually reduced from the initial value to the minimum threshold;

[0046] Only point pairs whose normal vector angle is within a preset threshold are retained for matching;

[0047] Point pairs with curvature greater than a preset value are assigned higher matching weights, while point pairs with low curvature are assigned lower weights. The Huber robust loss function is used to process the residuals.

[0048] Repeat the fine registration process until the transformation increment is lower than the convergence threshold, thus completing the precise alignment of the point cloud at the current time with the point cloud at the reference time.

[0049] In the above technical solution, the generation process of the continuous displacement field includes:

[0050] Using the point cloud at the reference time as a reference, the nearest neighbor point is searched point by point in the point cloud at the current time, and the normal vector-curvature joint constraint is applied. Only point pairs with a normal angle less than a preset threshold and a curvature difference less than a preset threshold are retained.

[0051] The three-dimensional displacement vector is calculated for the retained point pairs, and outliers with displacements exceeding three times the standard deviation are removed according to the Laida criterion to obtain the discrete displacement vector field.

[0052] The discrete displacement vector field is interpolated and densified using the moving least squares method: a first preset node spacing is used in flat areas; and a second preset node spacing smaller than the first spacing is used in steep slopes or areas with dense cracks.

[0053] An irregular triangular network is constructed based on the encrypted displacement point cloud to generate a continuous displacement field.

[0054] In the above technical solution, the risk assessment process includes:

[0055] The displacement value and displacement gradient value at each position in the displacement field are normalized according to the preset weighting coefficients.

[0056] The standardized displacement value and the standardized displacement gradient value are combined in a linear weighted manner to obtain the comprehensive risk score of the location.

[0057] Based on the comprehensive risk score, the area to be measured is divided into four categories: high-risk area, medium-risk area, low-risk area, and stable area.

[0058] This invention also provides a canal slope deformation monitoring system based on close-up photogrammetry using unmanned aerial vehicles (UAVs), comprising:

[0059] The global route optimization module is used to automatically generate a globally optimized route that covers the area to be measured and maintains a preset safe distance in any segment, based on a rough model of the area to be measured and historical manual planning data.

[0060] The UAV adjustment module is used to control the UAV to fly close to the globally optimized flight path and collect multi-view images. During the flight, the UAV's position and attitude are adjusted online based on the UAV's real-time status data and continuous aerial images to ensure image coverage and avoid obstacles in real time.

[0061] The downsampling module is used to input the multi-view images and the three-dimensional coordinates of the image control points deployed in the area to be measured into the aerial triangulation system to reconstruct the dense point cloud at the current moment; and to adaptively adjust the sampling density according to the geometric changes of the neighborhood of each point in the point cloud to maintain a high point cloud density in areas with abrupt terrain changes.

[0062] The point cloud registration module is used to input the adaptively downsampled current point cloud and the reference point cloud into an iterative nearest point registration algorithm with dynamic neighborhood search and feature weighting mechanisms to achieve accurate alignment of the two temporal point clouds.

[0063] The displacement field generation module is used to compute the three-dimensional displacement vectors of corresponding points based on the aligned two time phase points, and generate a continuous displacement field through multi-scale interpolation.

[0064] The risk assessment module is used to assess the risk of canal slope deformation by combining the displacement amount and spatial gradient of the displacement field, and output deformation monitoring results and early warning information.

[0065] The beneficial effects of this invention are as follows: By organically linking terrain-adaptive flight path planning, online dynamic obstacle avoidance, dense point cloud reconstruction, adaptive downsampling, improved ICP fine registration, and multi-scale displacement field construction, this invention forms a closed loop of "one flight → full-process monitoring". This invention achieves integrated automated monitoring from flight path design to risk warning, avoiding multiple manual interventions and switching between multiple devices, significantly improving overall monitoring efficiency and data consistency.

[0066] Furthermore, this invention provides two methods for obtaining a preliminary model: "on-site oblique photogrammetry" or "reusing historical DSM". This ensures that a high-precision terrain foundation is obtained during the initial deployment, and that a stable historical model can be quickly used in subsequent cycles, flexibly adapting to different project stages and reducing costs and on-site workload.

[0067] Furthermore, this invention utilizes Generative Adversarial Networks (GANs) to generate terrain-adaptive route probability maps online and extract optimal routes after offline training. Compared to empirical planning, the routes automatically generated by GANs better fit the terrain undulations and meet safe distance constraints, significantly shortening route design time and ensuring high coverage.

[0068] Furthermore, the discriminator of this invention uses image overlap and terrain adaptability as quality indicators to provide feedback on the probability map of candidate flight paths. This quantifies traditional empirical judgments into trainable evaluation criteria, enabling the generator to continuously optimize flight path quality during adversarial processes, ensuring that the generated flight paths simultaneously consider both "image quality" and "terrain flyability".

[0069] Furthermore, this invention introduces adversarial loss, IoU region overlap loss, and total variation regularization into adversarial training, and balances them with weights. The loss function simultaneously constrains "similarity to human route", "route smoothness", and "adversarial robustness", so that the automatically generated route is both consistent with human experience and smooth and feasible.

[0070] Furthermore, this invention takes a multi-dimensional state input, including historical state, obstacle distance, and terrain texture variance, and outputs continuous attitude fine-tuning commands. These commands are then optimized using a reward function that considers image coverage, safe distance, and endurance efficiency. This enables the UAV to fine-tune its attitude in real-time in complex terrain, improving tilt and overlap coverage, actively avoiding obstacles, and optimizing power consumption, thus achieving high-quality, stable, and continuous image acquisition.

[0071] Furthermore, this invention calculates obstacle threat weights through multi-frame convolution and spatiotemporal self-attention, predicts future waypoint risks, and generates climb, detour, or hovering commands online. When dynamic obstacles (birds, equipment, etc.) appear, it can quickly make decisions and correct course without returning to base or triggering large-scale disruptions, significantly reducing mission interruption rates and crash risks.

[0072] Furthermore, this invention combines ISS keypoint / FPFH coarse registration, RANSAC in-point screening, dynamic neighborhood shrinkage, normal constraint, curvature weighting, and Huber loss into an ICP fine registration process. This significantly improves the initial matching robustness and fine alignment accuracy of two-temporal point clouds, especially maintaining sub-centimeter registration error under vegetation shading or seasonal changes, ensuring high reliability of deformation calculation.

[0073] Furthermore, after eliminating outliers from the normal-curvature constraints, this invention performs moving least-squares interpolation on the discrete displacement vector field according to different node spacings in flat and steep regions, and generates a continuous displacement field using TIN. Through adaptive density refinement, millimeter-level minute deformation signals are preserved in the high-frequency region, while reducing data redundancy in flat regions, generating a smooth and refined continuous displacement field, thus improving the accuracy of deformation display and analysis.

[0074] Furthermore, this invention normalizes and linearly weights the displacement and gradient at each location to obtain a comprehensive risk score, dividing the risk into four levels: high, medium, low, and stable. By organically integrating different physical quantities, it quantitatively reflects the deformation hazard in a simple and easy-to-understand risk score format, and implements hierarchical management of the areas, providing an intuitive and operable decision-making basis for engineering early warning and safe operation and maintenance. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0076] Figure 2 This is a schematic diagram illustrating the principle of global flight path generation based on Generative Adversarial Network (GAN) in an embodiment.

[0077] Figure 3 Here is a flowchart illustrating the drone flight path planning and adjustment process using deep learning algorithms, as an example.

[0078] Figure 4The flowchart shows the point cloud registration process of the improved ICP algorithm as an example. Detailed Implementation

[0079] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments to facilitate a clear understanding of the present invention, but these descriptions do not constitute a limitation on the present invention.

[0080] Example 1

[0081] like Figure 1 As shown, this invention provides a method for monitoring canal slope deformation based on close-up photogrammetry using unmanned aerial vehicles (UAVs), comprising the following steps:

[0082] S1. Based on the approximate model of the area to be measured and historical manual planning data, automatically generate a globally optimized route that covers the area and maintains a preset safe distance in any flight segment;

[0083] S2, control the UAV to fly close to the globally optimized flight path and collect multi-view images; during the flight, adjust the position of the next waypoint and the attitude of the UAV online according to the real-time status data and continuous aerial images of the UAV to ensure image coverage and avoid obstacles in real time.

[0084] S3, input the multi-view images and the three-dimensional coordinates of the image control points deployed in the area to be measured into the aerial triangulation system to reconstruct the dense point cloud at the current moment; and adaptively adjust the sampling density according to the geometric changes of the neighborhood of each point in the point cloud to maintain a high point cloud density in areas with abrupt terrain changes.

[0085] S4, input the adaptively downsampled current point cloud and the reference point cloud into an iterative nearest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to complete the accurate alignment of the two temporal point clouds;

[0086] S5. Based on the aligned two time phase points, the three-dimensional displacement vectors of the corresponding points are calculated, and a continuous displacement field is generated by multi-scale interpolation.

[0087] S6, combining the displacement amount and spatial gradient of the displacement field, conducts a risk assessment of the canal slope deformation, and outputs deformation monitoring results and early warning information.

[0088] Specifically, in step S1, based on the area to be measured, and using conventional UAV oblique photogrammetry, the flight altitude is set to 150m, and the heading and lateral overlap rates are set to 80% and 70%, respectively, automatically generating the UAV flight path. A multi-lens UAV equipped with an RTK module is used to perform photogrammetry on the study area, acquiring multi-view images of the area to be measured. Uncontrolled aerial triangulation and dense matching are performed on the multi-view images to generate dense point cloud data and a digital surface model of the survey area. The digital surface model is used as a preliminary model of the study area. Alternatively, historical DSM data of the study area with no significant changes in topography in the recent past can be collected as a preliminary model.

[0089] A global route generation network is constructed using a rough model of the area to be measured and historical manually planned routes as core input data. The specific process includes the following steps:

[0090] Step 1.1, Data Preprocessing

[0091] The core input is a probabilistic digital surface model (DSM) of the monitoring area. Each DSM cell has a resolution of 1 m × 1 m, and the value represents the corresponding terrain height.

[0092] Multiple historical manually designed flight routes are divided into several local areas by grid. Each sub-block contains labels such as waypoint coordinates, typical flight altitude (30–100 m), heading angle (0°–360°), and image overlap rate (60%–80% horizontally and 70%–85% vertically), while also recording the regional slope data (10°–60°).

[0093] Step 1.2, Global Route Generation Network Structure Design

[0094] The global route generation network structure based on Generative Adversarial Networks (GANs) is as follows: Figure 2 As shown, a U-Net structure network with an attention mechanism is used as the generator, which consists of an encoder and a decoder.

[0095] The encoder extracts high-order features of the terrain (including slope, curvature, and concave / convex regions) through 4 layers of convolution (64→512 channels, 3×3 kernel size, stride 2).

[0096] The attention mechanism is embedded in the third layer of the encoder. It enhances the feature responses of steep slopes and crack areas through a spatial attention module, calculating the attention weight A(x,y) for each spatial location (x,y):

[0097]

[0098] In the formula, Let be the eigenvalue of the c-th channel in this layer at (x, y); These are learnable weights; A(x,y) is a sigmoid function; it reflects the importance of the terrain features at that location and is used to enhance the representation of areas such as cracks and steep slopes.

[0099] The decoder restores spatial resolution and reconstructs terrain features layer by layer through transposed convolutions (channel number from 512→256→128→64), and skip connections preserve details, finally outputting a route probability heatmap of the same size as the DSM.

[0100] The discriminator employs a 5-layer 3×3 convolutional network (stride 2), with the number of channels decreasing from 64 to 128 to 256 to 512 and then to 1. Finally, a sigmoid function is used to output the true / false probabilities. The discriminator primarily learns to distinguish the quality differences between the route probability blocks generated by the GAN and the human experience blocks. Its evaluation metric is:

[0101] Image overlap meets the standard: the percentage of flight segments with a horizontal overlap rate of ≥80% and a vertical overlap rate of ≥75% within a sub-block;

[0102] Terrain adaptability: Percentage of flight segments with a horizontal distance of more than 5 m between the flight path and the area with a slope of >30° on the DSM.

[0103] Step 1.3, Adversarial Training

[0104] In the offline phase, DSM heatmap tiles are input into the generator, and historical experience flight path data is input into the discriminator. Based on the discriminator feedback, the weights of each convolutional kernel and the number of channels in the generator are adjusted, and the total loss L is optimized through backpropagation until the generated flight path probability heatmap meets the preset quality requirements. Based on the discriminator feedback, the generator adjusts the weights of each convolutional kernel and the number of channels in the U-Net generator, and optimizes the loss function through backpropagation until the generated flight path meets the requirements of covering the entire target area, having a minimum distance to obstacles >3 m, and a heading angle change rate <5° / s.

[0105] The loss function for generative adversarial networks is:

[0106]

[0107] In the formula, To generate the total loss function for the adversarial network; To combat the losses; Losses due to overlap in the route area; This is the total variation regularization term; , and These are weighting coefficients, and recommended values ​​are 0.5, 0.3, and 0.2.

[0108]

[0109] In the formula, This is the total variation regularization term; The first line in the probability heatmap of the route output by the generator. line, number The pixel value in the column represents the probability (range 0-1) that the location is planned as a flight path; Represents the row and column indices of pixels in the heatmap.

[0110] Step 1.4, Heatmap Post-processing and Flight Line Extraction

[0111] The approximate digital surface model of the monitoring area is input into the trained generator, and non-maximum suppression is applied to the probability heatmap output by the generator to remove local suboptimal response points.

[0112] Set a probability threshold τ (e.g., 0.7) to retain all pixels with a probability ≥ τ;

[0113] Perform connectivity component analysis on the binarized heatmap, select the farthest / longest connectivity path covering the monitoring area, and smooth the node spacing to 5 m;

[0114] The selected main path is converted into a sequence of latitude and longitude waypoints, which serves as the globally optimized flight path for this UAV close-range photogrammetric flight.

[0115] Specifically, in step S2, the intelligent obstacle avoidance flight strategy is executed based on image control point deployment and deep reinforcement learning combined with the Transformer architecture, which includes the following steps:

[0116] Step 2.1: Measure the three-dimensional coordinates based on uniformly distributed image control points.

[0117] Image control points are evenly distributed throughout the monitoring area according to an equidistant grid (e.g., every 20 m × 20 m). For special terrains such as sloping canal slopes and cliffs, the image control points are designed as alternating "black and white fan-shaped" markers, which can be quickly located during manual field measurements and are also easy to identify in subsequent images.

[0118] The XYZ coordinates of each control point were measured using a total station or RTK.

[0119] The measured 3D coordinates of the image control points are stored in the ground control point database as auxiliary data for subsequent aerial triangulation reconstruction and registration.

[0120] Step 2.2, Pose Fine-tuning Based on Deep Reinforcement Learning (DRL)

[0121] The pose (latitude, longitude, altitude, roll, pitch, yaw) and control actions (throttle, yaw, etc.) of the UAV at the current and several historical moments during flight are stacked together with the Euclidean distance between the UAV and the nearest obstacle and the texture variance of the corresponding projection point in the terrain model (DSM) (quantifying the terrain undulation and the importance of the monitored target), forming a total of 32-dimensional state vector s. t .

[0122] Construct a policy network π containing several fully connected layers. θ (s t ), accept s t Then, a set of three-dimensional continuous control commands are output, which are mapped to fine-tuning amounts of roll angle, pitch angle and throttle thrust, respectively.

[0123] Synchronize s t Input value network V w (s t Estimate the value of the current state for subsequent advantage function calculation.

[0124] Set up rewards:

[0125]

[0126] In the formula, For deep reinforcement learning, the reward function is... The coverage rate of areas where the image overlap rate meets the standard; Distance to the nearest obstacle; This represents the remaining battery percentage. , and These are weighting coefficients, and recommended values ​​are 0.5, 0.3, and 0.2.

[0127] In simulation or small-scale flight test environments, the Proximal Policy Optimization (PPO) algorithm is used to iteratively update network parameters, enabling the policy to improve image coverage, increase obstacle avoidance distance, and optimize endurance efficiency. The trained policy network π... θ (s t Accept s t Then, a set of three-dimensional continuous control commands are output to control the flight status of the UAV in real time.

[0128] Step 2.3, Real-time obstacle avoidance based on spatiotemporal attention using Transformer

[0129] Five consecutive frames of RGB or depth aerial images in front of the drone are used to extract spatial feature vector sequences through a shared convolutional network. The sequence features are then mapped to query Q, K key, and V value vectors. Spatiotemporal information is aggregated through a multi-head attention mechanism to obtain the pixel-level threat weight distribution of obstacles and identify high-risk targets such as suspended cables and moving objects.

[0130] The spatiotemporal attention weights are calculated as follows:

[0131]

[0132] In the formula, This is the query vector, used to focus on the current location that needs to be processed; This is a key vector, representing the identifier of each position in the input sequence; It is a value vector that contains the actual information of the input sequence; The dimension of the key vector; This is the normalization function.

[0133] The pixel-level threat is mapped to the two-dimensional or three-dimensional location of the drone's next five waypoints, and the risk weight of each waypoint is calculated.

[0134] The system sets a threshold r for the risk weight. th When the predicted risk weight r of the i-th waypoint i >r th If the waypoint is deemed to have an unacceptable risk of collision or interference, the obstacle avoidance branch will be immediately initiated.

[0135] Obstacle avoidance strategy based on r i The size can be further subdivided:

[0136] If r i It's only slightly over the limit (e.g., r) th <r i <1.5 r th If so, the flight path will tend to be slightly detoured or the course will be slightly adjusted.

[0137] If r i Exceeding higher security thresholds (e.g., r) i >2 r th If it does, it will execute a more aggressive climb or hovering standby.

[0138] The obstacle avoidance commands (fly around, climb, hover) generated after the risk weight is triggered will be superimposed on the output control commands in step 2.2, and the two together determine the final attitude and heading at the next moment.

[0139] Specifically, in step S3, using the N sets of 3D coordinates of the image control points obtained in step 2.1 as the field control information, aerial triangulation and dense matching are performed on the multi-view images collected by the UAV along the globally optimized flight path to obtain a high-precision dense point cloud of the study area; subsequently, the dense point cloud is adaptively downsampled based on local geometric changes to balance processing efficiency and the preservation of details in key areas, as follows:

[0140] Step 3.1 Aerial triangulation + dense matching

[0141] The locations of N sets of image control points (XYZ) are imported into the SfM (Structure-from-Motion) software as ground control points (GCPs) to perform spatial orientation and scale constraints on the multi-view image sequences captured during flight. GCPs can significantly improve the absolute positioning accuracy of the reconstruction results and eliminate the drift of pure visual matching in absolute coordinates, scale, and direction. This ensures that the generated point cloud has a true scale and orientation in the geographic coordinate system, providing a unified reference for subsequent multi-time comparisons.

[0142] Based on camera parameters corrected by GCP, a dense point cloud of the entire region is generated using a multi-view matching algorithm (such as PMVS / CMVS, OpenMVS, COLMAP, etc.).

[0143] Step 3.2 Calculation and Zoning of Local Elevation Variance

[0144] To reduce data redundancy and improve point cloud data processing speed, the elevation variance of all point clouds within a spherical neighborhood with a radius of 0.5 m for each point cloud is calculated:

[0145]

[0146] In the formula, The elevation variance within the neighborhood of the point cloud; The total number of point clouds in the neighborhood; For the first Elevation values ​​of each point; This represents the average elevation of the point cloud within the neighborhood.

[0147] The region with an elevation variance ≤ 0.05 m² is classified as a flat region; the region with an elevation variance ≤ 0.2 m² is classified as a gradually changing region; and the region with an elevation variance > 0.2 m² is classified as a sudden change region.

[0148] Step 3.3 Adaptive Downsampling

[0149] The point cloud in the flat area is retained by the hierarchical downsampling method, with 50% retained in the flat area, 70% retained in the gradually changing area, and 90% retained in the abrupt change area, while ensuring that the point cloud density in the detailed area is >500 points / m².

[0150] Within each partition, random or grid sampling is performed, and the samples are selected according to the above proportions.

[0151] If the sampling density in a certain area is insufficient, the densest point cloud in the neighborhood can be sampled until the lower limit of 500 points / m² is met; the optimized point cloud is obtained, the number of points is greatly reduced and the accuracy of key areas is preserved; it provides lightweight input for subsequent point cloud registration, speeds up the registration process and improves the accuracy.

[0152] Specifically, in step S4, an improved Iterative Closest Point (ICP) algorithm is introduced into the adaptive downsampling point cloud obtained in step S3. It is divided into two stages: coarse registration (step 4.1) and fine registration (step 4.2). A dynamic optimization strategy is added in the fine registration stage to accelerate convergence while maintaining high accuracy.

[0153] Step 4.1 Coarse Registration – Key Feature Matching and Initial Transformation Estimation

[0154] Key points were extracted from the current point cloud and the reference point cloud (i.e., the point cloud at the reference time) using the ISS (Intrinsic Shape Signatures) algorithm, with a curvature threshold of 0.1 m. -1 The maximum suppression radius is 0.3 m;

[0155] Calculate the Fast Point Feature Histogram (FPFH) descriptor at each ISS keypoint, with a dimension of 33 and a search radius of 1 m;

[0156] The FPFH features of the two temporal point clouds are randomly sampled and matched using RANSAC. Pairs of corresponding points with a feature distance < 0.3 and an inlier ratio ≥ 70% are selected. Based on these pairs, the initial rigid transformation matrix T0 is quickly estimated. The current point cloud is then imported into the reference coordinate system to obtain the pre-transformed point cloud.

[0157] Step 4.2 Fine-tuning – Dynamic Neighborhood and Feature-Weighted Optimization

[0158] Based on the pre-transformed point cloud, perform the following iterative process until convergence:

[0159] The dynamic neighborhood search radius shrinks from an initial 1 m to 0.1 m during the k-th iteration.

[0160] Only point pairs with a normal vector angle < 15° are retained for matching to exclude inconsistent surfaces; a curvature-weighted strategy is adopted, with weights adjusted for regions with curvature > 0.1 / m. Set as ,in Point cloud curvature values, low curvature regions Set it to 0.8.

[0161] The Huber loss function is used when calculating the registration residuals to suppress the influence of outliers, suppress noise interference, and improve registration accuracy.

[0162] Simultaneously, an adaptive convergence criterion is designed to dynamically adjust the number of iterations and the error threshold based on the point cloud size.

[0163] The formula for adjusting the dynamic error convergence threshold is:

[0164]

[0165] In the formula, For the first The convergence threshold for the next iteration; The minimum convergence threshold is set (a value of 0.01 m is recommended). This is the attenuation coefficient (a value of 0.8 is recommended). This is the threshold value from the previous iteration.

[0166] For the total number of points in the input point cloud, dynamically calculate the maximum number of iterations:

[0167]

[0168] In the formula, To improve the dynamic maximum number of iterations in the ICP algorithm; The number of basic iterations (recommended value: 50); This is the scale impact coefficient (recommended value: 10). Input the total number of points in the point cloud.

[0169] Each iteration updates the transformation matrix based on the search radius, normal / curvature weights, and Huber loss, until the transformation increment is less than ε. k If the maximum number of iterations is reached, the fine registration is considered complete.

[0170] Output the final rigid transformation matrix to precisely align the point cloud at the current time with the point cloud at the reference time.

[0171] Specifically, in step S5, the input is a two-phase point cloud that has been finely registered, and the output is a high-density continuous displacement field.

[0172] Using the previous point cloud as the reference point cloud, the nearest neighbor point of the next point cloud in the reference point cloud is matched point by point. Valid point pairs are selected based on the joint constraint of normal vector and curvature (normal vector angle <15° and curvature difference <0.05 / m), and the displacement of all retained point pairs is calculated.

[0173] By combining the Laida criterion to remove outliers with displacements exceeding 3 times the standard deviation, the three-dimensional displacement vector is calculated point by point.

[0174] Based on the baseline point cloud or DSM slope information, the study area is divided into:

[0175] In flat areas (slope < a certain threshold), the node spacing is set to 0.2 m;

[0176] For steep slopes / cracked areas (slope ≥ threshold), the node spacing is set to 0.05 m.

[0177] Generate an interpolation node grid across the entire region at the aforementioned intervals;

[0178] For each interpolation node, select several nearest discrete displacement vectors and fit the continuous displacement at that point using a weighted least squares method. The weights can be Gaussian kernels or inverse distance weights to ensure that the sampling density matches the node spacing.

[0179] Construct an irregular triangular mesh (TIN) by connecting all interpolation nodes with their corresponding displacement values. Interpolate on the TIN using triangular elements as the basis to form a high-density continuous displacement field. The displacement vector or displacement amplitude distribution map at any location can be output as needed.

[0180] Specifically, in step S6, the high-density continuous displacement field generated in step S5 is used as input. The displacement amount and displacement gradient are weighted and scored to quantitatively assess the channel slope deformation risk and divide the monitoring area into four sub-regions. The specific steps are as follows:

[0181] For each interpolation node in the continuous displacement field, the magnitude of its displacement vector is calculated as the displacement amount;

[0182] The displacement gradient is obtained by performing local difference on the continuous displacement field or by calculating the gradient magnitude on the triangular mesh element.

[0183] The overall risk value is calculated using the following formula:

[0184]

[0185] In the formula, This is the overall risk value; This is the displacement, in mm; This represents the displacement gradient, in mm / m. and These are weighting coefficients, and it is recommended that they be 0.6 and 0.4 respectively.

[0186] Will The area is defined as a high-risk area; The area is defined as a medium-risk area; The area is defined as a low-risk zone; The region is defined as the stable region.

[0187] The risk level of each interpolation node is mapped back to the triangular mesh element of the continuous displacement field to generate a four-color risk zoning map; at the same time, statistical information such as the area and average risk value of each sub-region is output to facilitate subsequent early warning and operation and maintenance decisions.

[0188] Example 2

[0189] This invention provides a canal slope deformation monitoring system based on close-up photogrammetry using unmanned aerial vehicles (UAVs), comprising:

[0190] The global route optimization module is used to automatically generate a globally optimized route that covers the area to be measured and maintains a preset safe distance in any segment, based on a rough model of the area to be measured and historical manual planning data.

[0191] The UAV adjustment module is used to control the UAV to fly close to the globally optimized flight path and collect multi-view images. During the flight, the UAV's position and attitude are adjusted online based on the UAV's real-time status data and continuous aerial images to ensure image coverage and avoid obstacles in real time.

[0192] The downsampling module is used to input the multi-view images and the three-dimensional coordinates of the image control points deployed in the area to be measured into the aerial triangulation system to reconstruct the dense point cloud at the current moment; and to adaptively adjust the sampling density according to the geometric changes of the neighborhood of each point in the point cloud to maintain a high point cloud density in areas with abrupt terrain changes.

[0193] The point cloud registration module is used to input the adaptively downsampled current point cloud and the reference point cloud into an iterative nearest point registration algorithm with dynamic neighborhood search and feature weighting mechanisms to achieve accurate alignment of the two temporal point clouds.

[0194] The displacement field generation module is used to compute the three-dimensional displacement vectors of corresponding points based on the aligned two time phase points, and generate a continuous displacement field through multi-scale interpolation.

[0195] The risk assessment module is used to assess the risk of canal slope deformation by combining the displacement amount and spatial gradient of the displacement field, and output deformation monitoring results and early warning information.

[0196] This system, based on a GAN-based route generation framework, integrates historical manually planned data (200 sets) with DSM terrain features (slope, curvature), reducing route generation time to 2.3 h / km. 2 Reduced to 0.5 h / km 2 The coverage of steep slope areas (slope > 30°) increased from 75% to 98%, and the image overlap rate stabilized at ≥ 85% horizontally and ≥ 80% vertically.

[0197] The PPO policy network adjusts the UAV's attitude in real time (roll and pitch angle control accuracy ±0.5°), and combined with the Transformer architecture, predicts the next 5 waypoints (confidence >90%). The data acquisition interruption rate is reduced from 18% to 3%, and the crash risk is reduced to 1.2%.

[0198] The improved ICP algorithm, through curvature weighting strategy and dynamic search radius, reduced the point cloud registration error from 2.1 cm to 0.8 cm. Combined with multi-scale displacement field densification (0.05 m in steep slope area and 0.2 m in flat area), the deformation signal signal-to-noise ratio (SNR) increased from 3 dB to 12 dB, and the monitoring accuracy reached 0.5 cm.

[0199] Based on a weighted scoring model of displacement and gradient, the accuracy of high-risk area identification has been improved to 92%, providing graded early warning data (high-risk, medium-risk, low-risk, and stable areas) for the operation and maintenance of water conservancy projects.

[0200] Example 3

[0201] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the canal slope deformation monitoring method based on UAV close-up photogrammetry described in the above technical solution.

[0202] Example 4

[0203] The present invention also provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the canal slope deformation monitoring method based on UAV close-up photogrammetry described above.

[0204] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0205] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0206] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0207] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0208] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A method for monitoring the deformation of a canal slope based on close-range photogrammetry of a UAV, characterized in that: Includes the following steps: Based on a rough model of the area to be measured and historical manually planned data, a globally optimized route that covers the area and maintains a preset safe distance in any flight segment is automatically generated. The drone is controlled to fly close to the globally optimized flight path and collect multi-view images. During the flight, the position of the next waypoint and the attitude of the drone are adjusted online based on the drone's real-time status data and continuous aerial images to ensure image coverage and avoid obstacles in real time. The multi-view images and the three-dimensional coordinates of the image control points deployed in the area to be measured are input into the aerial triangulation system to reconstruct the dense point cloud at the current moment; and the sampling density is adaptively adjusted according to the geometric changes of the neighborhood of each point in the point cloud to maintain a high point cloud density in areas with abrupt terrain changes. The current point cloud with adaptively adjusted sampling density and the reference point cloud are input into an iterative nearest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to achieve accurate alignment of the two temporal point clouds. Based on the aligned two time-phase points, the three-dimensional displacement vectors of corresponding points are calculated, and a continuous displacement field is generated through multi-scale interpolation. Risk assessment of canal slope deformation is conducted by combining the displacement amount and spatial gradient of the displacement field; The execution process of the iterative nearest point registration algorithm includes: Based on the ISS algorithm, key points with curvature greater than a preset threshold are extracted from the point cloud at the current time, and FPFH feature descriptors are calculated for each key point; The FPFH descriptors are matched using the RANSAC algorithm to filter matching pairs where the distance between the FPFH feature descriptors is less than a preset threshold and the proportion of points in RANSAC is not less than a preset threshold. The initial rigid transformation matrix is ​​then estimated based on the matching pair. The initial rigid transformation matrix is ​​applied to the point cloud at the current time step as the initial value for the fine registration iteration to obtain the pre-transformed point cloud; Perform iterative nearest-point fine registration on the pre-transformed point cloud: In each iteration, the neighborhood search radius is gradually reduced from the initial value to the minimum threshold; Only point pairs whose normal vector angle is within a preset threshold are retained for matching; Point pairs with curvature greater than a preset value are assigned higher matching weights, while point pairs with low curvature are assigned lower weights. The Huber robust loss function is used to process the residuals. Repeat the fine registration process until the transformation increment is lower than the convergence threshold, and complete the precise alignment of the point cloud at the current time with the point cloud at the reference time. The generation process of the continuous displacement field includes: Using the point cloud at the reference time as a reference, the nearest neighbor point is searched point by point in the point cloud at the current time, and the normal vector-curvature joint constraint is applied. Only point pairs with a normal angle less than a preset threshold and a curvature difference less than a preset threshold are retained. The three-dimensional displacement vector is calculated for the retained point pairs, and outliers with displacements exceeding three times the standard deviation are removed according to the Laida criterion to obtain the discrete displacement vector field. The discrete displacement vector field is interpolated and densified using the moving least squares method: a first preset node spacing is used in flat areas; and a second preset node spacing smaller than the first spacing is used in steep slopes or areas with dense cracks. An irregular triangular network is constructed based on the encrypted displacement point cloud to generate a continuous displacement field.

2. The method according to claim 1, characterized in that: The process of obtaining the globally optimized flight path includes: Offline training phase: A rough model of the area to be measured and historical manually planned flight route data are simultaneously input into a generative adversarial network (GAN) flight route planning model. The GAN includes: A generator is used to generate a probability map of candidate routes based on a probabilistic model; The discriminator judges the quality of the candidate route probability map based on historical manual route data and feeds the judgment result back to the generator. Through multiple rounds of adversarial training between the generator and the discriminator until they reach stable convergence, the final network parameters of the generator are determined. Online generation stage: The probabilistic model is input into the trained generator to generate a probability map of flight routes covering the region. Non-maximum suppression is first applied to the route probability map, and then connectivity extraction is performed based on the probability threshold to obtain a continuous globally optimized route. This globally optimized flight path will be used as the initial flight path output for subsequent UAV close-up photogrammetry flights.

3. The method according to claim 2, characterized in that: During the offline training phase, the discriminator evaluates the quality of the image overlap and terrain adaptability of the route probability map output by the generator based on historical manually planned route data, and feeds the evaluation results back to the generator.

4. The method according to claim 2, characterized in that: During the adversarial training phase, the generator's loss function includes: The adversarial loss term is used to constrain the game balance between the generator and the discriminator. The route area intersection and union ratio loss term is used to constrain the regional overlap between generated routes and historical manual routes; The total variation regularization term is used to constrain the spatial smoothness of the route probability graph. Each loss item is summed by weighting it according to a predetermined weighting coefficient.

5. The method according to claim 1, characterized in that: During flight, an attitude control policy network based on deep reinforcement learning is constructed; The input state vector of the policy network includes: the pose and motion stacking information of the UAV at the current and several historical moments; the relative distance to environmental obstacles; and the terrain texture variance features extracted from the general model. The strategy network outputs three-dimensional continuous control commands in real time, which are used to fine-tune the roll angle, pitch angle and throttle thrust of the UAV online. The reward function of the policy network is based on a comprehensive evaluation of three indicators: image coverage quality, obstacle safety distance, and remaining battery power. Accordingly, the improvement of image coverage, the increase of obstacle distance, and the improvement of battery life are set as positive rewards. Using this reward function, the parameters of the policy network are iteratively updated using a near-end policy optimization algorithm.

6. The method according to claim 1, characterized in that: During flight, convolutional features are extracted from multiple frames of aerial images continuously captured in front of the drone, and the threat weight of each obstacle pixel is calculated through spatiotemporal self-attention. Based on the threat weights, calculate the risk weights at several future waypoints; When the risk weight of any predicted waypoint exceeds a preset threshold, obstacle avoidance correction commands are generated in real time to adjust the drone's flight path or to avoid dynamic obstacles online by climbing or hovering.

7. The method according to claim 1, characterized in that: The risk assessment process includes: The displacement value and displacement gradient value at each position in the displacement field are normalized according to the preset weighting coefficients. The standardized displacement value and the standardized displacement gradient value are combined in a linear weighted manner to obtain the comprehensive risk score of the location. Based on the comprehensive risk score, the area to be measured is divided into four categories: high-risk area, medium-risk area, low-risk area, and stable area.

8. A canal slope deformation monitoring system based on UAV close-up photogrammetry, used to implement the method described in any one of claims 1-7, characterized in that: include: The global route optimization module is used to automatically generate a globally optimized route that covers the area to be measured and maintains a preset safe distance in any segment, based on a rough model of the area to be measured and historical manual planning data. The UAV adjustment module is used to control the UAV to fly close to the globally optimized flight path and collect multi-view images. During the flight, the UAV's position and attitude are adjusted online based on the UAV's real-time status data and continuous aerial images to ensure image coverage and avoid obstacles in real time. The downsampling module is used to input the multi-view images and the three-dimensional coordinates of the image control points deployed in the area to be measured into the aerial triangulation system to reconstruct the dense point cloud at the current moment; and to adaptively adjust the sampling density according to the geometric changes of the neighborhood of each point in the point cloud to maintain a high point cloud density in areas with abrupt terrain changes. The point cloud registration module is used to input the current point cloud with adaptively adjusted sampling density and the reference point cloud into an iterative nearest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to achieve accurate alignment of the two temporal point clouds. The displacement field generation module is used to compute the three-dimensional displacement vectors of corresponding points based on the aligned two time phase points, and generate a continuous displacement field through multi-scale interpolation. The risk assessment module is used to assess the risk of canal slope deformation by combining the displacement amount and spatial gradient of the displacement field, and output deformation monitoring results and early warning information.

Images