Canal slope deformation monitoring method based on unmanned aerial vehicle approach photogrammetry

Abstract

The invention provides a channel slope deformation monitoring method based on unmanned aerial vehicle approach photogrammetry. The method comprises the following steps: automatically generating a global optimization route; performing online adjustment on the position of the next waypoint and the attitude of the unmanned aerial vehicle; inputting the multi-view image and three-dimensional coordinates of image control points arranged in the to-be-measured area into an aerial triangulation system, and reconstructing a dense point cloud at the current moment; the sampling density is adaptively adjusted according to the geometric change of the neighborhood of each point of the point cloud; inputting the point cloud at the current moment and the point cloud at the reference moment after adaptive downsampling into an iterative nearest point registration algorithm with dynamic neighborhood search and a feature weighting mechanism to finish accurate alignment of the point clouds at the two time phases; calculating a three-dimensional displacement vector of each corresponding point according to the two aligned time phase point clouds, and generating a continuous displacement field through multi-scale interpolation; and carrying out risk assessment on the channel slope deformation by combining the displacement amount of the displacement field and the spatial gradient thereof. According to the invention, the precision and efficiency of channel slope deformation monitoring are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned aerial vehicle remote sensing monitoring, and particularly relates to a channel slope deformation monitoring method based on unmanned aerial vehicle close-range photogrammetry. BACKGROUND

[0002] Channel slope deformation monitoring is an important basis for water conservancy project safety operation and maintenance and geological disaster early warning. The traditional channel slope deformation monitoring method mainly relies on ground measurement technologies such as total station and GNSS (Global Navigation Satellite System). Although these methods have high accuracy, they have the disadvantages of low measurement efficiency, high labor and economic costs, especially in complex terrain areas such as steep slopes and cracks, it is difficult to effectively implement continuous and comprehensive monitoring, and the terrain coverage is limited.

[0003] In recent years, unmanned aerial vehicle aerial photogrammetry technology has been widely used in channel slope deformation monitoring, greatly improving the monitoring efficiency. However, the existing unmanned aerial vehicle aerial photogrammetry technology also has many shortcomings, mainly as follows:

[0004] (1) The flight route planning method with fixed flight height and speed cannot effectively adapt to the steep and undulating terrain characteristics of the channel slope, resulting in insufficient image overlap at the top of the channel slope, and insufficient image resolution at the bottom of the channel slope due to the relatively high flight height.

[0005] (2) Static flight route planning cannot respond to dynamic obstacles in the flight process, such as birds, temporary construction equipment, etc., resulting in a high risk of unmanned aerial vehicle crash. In addition, the existing technology usually relies on threshold triggering to avoid obstacles by hovering or returning, which frequently interrupts the task, resulting in low image data collection efficiency and significant decline in the consistency of reflight data in image overlap.

[0006] (3) In the data processing stage, the classic Iterative Closest Point (ICP) algorithm is generally used for registration of point clouds in different periods, but this algorithm is sensitive to the initial position of the point cloud. Under the conditions of vegetation obstruction or seasonal feature changes (such as snow cover), the registration error is obviously accumulated, especially in high-curvature areas, the detail features are lost seriously, and it is difficult to meet the required high-precision monitoring requirements.

[0007] (4) The existing displacement field construction technology usually uses a fixed node spacing moving least squares method, which cannot effectively capture high-frequency deformation signals in steep slopes and crack areas, resulting in a smooth displacement field that cannot accurately express millimeter-level micro-deformation information, severely restricting the accuracy and reliability of channel slope deformation monitoring.

[0008] The above-mentioned existing technical problems greatly limit the application effect of channel slope deformation monitoring technology in actual engineering, and therefore there is an urgent need for a more efficient, accurate and adaptable channel slope deformation monitoring method to meet the actual needs of water conservancy project safety operation and maintenance and geological disaster early warning. SUMMARY

[0009] The present application aims to solve the above problems in the background art, and provides a channel slope deformation monitoring method based on unmanned aerial vehicle close photogrammetry, which realizes terrain-adaptive unmanned aerial vehicle close photogrammetry flight path planning and dynamic obstacle avoidance, and significantly improves the accuracy and efficiency of channel slope deformation monitoring.

[0010] The technical scheme adopted by the present application is: a channel slope deformation monitoring method based on unmanned aerial vehicle close photogrammetry, comprising the following steps:

[0011] According to the rough model of the region to be measured and the historical manual planning data, a globally optimized flight path covering the region and maintaining a predetermined safety distance at any flight segment is automatically generated;

[0012] The unmanned aerial vehicle is controlled to fly close to the globally optimized flight path and collect multi-view images; during the flight process, the next flight point position and the unmanned aerial vehicle attitude are adjusted online according to the real-time state data of the unmanned aerial vehicle and the continuous aerial image, so as to ensure the image coverage rate and avoid obstacles in real time;

[0013] The multi-view images and the image control point three-dimensional coordinates arranged in the region to be measured are input into an aerial triangulation system to reconstruct a dense point cloud at the current time; and the sampling density is adaptively adjusted according to the geometric changes of the neighborhood of each point in the point cloud, so as to maintain a high point cloud density in the terrain mutation area;

[0014] The current time point cloud after adaptive downsampling and the reference time point cloud are input into an iterative closest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to complete the accurate alignment of the two time phase point clouds;

[0015] The three-dimensional displacement vectors of each corresponding point are calculated according to the aligned two time phase point clouds, and a continuous displacement field is generated through multi-scale interpolation;

[0016] The displacement and its spatial gradient of the displacement field are combined to evaluate the risk of channel slope deformation, and the deformation monitoring result and early warning information are output.

[0017] In the above technical scheme, the process of obtaining the globally optimized flight path comprises:

[0018] Offline training stage:

[0019] The rough model of the region to be measured and the historical manual planning flight path data are simultaneously input into a generative adversarial network flight path planning model, and the generative adversarial network comprises:

[0020] A generator for generating a candidate flight path probability map according to the rough model;

[0021] The discriminator judges the quality of the candidate flight path probability map based on historical manual flight path data, and feeds back the judgment result to the generator;

[0022] Through multi-round adversarial training of the generator and the discriminator, until the two reach stable convergence, the final network parameters of the generator are determined;

[0023] In the online generation phase:

[0024] The rough model is input into the trained generator to generate a flight path probability map covering the area;

[0025] The flight path probability map is first subjected to non-maximum suppression, and then connectivity extraction is performed based on a probability threshold to obtain a continuous global optimization flight path;

[0026] The global optimization flight path is output as the initial flight path for subsequent UAV close photogrammetry flight.

[0027] In the above technical solution, the discriminator judges the image overlap degree of the flight path probability map output by the generator and the terrain adaptability based on historical manual planning flight path data in the offline training phase, and feeds back the judgment result to the generator.

[0028] In the above technical solution, in the adversarial training phase, the loss function of the generator includes:

[0029] An adversarial loss term is used to constrain the game balance between the generator and the discriminator.

[0030] A flight path area intersection ratio loss term is used to constrain the area coincidence degree of the generated flight path and the historical manual flight path.

[0031] A total variation regularization term is used to constrain the spatial smoothness of the flight path probability map.

[0032] Each loss term is weighted and summed according to a predetermined weight coefficient.

[0033] In the above technical solution, in the flight process, a pose control strategy network based on deep reinforcement learning is constructed;

[0034] The input state vector of the strategy network includes: the current and historical pose and action stacking information of the UAV; the relative distance to the environmental obstacles; and the terrain texture variance features extracted from the rough model.

[0035] The strategy network outputs three-dimensional continuous control instructions in real time, which are used to fine-tune the roll angle, pitch angle and throttle thrust of the UAV online.

[0036] The reward function of the policy network comprehensively evaluates three indexes of image coverage quality, obstacle safety distance and remaining power, and sets the increase of image coverage rate, obstacle distance and endurance efficiency as positive rewards. By using the reward function, the policy network parameters are iteratively updated by using the proximal policy optimization algorithm.

[0037] In the technical solution, during flight, convolution feature extraction is performed on multiple frames of aerial images continuously collected in front of the unmanned aerial vehicle, and threat weights of pixels of each obstacle are calculated through spatio-temporal self-attention.

[0038] Based on the threat weights, risk weights at future waypoints are calculated.

[0039] When the risk weight of any predicted waypoint exceeds a preset threshold, an obstacle avoidance correction instruction is generated in real time to adjust the flight path of the unmanned aerial vehicle or to realize online avoidance of dynamic obstacles by climbing or hovering.

[0040] In the technical solution, the execution process of the iterative closest point registration algorithm includes:

[0041] Based on the ISS algorithm, key points with a curvature greater than a preset threshold are extracted from the point cloud at the current time, and an FPFH feature descriptor is calculated for each key point.

[0042] The RANSAC algorithm is used to match the FPFH descriptors to screen matching pairs with a distance less than a preset threshold and a RANSAC inlier ratio not lower than a preset threshold, and an initial rigid transformation matrix is estimated based on the matching pairs.

[0043] The initial rigid transformation matrix is applied to the point cloud at the current time as an initial value for iterative fine registration to obtain a pre-transformed point cloud.

[0044] Iterative closest point fine registration is performed on the pre-transformed point cloud.

[0045] In each iteration, the neighborhood search radius is gradually reduced from an initial value to a minimum threshold.

[0046] Only point pairs with a normal vector included within a preset threshold are allowed to participate in matching.

[0047] Point pairs with a curvature greater than a preset value are given a higher matching weight, and point pairs with a low curvature are given a lower weight, and a Huber robust loss function is used to process residuals.

[0048] The fine registration process is repeatedly performed until the transformation increment is lower than a convergence threshold, and the point cloud at the current time is accurately aligned with the reference point cloud.

[0049] In the technical solution, the generation process of the continuous displacement field includes:

[0050] With the reference time point cloud as reference, the nearest neighbor points of the current time point cloud are searched point by point, and a normal vector-curvature joint constraint is applied, and only the point pairs with a normal angle less than a preset threshold and a curvature difference less than a preset threshold are reserved;

[0051] The three-dimensional displacement vectors of the reserved point pairs are calculated, and the outliers with a displacement exceeding three times the standard deviation are removed according to the Lilliefors criterion to obtain a discrete displacement vector field;

[0052] The discrete displacement vector field is interpolated and encrypted by using a moving least squares method: a first preset node spacing is used in a flat area; and a second preset node spacing smaller than the first spacing is used in a steep slope or a crack dense area;

[0053] An irregular triangle net is constructed based on the encrypted displacement point cloud to generate a continuous displacement field.

[0054] In the above technical solution, the risk assessment process comprises:

[0055] The displacement value and the displacement gradient value of each position in the displacement field are normalized according to a preset weight coefficient, respectively;

[0056] The normalized displacement value and the normalized displacement gradient value are combined in a linear weighting manner to obtain a comprehensive risk score of the position;

[0057] According to the size of the comprehensive risk score, the to-be-measured area is divided into four categories: a high-risk area, a medium-risk area, a low-risk area and a stable area.

[0058] The application also provides a channel slope deformation monitoring system based on close photogrammetry of a UAV, comprising:

[0059] A global optimization flight path module is configured to automatically generate a global optimization flight path covering the to-be-measured area and maintaining a preset safety distance at any flight segment according to a sketch model of the to-be-measured area and historical artificial planning data;

[0060] A UAV adjustment module is configured to control the UAV to fly close to the global optimization flight path and collect multi-view images; during the flight, the next flight point position and the UAV attitude are adjusted online according to real-time state data of the UAV and continuous aerial photography images, so as to ensure image coverage and avoid obstacles in real time;

[0061] A downsampling module is configured to input the multi-view images and three-dimensional coordinates of image control points arranged in the to-be-measured area into an aerial triangulation system to reconstruct a dense point cloud at the current time; and the sampling density is adaptively adjusted according to the geometric changes of the neighborhood of each point of the point cloud, so as to maintain a high point cloud density in a topographic mutation area;

[0062] a point cloud registration module, configured to input the current time point cloud after adaptive down-sampling and a reference time point cloud into an iterative closest point registration algorithm with a dynamic neighborhood search and feature weighting mechanism, to complete accurate alignment of the two time point clouds;

[0063] a displacement field generation module, configured to calculate three-dimensional displacement vectors of corresponding points according to the aligned two time point clouds, and generate a continuous displacement field through multi-scale interpolation;

[0064] a risk assessment module, configured to perform risk assessment on the deformation of the ditch slope in combination with the displacement amount and spatial gradient of the displacement field, and output a deformation monitoring result and early warning information.

[0065] The present application has the advantages that: the present application organically connects the links of terrain adaptive route planning, online dynamic obstacle avoidance, dense point cloud reconstruction, adaptive down-sampling, improved ICP precise registration and multi-scale displacement field construction, and forms a closed loop of "one flight -> whole-process monitoring". The present application realizes integrated and automated monitoring from route design to risk warning, avoids multiple manual interventions and multiple equipment switching, and significantly improves the overall monitoring efficiency and data continuity.

[0066] Further, the present application provides two ways of "on-site oblique photogrammetry" or "reuse of historical DSM" to obtain a rough model. The high-precision terrain basis can be ensured when deployed for the first time, and the stable historical model can be quickly used in subsequent periods, which flexibly adapts to different project stages and reduces costs and on-site workload.

[0067] Further, the present application uses a generative adversarial network to generate a terrain-adaptive flight route probability map and extract an optimal flight route online after offline training. Compared with experience planning, the flight route automatically generated by the GAN is more suitable for terrain undulations and meets the safety distance constraint, which can greatly shorten the flight route design time and ensure high coverage.

[0068] Further, the discriminator of the present application takes image overlap and terrain adaptability as quality indicators to give feedback to the candidate flight route probability map. The traditional experience is quantified as a trainable evaluation standard, so that the generator continuously optimizes the flight route quality in the adversarial process, ensuring that the generated flight route takes into account both "imaging quality" and "terrain flyability".

[0069] Further, the present application introduces an adversarial loss, an IoU region overlap loss and a total variation regularization in the adversarial training, and balances the weights. The loss function simultaneously constrains the "similarity to manual flight route", "flight route smoothness" and "adversarial robustness", so that the automatically generated flight route is both in line with human experience and smooth and feasible.

[0070] Further, the present application inputs multi-dimensional state including historical state, obstacle distance and terrain texture variance, outputs continuous attitude fine-tuning instructions, and optimizes reward functions of image coverage, safety distance and endurance efficiency. The unmanned aerial vehicle can fine-tune the attitude in complex terrain in real time, improve the lateral tilt and overlap coverage rate, actively avoid obstacles and optimize power consumption, and realize high-quality and stable continuous image acquisition.

[0071] Further, the present application calculates obstacle threat weight through multi-frame convolution and space-time self-attention, predicts future waypoint risk and generates climbing, circling or hovering instructions online. When dynamic obstacles (birds, equipment, etc.) appear, the unmanned aerial vehicle can quickly make decisions and correct the flight path without returning or triggering a large-scale interruption, thereby significantly reducing the task interruption rate and crash risk.

[0072] Further, the present application combines ISS key points / FPFH coarse registration, RANSAC inlier screening, dynamic shrinkage neighborhood, normal constraint, curvature weighting and Huber loss ICP fine registration process. The initial matching robustness and fine alignment accuracy of two time-phase point clouds are greatly improved, and sub-centimeter registration error can still be maintained under the conditions of vegetation obstruction or seasonal change, thereby ensuring high reliability of deformation calculation.

[0073] Further, the present application performs moving least squares interpolation on the discrete displacement vector field according to different node spacings of flat areas and steep slope areas after removing outliers under the normal-curvature constraint, and generates a continuous displacement field by TIN. Through adaptive density refinement, millimeter-level micro-deformation signals are reserved in the high-frequency area, while the data redundancy in the flat area is reduced, thereby generating a continuous displacement field that is both smooth and fine, and improving the deformation display and analysis accuracy.

[0074] Further, the present application normalizes and linearly weights the displacement amount and gradient of each position to obtain a comprehensive risk score, and divides the position into high / medium / low / stable four risk levels. Different physical quantities are organically integrated to quantitatively reflect the deformation danger in the form of a simple and understandable risk score, and the region is classified and managed, thereby providing intuitive and operable decision basis for engineering early warning and safe operation. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 The figure is a schematic diagram of the method of the present application;

[0076] Figure 2 The figure is a schematic diagram of the method of the present application;

[0077] Figure 3 The figure is a schematic diagram of the method of the present application;

[0078] Figure 4A point cloud registration flowchart of the improved ICP algorithm for the embodiment. DETAILED DESCRIPTION

[0079] The application will be further described below in connection with the drawings and specific embodiments, so as to make the application clear, but they do not constitute limitations to the application.

[0080] Embodiment 1

[0081] As shown in the drawings, Figure 1 The application provides a channel and slope deformation monitoring method based on close photogrammetry of a UAV, comprising the following steps:

[0082] S1, automatically generating a globally optimized flight route covering a to-be-measured region and maintaining a preset safety distance at any flight segment according to a sketch model of the to-be-measured region and historical manual planning data;

[0083] S2, controlling the UAV to fly close to the globally optimized flight route and collect multi-view images; in the flight process, adjusting the position of the next flight point and the attitude of the UAV online according to real-time state data of the UAV and continuous aerial photography images, so as to ensure image coverage and avoid obstacles in real time;

[0084] S3, inputting the multi-view images and three-dimensional coordinates of image control points arranged in the to-be-measured region into an aerial triangulation system to reconstruct a dense point cloud at the current time; and adaptively adjusting the sampling density according to the geometric changes of the neighborhood of each point of the point cloud, so as to maintain high point cloud density in the terrain mutation area;

[0085] S4, inputting the current time point cloud after adaptive downsampling and the reference time point cloud into an iterative closest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to complete accurate alignment of the two time phase point clouds;

[0086] S5, calculating three-dimensional displacement vectors of each corresponding point according to the aligned two time phase point clouds, and generating a continuous displacement field through multi-scale interpolation;

[0087] S6, combining the displacement amount and spatial gradient of the displacement field to perform risk assessment on channel and slope deformation, and outputting deformation monitoring results and early warning information.

[0088] Specifically, in step S1, according to the range of the region to be measured, based on the conventional unmanned aerial photogrammetry method, the flight height is set to 150 m, the heading and lateral overlap rates are set to 80% and 70% respectively, and the unmanned aerial flight route is automatically generated. The multi-lens unmanned aerial vehicle equipped with an RTK module is used to carry out photogrammetry on the study area, and multi-view images of the region to be measured are collected. The multi-view images are subjected to uncontrolled aerial triangulation and dense matching to generate dense point cloud data and a digital surface model of the survey area, and the digital surface model is taken as a rough model of the study area. Or collect historical DSM data of the study area with no significant changes in topography and geomorphology as a rough model.

[0089] A global route generation network is constructed with the rough model of the region to be measured and the historical manually planned route as the core input data, specifically including the following processes:

[0090] Step 1.1, data preprocessing

[0091] The rough digital surface model (DSM) of the monitoring area is taken as the core input, and each DSM cell has a resolution of 1 m x 1 m, with the value representing the corresponding terrain height.

[0092] The multiple historical manually designed routes are divided into several local areas by grid, and each sub-block contains route point coordinates, normal flight height (30-100 m), heading angle (0°-360°) and image overlap rate (lateral 60%-80%, vertical 70%-85%) and other labels, and the regional slope (10°-60°) data is also recorded.

[0093] Step 1.2, design of global route generation network structure

[0094] The global route generation network structure based on generative adversarial network (GAN) is shown in Figure 2 The U-Net structure network with attention mechanism is used as the generator, and the generator is composed of an encoder and a decoder.

[0095] The encoder extracts high-order features of the terrain (including slope, curvature, concave and convex areas) through 4 layers of convolution (channel number 64→512, kernel size 3×3, step 2).

[0096] The attention mechanism is embedded in the 3rd layer of the encoder, and the spatial attention module is used to enhance the feature response of steep slope and crack area. The attention weight A(x,y) is calculated for each spatial position (x,y):

[0097]

[0098] wherein, is the feature value of the cth channel at (x,y) of the layer; is a learnable weight; is the Sigmoid function; A(x, y) reflects the importance of the terrain characteristics at this position, which is used to enhance the expression of areas such as cracks and steep slopes.

[0099] The decoder recovers the spatial resolution of the terrain features layer by layer through transposed convolution (channel number: 512→256→128→64), and the skip connection preserves the details. Finally, the output is a flight path probability heat map with the same size as the DSM.

[0100] The discriminator adopts a 5-layer 3×3 convolutional network (stride 2), with the number of channels being 64→128→256→512→1, and finally outputs the true and false probabilities through Sigmoid. The discriminator mainly learns to distinguish the quality difference between the flight path probability blocks generated by the GAN and the artificial experience blocks, and its evaluation index is:

[0101] Image overlap standard: the proportion of flight segments with horizontal overlap rate ≥80% and vertical overlap rate ≥75% within the sub-block;

[0102] Terrain adaptability: the proportion of flight segments with horizontal distance >5 m in the area where the flight path and the DSM slope >30°.

[0103] Step 1.3, adversarial training

[0104] In the offline stage, input the DSM heat map block into the generator and the historical experience flight path data into the discriminator; adjust the weights and channel numbers of each layer of the generator according to the feedback from the discriminator, and optimize the total loss L through back propagation until the generated flight path probability heat map meets the preset quality requirements. The generator adjusts the weights and channel numbers of each layer of the U-Net generator according to the feedback from the discriminator, optimizes the loss function through back propagation, and finally generates a flight path that meets the requirements of covering all target areas, having a minimum distance to obstacles >3 m, and having a heading angle change rate <5° / s.

[0105] The loss function of the generative adversarial network is:

[0106]

[0107] wherein, L is the total loss function of the generative adversarial network; L is the adversarial loss; L is the intersection-over-union loss of the flight path area; L is the total variation regularization term; , and are weight coefficients, and the recommended values are 0.5, 0.3, and 0.2, respectively.

[0108]

[0109] wherein, is a total variation regularization term; is the pixel value of the th row and th column in the route probability heat map output by the generator, representing the probability (range 0-1) that the location is planned as a route; represents the row and column index of the heat map pixel.

[0110] Step 1.4, heat map post-processing and route extraction

[0111] Input the rough digital surface model of the monitoring area into the trained generator, and perform non-maximum suppression on the probability heat map output by the generator to remove local suboptimal response points;

[0112] Set a probability threshold τ (such as 0.7), and retain all pixel points with probability ≥ τ;

[0113] Perform connected component analysis on the binary heat map, select the connected path covering the farthest / longest monitoring area, and smooth the node spacing to 5 m;

[0114] Convert the selected main path into a sequence of latitude and longitude waypoints as the global optimization route for the current UAV photogrammetry flight.

[0115] Specifically, in step S2, the intelligent obstacle avoidance flight strategy is executed based on the combination of image control point layout and deep reinforcement learning combined with the Transformer architecture, which specifically includes the following steps:

[0116] Step 2.1, measuring three-dimensional coordinates based on uniformly laid image control points

[0117] Uniformly lay image control points in equidistant grids (e.g., every 20 m x 20 m) throughout the monitoring area; for special terrains such as canal slopes and cliffs, design the image control points as "black and white fan-shaped" alternately arranged markers, which can quickly locate during field manual measurement and facilitate accurate identification in subsequent images.

[0118] Measure the XYZ coordinates of each image control point using a total station or RTK;

[0119] Store the measured three-dimensional coordinates of the image control points in the ground control point database as auxiliary field data for subsequent aerial triangulation reconstruction and registration.

[0120] Step 2.2, pose fine-tuning based on deep reinforcement learning (DRL)

[0121] The pose (latitude, longitude, altitude, roll, pitch, yaw) and control actions (throttle, yaw, etc.) of the UAV at the current and several historical moments during flight are stacked together with the Euclidean distance between the UAV and the nearest obstacle and the texture variance of the corresponding projection point in the terrain model (DSM) (quantifying the terrain undulation and the importance of the monitored target), forming a total of 32-dimensional state vector s. t .

[0122] Construct a policy network π containing several fully connected layers. θ (s t ), accept s t Then, a set of three-dimensional continuous control commands are output, which are mapped to fine-tuning amounts of roll angle, pitch angle and throttle thrust, respectively.

[0123] Synchronize s t Input value network V w (s t Estimate the value of the current state for subsequent advantage function calculation.

[0124] Set up rewards:

[0125]

[0126] In the formula, For deep reinforcement learning, the reward function is... The coverage rate of areas where the image overlap rate meets the standard; Distance to the nearest obstacle; This represents the remaining battery percentage. , and These are weighting coefficients, and recommended values ​​are 0.5, 0.3, and 0.2.

[0127] In simulation or small-scale flight test environments, the Proximal Policy Optimization (PPO) algorithm is used to iteratively update network parameters, enabling the policy to improve image coverage, increase obstacle avoidance distance, and optimize endurance efficiency. The trained policy network π... θ (s t Accept s t Then, a set of three-dimensional continuous control commands are output to control the flight status of the UAV in real time.

[0128] Step 2.3, Real-time obstacle avoidance based on spatiotemporal attention using Transformer

[0129] Five consecutive frames of RGB or depth aerial images in front of the drone are used to extract spatial feature vector sequences through a shared convolutional network. The sequence features are then mapped to query Q, K key, and V value vectors. Spatiotemporal information is aggregated through a multi-head attention mechanism to obtain the pixel-level threat weight distribution of obstacles and identify high-risk targets such as suspended cables and moving objects.

[0130] The spatiotemporal attention weights are calculated as follows:

[0131]

[0132] In the formula, This is the query vector, used to focus on the current location that needs to be processed; This is a key vector, representing the identifier of each position in the input sequence; It is a value vector that contains the actual information of the input sequence; The dimension of the key vector; This is the normalization function.

[0133] The pixel-level threat is mapped to the two-dimensional or three-dimensional location of the drone's next five waypoints, and the risk weight of each waypoint is calculated.

[0134] The system sets a threshold r for the risk weight. th When the predicted risk weight r of the i-th waypoint i >r th If the waypoint is deemed to have an unacceptable risk of collision or interference, the obstacle avoidance branch will be immediately initiated.

[0135] Obstacle avoidance strategy based on r i The size can be further subdivided:

[0136] If r i It's only slightly over the limit (e.g., r) th <r i <1.5 r th If so, the flight path will tend to be slightly detoured or the course will be slightly adjusted.

[0137] If r i Exceeding higher security thresholds (e.g., r) i >2 r th If it does, it will execute a more aggressive climb or hovering standby.

[0138] The obstacle avoidance commands (fly around, climb, hover) generated after the risk weight is triggered will be superimposed on the output control commands in step 2.2, and the two together determine the final attitude and heading at the next moment.

[0139] Specifically, in step S3, the N groups of image control point three-dimensional coordinates measured in step 2.1 are taken as the field control information, and the multi-view images collected by the unmanned aerial vehicle along the globally optimized route are subjected to aerial triangulation and dense matching to obtain a high-precision dense point cloud of the study area; then, the dense point cloud is adaptively down-sampled according to local geometric changes to balance the processing efficiency and the detail preservation of key areas, as follows:

[0140] Step 3.1 Aerial triangulation + dense matching

[0141] The N groups of image control point positions (XYZ) are imported into SfM (Structure-from-Motion) software as ground control points (GCPs) for spatial orientation and scale constraint of the multi-view image sequence taken during flight; the GCPs can significantly improve the absolute positioning accuracy of the reconstruction results and eliminate the drift of pure visual matching in absolute coordinates, scale and direction; and ensure that the generated point cloud has a true scale and orientation in the geographic coordinate system, providing a unified reference for subsequent multi-temporal comparison.

[0142] Based on the camera parameters corrected by the GCPs, a multi-view matching algorithm (such as PMVS / CMVS, OpenMVS, COLMAP, etc.) is used to generate a dense point cloud of the entire area.

[0143] Step 3.2 Local elevation variance calculation and zoning

[0144] To reduce data redundancy and improve point cloud data processing speed, the elevation variance of all points within a 0.5 m radius spherical neighborhood of each point cloud is calculated:

[0145]

[0146] In the formula, is the elevation variance within the neighborhood of the point cloud; is the total number of point clouds within the neighborhood; is the elevation value of the th point; is the average elevation of the point clouds within the neighborhood.

[0147] Regions with an elevation variance ≤0.05 m² within the neighborhood are classified as flat regions; regions with 0.05 m²

[0148] Step 3.3 Adaptive down-sampling

[0149] According to the hierarchical down-sampling method, the point cloud of the flat region is retained at 50%, the point cloud of the gently varying region is retained at 70%, and the point cloud of the abrupt change region is retained at 90%, while ensuring that the point cloud density of the detail region is >500 points / m².

[0150] Randomly or grid sampling within each partition, respectively, screening according to the above proportion;

[0151] If the density of a certain area is insufficient after sampling, the most dense point cloud in the neighborhood can be supplemented until the lower limit of 500 points / m2 is met; the optimized point cloud has a large reduction in the number of points and the accuracy of key areas is preserved; it provides a lightweight input for subsequent point cloud registration, speeds up the registration speed and improves the accuracy.

[0152] Specifically, in step S4, the adaptive down-sampling point cloud obtained in step S3 is introduced into the improved Iterative Closest Point (ICP) algorithm, which is divided into two stages of coarse registration (step 4.1) and fine registration (step 4.2), and a dynamic optimization strategy is added in the fine registration stage to speed up the convergence while maintaining high accuracy.

[0153] Step 4.1 Coarse Registration - Key Feature Matching and Initial Transformation Estimation

[0154] The current time point cloud and the reference time point cloud (i.e. the reference time point cloud) are respectively extracted by the ISS (Intrinsic Shape Signatures) algorithm, and the curvature threshold is 0.1 m -1 , and the maximum suppression radius is 0.3 m;

[0155] The FPFH (Fast Point Feature Histogram) descriptor is calculated at each ISS key point, with a dimension of 33 and a search radius of 1 m;

[0156] The FPFH features of the two time point clouds are matched by RANSAC random sampling, and the corresponding point pairs with a feature distance < 0.3 and an inlier ratio ≥ 70% are selected, and an initial rigid transformation matrix T0 is estimated based on these corresponding pairs to import the current point cloud into the reference coordinate system, obtaining the pre-transformed point cloud.

[0157] Step 4.2 Fine Registration - Dynamic Neighborhood and Feature Weighted Optimization

[0158] On the basis of the pre-transformed point cloud, the following iterative process is performed until convergence:

[0159] Dynamic neighborhood search radius, the neighborhood search radius is gradually reduced from the initial 1 m to 0.1 m at the kth iteration;

[0160] Only point pairs with a normal vector angle < 15° are retained for matching to exclude inconsistent surfaces; a curvature weighting strategy is used, and the weight is set to , where is the point cloud curvature value, and the low curvature area is set to 0.8.

[0161] Huber loss function is adopted in the calculation of registration residual, which suppresses the influence of outliers, suppresses noise interference and improves registration accuracy.

[0162] At the same time, the adaptive convergence criterion is designed, and the iteration number and error threshold are dynamically adjusted according to the point cloud size.

[0163] The dynamic error convergence threshold adjustment formula is:

[0164]

[0165] In the formula, is the convergence threshold of the iteration; is the minimum convergence threshold (recommended value 0.01 m); is the attenuation coefficient (recommended value 0.8); is the threshold value of the previous iteration.

[0166] According to the total number of input point clouds, the maximum iteration number is dynamically calculated:

[0167]

[0168] In the formula, is the dynamic maximum iteration number of the improved ICP algorithm; is the basic iteration number (recommended value 50); is the size influence coefficient (recommended value 10); is the total number of input point clouds.

[0169] Each iteration updates the transformation matrix according to the search radius, normal / curvature weight and Huber loss, until the transformation increment is lower than ε k or the maximum iteration number is reached, then the fine registration is completed.

[0170] Output the final rigid transformation matrix to accurately align the current time point cloud with the reference time point cloud.

[0171] Specifically, in step S5, the input is two time point clouds that have completed fine registration, and the output is a high-density continuous displacement field.

[0172] Taking the previous point cloud as the reference point cloud, the nearest neighbor point of the next point cloud in the reference point cloud is matched point by point, and the effective point pair is selected based on the normal-curvature joint constraint (normal vector angle < 15° and curvature difference < 0.05 / m), and the displacement of all reserved point pairs is calculated.

[0173] Combined with the Laplace criterion, outliers with displacement greater than 3 times the standard deviation are removed, and three-dimensional displacement vectors are calculated point by point.

[0174] According to the reference point cloud or the DSM slope information, the study area is divided into:

[0175] Flat area (slope < a certain threshold value), the node spacing is set to 0.2 m;

[0176] Steep slope / crack area (slope ≥ threshold value), the node spacing is set to 0.05 m.

[0177] Generate an interpolation node grid in the whole area according to the above spacing;

[0178] Select several nearest discrete displacement vectors around each interpolation node, and fit the continuous displacement at this place in a weighted least squares manner. The weight can be Gaussian kernel or inverse distance weight, which ensures that the sampling density matches the node spacing.

[0179] Construct an irregular triangle network (TIN) with all interpolation nodes and corresponding displacement values, and interpolate based on triangular elements on the TIN to form a high-density continuous displacement field. Displacement vectors or displacement amplitude distribution maps at any position can be output as needed.

[0180] Specifically, the high-density continuous displacement field generated in the above step S5 is input in step S6, and the deformation risk of the channel slope is quantitatively evaluated by a weighted scoring method of displacement amount and displacement gradient, and the monitoring area is divided into four sub-areas. The specific steps are as follows:

[0181] For each interpolation node on the continuous displacement field, calculate the amplitude of its displacement vector as the displacement amount;

[0182] Perform local difference on the continuous displacement field or calculate the gradient size on the triangular network element as the displacement gradient.

[0183] The comprehensive risk value is calculated by the following formula:

[0184]

[0185] In the formula, is the comprehensive risk value; is the displacement amount, with the unit of mm; is the displacement gradient, with the unit of mm / m; and are weight coefficients, and the recommended values are 0.6 and 0.4 respectively.

[0186] The area of is defined as a high-risk area; The area of is defined as a medium-risk area; The area of is defined as a low-risk area;

[0187] is defined as a stable area.

[0187] The risk level of each interpolation node is mapped back to the triangular mesh unit of the continuous displacement field, a four-color risk zoning map is generated, and statistical information such as the area and average risk value of each subzone is output, facilitating subsequent early warning and operation decision-making.

[0188] Embodiment 2

[0189] The application provides a channel and slope deformation monitoring system based on close photogrammetry of a UAV, comprising:

[0190] The global optimization flight path module is used for automatically generating a global optimization flight path covering the region and maintaining a preset safety distance at any flight segment according to a sketch model of the region to be measured and historical artificial planning data;

[0191] The UAV adjustment module is used for controlling the UAV to fly close to the global optimization flight path and collect multi-view images; during the flight process, the position of the next flight point and the attitude of the UAV are adjusted online according to real-time state data of the UAV and continuous aerial image shooting images, so that the image coverage rate is ensured and obstacles are avoided in real time;

[0192] The down-sampling module is used for inputting the multi-view images and the image control point three-dimensional coordinates arranged in the region to be measured into an aerial triangulation system to reconstruct a dense point cloud at the current time; and the sampling density is adaptively adjusted according to the geometric change of the neighborhood of each point of the point cloud, so that the high point cloud density is maintained in the terrain mutation area;

[0193] The point cloud registration module is used for inputting the current time point cloud after adaptive down-sampling and the reference time point cloud into an iterative closest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to complete accurate alignment of the two time phase point clouds;

[0194] The displacement field generation module is used for calculating three-dimensional displacement vectors of corresponding points according to the aligned two time phase point clouds, and generating a continuous displacement field through multi-scale interpolation;

[0195] The risk assessment module is used for combining the displacement amount and the spatial gradient of the displacement field to perform risk assessment on the channel and slope deformation, and outputting the deformation monitoring result and early warning information.

[0196] The system shortens the time consumption of the flight path generation framework based on GAN from 2.3 h / km 2 To 0.5 h / km 2 , the coverage rate of steep slope areas (slope > 30°) is increased from 75% to 98%, and the image overlap rate is stably reached ≥85% horizontally and ≥80% vertically;

[0197] The PPO policy network adjusts the unmanned aerial vehicle attitude (roll angle and pitch angle control accuracy is ±0.5°) in real time, predicts the future 5 waypoints (confidence is greater than 90%) in combination with the Transformer architecture, and the data collection interruption rate is reduced from 18% to 3%, and the crash risk is reduced to 1.2%;

[0198] The improved ICP algorithm reduces the point cloud registration error from 2.1 cm to 0.8 cm through a curvature weighting strategy and a dynamic search radius, and in combination with multi-scale displacement field encryption (0.05 m in steep slope area and 0.2 m in flat area), the signal-to-noise ratio (SNR) of the deformation signal is improved from 3 dB to 12 dB, and the monitoring accuracy is 0.5 cm;

[0199] Based on the weighted scoring model of displacement and gradient, the high-risk area identification accuracy is improved to 92%, and the grading early warning data (high risk, medium risk, low risk and stable area) is provided for water conservancy engineering operation and maintenance.

[0200] Embodiment 3

[0201] The application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the channel and slope deformation monitoring method based on close photogrammetry of an unmanned aerial vehicle.

[0202] Embodiment 4

[0203] The application also provides an electronic device, which comprises a memory and a processor, the memory and the processor are in communication connection with each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the channel and slope deformation monitoring method based on close photogrammetry of an unmanned aerial vehicle.

[0204] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system or a computer program product. Therefore, the application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0205] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block or blocks in the block diagrams. Figure 1 one or more functions specified in the flowchart or flows and / or block or blocks of the block diagrams. Figure 1 one or more functions specified in the flowchart or flows and / or block or blocks of the block diagrams.

[0206] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more functions specified in the flowchart or flows and / or block or blocks of the block diagrams. Figure 1 one or more functions specified in the flowchart or flows and / or block or blocks of the block diagrams.

[0207] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block or blocks in the block diagrams. Figure 1 one or more functions specified in the flowchart or flows and / or block or blocks of the block diagrams. Figure 1 one or more functions specified in the flowchart or flows and / or block or blocks of the block diagrams.

[0208] Those portions of the description which follow can contain material that is intended to be interpreted by those skilled in the art.

Claims

1. A method for monitoring the deformation of a canal slope based on close-range photogrammetry of a UAV, characterized in that: The method comprises the following steps: According to the rough model of the region to be measured and the historical manual planning data, a globally optimized flight path covering the region and maintaining a preset safety distance at any flight segment is automatically generated; The UAV is controlled to fly close to the globally optimized flight path and collect multi-view images; during the flight, the position of the next flight point and the attitude of the UAV are adjusted online according to the real-time state data of the UAV and the continuous aerial images, so as to ensure image coverage and avoid obstacles in real time; The multi-view images and the three-dimensional coordinates of the image control points arranged in the region to be measured are input into an aerial triangulation system to reconstruct a dense point cloud at the current time; and the sampling density is adaptively adjusted according to the geometric changes of the neighborhood of each point in the point cloud, so as to maintain a high point cloud density in the area with sudden changes in topography; The current time point cloud after adaptive downsampling and the reference time point cloud are input into an iterative closest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to complete the accurate alignment of the two time phase point clouds; The three-dimensional displacement vectors of each corresponding point are calculated according to the aligned two time phase point clouds, and a continuous displacement field is generated through multi-scale interpolation; The displacement and its spatial gradient of the displacement field are combined to evaluate the risk of the deformation of the canal slope.

2. The method of claim 1, wherein: The process of obtaining the globally optimized flight path comprises: An offline training stage: The rough model of the region to be measured and the historical manual planning flight path data are simultaneously input into a generative adversarial network flight path planning model, the generative adversarial network comprising: a generator for generating a candidate flight path probability map according to the rough model; a discriminator for judging the quality of the candidate flight path probability map based on the historical manual flight path data and feeding back the judgment result to the generator; Through multiple rounds of adversarial training of the generator and the discriminator, stable convergence is reached between the two, so as to determine the final network parameters of the generator; An online generation stage: The rough model is input into the trained generator to generate a flight path probability map covering the region; The flight path probability map is first subjected to non-maximum suppression, and then connectivity extraction is performed based on a probability threshold to obtain a continuous globally optimized flight path; The globally optimized flight path is output as the initial flight path for subsequent UAV close photogrammetry flight.

3. The method of claim 2, wherein: The discriminator judges the quality of the flight path probability map output by the generator in terms of the degree of image overlap meeting the requirements and terrain adaptability based on the historical manual planning flight path data in the offline training stage, and feeds back the judgment result to the generator.

4. The method of claim 2, wherein: In the adversarial training stage, the loss function of the generator comprises: an adversarial loss term for constraining the game balance between the generator and the discriminator; a flight path area intersection-over-union loss term for constraining the area coincidence degree between the generated flight path and the historical manual flight path; a total variation regularization term for constraining the spatial smoothness of the flight path probability map; each loss term is weighted and summed according to a predetermined weight coefficient.

5. The method of claim 1, wherein: During the flight, a deep reinforcement learning-based attitude control strategy network is constructed; The input state vector of the strategy network comprises: the current and historical attitude and action stacking information of the UAV; the relative distance to the environmental obstacles; and the terrain texture variance features extracted from the rough model; The policy network outputs three-dimensional continuous control instructions in real time, which are used for online fine-tuning of the roll angle, pitch angle and throttle thrust of the unmanned aerial vehicle; The reward function of the policy network comprehensively evaluates three indexes, namely, image coverage quality, obstacle safety distance and remaining power, and sets the increase of image coverage, obstacle distance and endurance efficiency as positive rewards. By using the reward function and the proximal policy optimization algorithm, the parameters of the policy network are iteratively updated.

6. The method of claim 1, wherein: During flight, multiple frames of aerial images continuously collected in front of the unmanned aerial vehicle are subjected to convolution feature extraction, and threat weights of each obstacle pixel are calculated through spatio-temporal self-attention; Based on the threat weights, risk weights at future waypoints are calculated; When the risk weight of any predicted waypoint exceeds a preset threshold, an obstacle avoidance correction instruction is generated in real time to adjust the flight path of the unmanned aerial vehicle or to realize online avoidance of dynamic obstacles by climbing or hovering.

7. The method of claim 1, wherein: The execution process of the iterative closest point registration algorithm comprises: Based on the ISS algorithm, key points with a curvature greater than a preset threshold are extracted from the point cloud at the current time, and an FPFH feature descriptor is calculated for each key point; The FPFH descriptors are matched by using the RANSAC algorithm to screen matching pairs with a distance less than a preset threshold and a RANSAC inlier proportion not lower than a preset threshold, and an initial rigid transformation matrix is estimated based on the matching pairs; The initial rigid transformation matrix is applied to the point cloud at the current time as an initial value for iterative fine registration to obtain a pre-transformed point cloud; Iterative closest point fine registration is implemented on the pre-transformed point cloud: In each iteration, the neighborhood search radius is gradually shrunk from an initial value to a minimum threshold; Only point pairs with a normal vector included angle within a preset threshold are retained for matching; Point pairs with a curvature greater than a preset value are given a higher matching weight, and point pairs with a low curvature are given a lower weight, and a Huber robust loss function is used to process residuals; The fine registration process is repeatedly performed until the transformation increment is lower than a convergence threshold, and the accurate alignment of the point cloud at the current time and the point cloud at the reference time is completed.

8. The method of claim 1, wherein: The generation process of the continuous displacement field comprises: With reference to the point cloud at the reference time, the closest neighbor point of the point cloud at the current time is searched point by point, and a normal vector-curvature joint constraint is applied, and only point pairs with a normal included angle less than a preset threshold and a curvature difference less than a preset threshold are retained; Three-dimensional displacement vectors are calculated for the retained point pairs, and outliers with a displacement greater than three times the standard deviation are removed according to the Laplace criterion to obtain a discrete displacement vector field; The discrete displacement vector field is interpolated by using a moving least squares method: a first preset node spacing is used in flat areas, and a second preset node spacing smaller than the first spacing is used in steep slope or crack dense areas; Based on the encrypted displacement point cloud, an irregular triangle mesh is constructed to generate a continuous displacement field.

9. The method of claim 1, wherein: The risk assessment process comprises: The displacement value and the displacement gradient value of each position in the displacement field are normalized according to preset weight coefficients, respectively; The normalized displacement value and the normalized displacement gradient value are combined in a linear weighting manner to obtain a comprehensive risk score of the position. According to the size of the comprehensive risk score, the region to be measured is divided into four categories: high-risk area, medium-risk area, low-risk area and stable area.

10. A canal slope deformation monitoring system based on close-range photogrammetry of unmanned aerial vehicles, characterized by: Comprise: Global optimization route module, for automatically generating a global optimization route covering the region and maintaining a preset safety distance at any flight segment according to the sketch model of the region to be measured and historical artificial planning data; Unmanned aerial vehicle adjustment module, for controlling the unmanned aerial vehicle to fly close to the global optimization route and collect multi-view images; during the flight, according to the real-time state data of the unmanned aerial vehicle and the continuous aerial image, the position of the next flight point and the attitude of the unmanned aerial vehicle are adjusted online to ensure the image coverage and avoid obstacles in real time; Down-sampling module, for inputting the multi-view images and the three-dimensional coordinates of the image control points arranged in the region to be measured into an aerial triangulation system to reconstruct a dense point cloud at the current time; and according to the geometric changes of the neighborhood of each point in the point cloud, the sampling density is adaptively adjusted to maintain high point cloud density in the terrain mutation area; Point cloud registration module, for inputting the current time point cloud after adaptive down-sampling and the reference time point cloud into an iterative closest point registration algorithm with dynamic neighborhood search and feature weighting mechanism to complete the accurate alignment of the two time phase point clouds; Displacement field generation module, for calculating the three-dimensional displacement vector of each corresponding point according to the aligned two time phase point clouds, and generating a continuous displacement field through multi-scale interpolation; Risk assessment module, for combining the displacement amount and its spatial gradient of the displacement field to assess the risk of the deformation of the canal slope, and outputting the deformation monitoring result and early warning information.

Images