A dam surface disease identification method and system based on drone

Abstract

The present invention discloses a method for identifying dam surface defects based on unmanned aerial vehicles, which is specifically as follows: the unmanned aerial vehicle is equipped with a millimeter-wave radar to complete the track planning, and then the high-definition camera carried is used to complete the identification and classification of the dam surface defect phenomena; the unmanned aerial vehicle track planning process is that the unmanned aerial vehicle is equipped with a millimeter-wave radar to complete the establishment of the dam environment model, and the BCO algorithm is used to complete the planning of the unmanned aerial vehicle track. During the unmanned aerial vehicle track process, a multi-factor constraint function is used to constrain the unmanned aerial vehicle track to obtain the shortest and safest unmanned aerial vehicle flight track; during the unmanned aerial vehicle track process, a camera is carried to collect dam defect information, and a novel CCNG algorithm proposed by the present invention is used to complete the identification and classification of the defect image information such as penetration, cracks, settlement, deformation, etc. on the dam surface. The present invention can complete the identification and detection of dam surface defect information more quickly and accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection, and in particular to a method and system for identifying dam surface defects based on an unmanned aerial vehicle. Background Art

[0002] Dams are important water conservancy facilities for maintaining high-quality national and local water resources, resisting floods, and promoting regional development. They are also important infrastructure for protecting the lives and property of the people. The emergence of dams has not only changed the situation of water conservancy projects, but also brought a series of challenges and problems. Due to long-term use, natural disasters, illegal occupation and other reasons, various diseases may appear on the surface of the dam, such as seepage, cracks, settlement, deformation and other diseases. If these diseases are not repaired, maintained and managed in time, they will seriously affect the safety and stability of the dam and may even cause disasters.

[0003] At present, the identification and detection of dam surface defects mainly rely on manual inspection and visual inspection. However, manual inspection faces limitations in terms of manpower, material resources, and time, while the accuracy and reliability of visual inspection also have certain limitations, which hinder the discovery and treatment of dam surface defects. Summary of the invention

[0004] Purpose of the invention: The present invention proposes a method and system for identifying dam surface defects based on drones, aiming to improve the recognition accuracy of dam defect phenomena.

[0005] Technical solution: A method for identifying dam surface defects based on drones, including the following steps:

[0006] Step 1: The millimeter-wave radar carried by the drone collects point cloud data of the dam environment, builds an environmental model, and plans the trajectory of the drone;

[0007] Step 2: In the process of UAV trajectory planning, a multi-factor constraint function is used to constrain the UAV trajectory to obtain the optimal path;

[0008] Step 3: During the flight of the drone, the camera carried by the drone collects image information of dam surface defects, and uses the novel CCNG algorithm to identify and classify the dam surface defect information.

[0009] Furthermore, the millimeter wave radar in step 1 completes the collection of point cloud data of the dam environment, and the specific method is:

[0010] The point cloud data set collected by the millimeter wave radar is S = {s i}, where i∈N, s i =[x i ,y i ,z i ]T , i represents the i-th point cloud data, x i ,y i 、z i Represents the height, width, and depth of the i-th point cloud; divides the three-dimensional point set area into pixels with an imaging side length of d s A finite voxel set V with side length i , to ensure the integrity of the surface image captured by the viewpoint, the center point of the voxel is regarded as the voxel coordinate, and the calculation formula for selecting the viewpoint is:

[0011] p o =v i , p o ∈P o

[0012] Among them, p o is the point that the UAV track needs to pass through; P o This is the point cloud set of the entire dam surface collected by the UAV millimeter wave radar; i Represents a three-dimensional pixel point.

[0013] Furthermore, in step 1, the trajectory of the UAV is planned based on the collected point cloud data, and the BCO algorithm is used to complete the optimal trajectory planning of the UAV. When the UAV moves through all viewpoint coordinate positions, the dam inspection task is completed.

[0014] Furthermore, the multi-factor constraint function in step 2 is used to constrain the UAV track, combined with the shortest track constraint function F of the UAV flight track. 1 , minimum threat constraint function F 2 , flight altitude constraint function F 3 and the flight angle constraint function F 4 , then the multi-factor constraint function is:

[0015]

[0016] Among them, b l is the weight coefficient of each factor, X i for the entire flight path.

[0017] Furthermore, the shortest track constraint function F 1 Specifically:

[0018] Define the flight track point of the drone as W ij =(x ij ,y ij ,z ij ), represents the three-dimensional spatial position of the UAV at the jth path point in the i-th flight path, then the entire flight path X iis a three-dimensional array containing n path points; the Euclidean distance between two track points is recorded as the track segment || W ij W i,j+1 ||, then the shortest track constraint function F 1 The calculation formula is:

[0019]

[0020] Furthermore, the minimum threat constraint function F 2 Specifically:

[0021] Setting the obstacle threat area, the obstacle avoidance threat cost of the drone and its track segment || W ij W i,j+1 || and obstacle threat area center C k The distance d k Inversely proportional to, the minimum threat constraint function F 2 The calculation formula is:

[0022]

[0023] Among them, K is the obstacle threat area set in the flight environment; and W is the current track point of the UAV. ij The relevant obstacle constraint function T ij The calculation formula is:

[0024]

[0025] Among them, the radius of the obstacle threat area is R k , the outer periphery of the obstacle threat area is a circular collision threat warning area D; γ c is the obstacle threat cost penalty coefficient.

[0026] Furthermore, the flight altitude constraint function F 3 Specifically:

[0027] The flight altitude of the drone during the process of collecting dam disease information is subject to the minimum altitude h min and the maximum height h max Constraints; Move the UAV to the track point W ij The height from the reference terrain is denoted as h ij , then the flight altitude constraint function F 3 The calculation formula is:

[0028]

[0029] Among them, the current track point W of the UAV ij The related height constraint function H ij The calculation formula is:

[0030]

[0031] Among them, γ h It is the penalty coefficient for the UAV flight altitude exceeding the constraint limit.

[0032] Furthermore, the flight angle constraint function F 4 Specifically:

[0033] The flight angle control parameters of the UAV include the horizontal turning angle and the vertical pitch angle. The flight angle constraint function F 4 The calculation formula is:

[0034]

[0035] Among them, a 1 and a 2 are the penalty coefficients for the horizontal turning angle and vertical pitch angle of the UAV exceeding the constraint conditions; α ij and β i,j are the horizontal turning angle and vertical pitch angle of the jth path point in the UAV’s trajectory, β i,j-1 represents the vertical pitch angle of the j-1th path, β i,j+1 It represents the vertical pitch angle of the j+1th path point. The relevant calculation formula is as follows:

[0036]

[0037]

[0038] Among them, ||W ij W i,j+1 || and ||W i,j+1 W i,j+2 || represents two consecutive track segments in the UAV flight path, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 ' is the projection of two consecutive track points on the three-dimensional plane, z i,j+1 Indicates the vertical depth of the j+1th path point, z i,j represents the vertical depth of the jth path point, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 The calculation formula of ' is:

[0039] W ij 'W i,j+1 '=k×(W ij W i,j+1 ×k)

[0040] W ij+1 'W i,j+2 '=k×(W ij+1 W i,j+2 ×k)

[0041] Where k is the unit vector in the positive direction of the coordinate axis;

[0042] Furthermore, the novel CCNG algorithm described in step 3 includes CBDNet network to perform denoising on the collected dam surface disease images, CutMix data enhancement to obtain enhanced image data by mixing and superimposing the denoised image data, NeRF multi-view segmentation model to perform feature extraction and segmentation on the enhanced image data, and GoogLeNet network to classify and identify the characteristics of dam surface diseases.

[0043] Furthermore, the surface defects of the dam include seepage, cracks, settlement and deformation.

[0044] A dam surface disease identification system based on UAV, including a UAV trajectory planning system and a dam surface disease identification system;

[0045] The UAV trajectory planning system uses a multi-factor constraint function to constrain the UAV trajectory during the UAV trajectory planning process to obtain the optimal path;

[0046] During the flight of the UAV, the camera carried by the UAV collects image information of dam surface diseases and uses a novel CCNG algorithm to identify and classify the dam surface disease information.

[0047] The novel CCNG algorithm includes CBDNet network to perform denoising on the collected dam surface disease images, CutMix data enhancement to obtain enhanced image data by mixing and superimposing the denoised image data, NeRF multi-view segmentation model to perform feature extraction and segmentation on the enhanced image data, and GoogLeNet network to classify and identify the dam surface disease features.

[0048] Beneficial effects: Taking into account the complex real environment, the present invention adopts a more complex multi-factor track evaluation function to plan the UAV track and constrain the UAV track to improve the safety of UAV flight.

[0049] The present invention proposes a novel CCNG algorithm to identify and classify dam surface disease information, thereby improving the recognition accuracy of dam surface disease information. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of a dam surface disease identification system based on UAV;

[0051] Figure 2 This is the flow chart of the novel CCNG algorithm. DETAILED DESCRIPTION

[0052] The present invention provides a method for identifying dam surface defects based on unmanned aerial vehicles. The solution uses a millimeter-wave radar on the unmanned aerial vehicle to complete track planning, and then uses a high-definition camera on board to complete the identification and classification of dam surface defects. The specific implementation steps are as follows:

[0053] Step 1: The UAV is equipped with millimeter wave radar to complete the point cloud data collection of the dam environment and complete the UAV flight environment modeling. Point cloud data is a set of three-dimensional points used to describe the nonlinear surface shape of objects in three-dimensional space. Each point in the point set is represented by orthogonal length, width and height three-dimensional coordinates in space. The point cloud data set collected by the millimeter wave radar is S = {s i}, where i∈N, s i =[x i ,y i ,z i ] T ∈R 3 .

[0054] The three-dimensional point set area is divided into two parts with imaging side length d s A finite voxel set V with side length i , to ensure the integrity of the surface image captured by the viewpoint, the center point of the voxel is regarded as the voxel coordinate, and the calculation formula for selecting the viewpoint is:

[0055]

[0056] Among them, p o is the point that the UAV track needs to pass through; P o A point cloud set of the entire dam surface collected by the UAV millimeter wave radar;

[0057] According to the collected point cloud data, the BCO algorithm is used to plan the optimal trajectory of the UAV. When the UAV moves through all viewpoint coordinate positions, the dam inspection mission is completed.

[0058] Step 2: Considering the complex real environment, the present invention adopts a track evaluation function to perform UAV track planning.

[0059] The primary goal of UAV trajectory planning is to find the flight trajectory with the shortest distance between the take-off point and the target point. The flight trajectory point of the UAV is defined as W. ij =(x ij ,y ij ,z ij ), represents the three-dimensional spatial position of the UAV at the jth path point in the i-th flight path, then the entire flight path Xi It can be represented as a three-dimensional array containing n path points. The Euclidean distance between two track points is recorded as the track segment || W ij W i,j+1 ||, then the constraint function F related to the UAV flight track 1 The calculation formula is:

[0060]

[0061] When collecting dam disease information, the drone needs to avoid obstacles to ensure a safe operation track. Set the obstacle threat area, the obstacle avoidance threat cost of the drone and its track segment || W ij W i,j+1 || and obstacle center C k The distance d k Inversely proportional to the obstacle constraint function F related to the obstacle avoidance threat of the UAV 2 The calculation formula is:

[0062]

[0063] Among them, the current track point W of the UAV ij The relevant obstacle constraint function T ij The calculation formula is:

[0064]

[0065] Among them, the center coordinate of the obstacle threat area is C k , the coverage area radius is R k , the periphery of the coverage area is the collision threat area D; K is the set of obstacle threat areas in the flight environment; γ c is the obstacle threat cost penalty coefficient;

[0066] The flight altitude of UAVs in the process of collecting dam disease information is usually subject to the minimum altitude h min and the maximum height h max The constraints of the drone are set at the track point W. ij The height from the reference terrain is denoted as h ij , that is, Z ij and T ij The difference between the height constraint function F of the drone obstacle avoidance threat is 3 The calculation formula is:

[0067]

[0068] Among them, the current track point W of the UAV ij The related height constraint function H ij The calculation formula is:

[0069]

[0070] Among them, Z ij is the height of the drone relative to the sea level; γ h The penalty coefficient for the drone’s flight altitude exceeding the constraint limit;

[0071] The flight angle control parameters of the UAV mainly include the horizontal turning angle and the vertical pitch angle. These two parameter variables must meet the actual turning angle constraint of the UAV, otherwise the trajectory planning model cannot generate a feasible flight path. The calculation formula of the constraint function related to the flight angle of the UAV during the trajectory process is:

[0072]

[0073] Among them, a 1 and a 2 are the penalty coefficients for the horizontal turning angle and vertical pitch angle of the UAV exceeding the constraint conditions; α ij and β i,j+1 They are the horizontal turning angle and vertical pitch angle of the UAV during its flight path, and the relevant calculation formulas are as follows:

[0074]

[0075]

[0076] Among them, ||W ij W i,j+1 || and ||W i,j+1 W i,j+2 || represents two consecutive track segments in the UAV flight path, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 ' is the projection of two consecutive track points on the three-dimensional plane, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 The calculation formula of ' is:

[0077] W ij 'W i,j+1 '=k×(W ij W i,j+1 ×k)

[0078] W ij+1 'W i,j+2 '=k×(W ij+1 W i,j+2 ×k)

[0079] Where k is the unit vector in the positive direction of the coordinate axis;

[0080] Combining the shortest flight path, minimum threat, flight altitude, flight angle and other constraints related to the above UAV flight path, the multi-factor constraint function is:

[0081]

[0082] Among them, b l is the weight coefficient of each factor.

[0083] Step 3: The present invention proposes to use a novel CCNG algorithm to identify and classify surface disease phenomena of the dam. The drone is equipped with a high-definition camera to collect images of the entire surface of the dam, and a dam disease image dataset is established based on the collected images. The surface disease phenomena of the dam include seepage, cracks, settlement, and deformation. The present invention proposes to use a joint CCNG algorithm to identify and classify surface disease phenomena of the dam. The recognition process of the joint CCNG algorithm is to use the CBDNet network to remove noise interference in the collected dam disease images, and the CutMix data enhancement process to improve image clarity and eliminate the intact information of the dam surface. Then, a multi-view segmentation model based on NeRF is used to extract features from the image and segment it. Finally, a mathematical model of the GoogLeNet network is constructed to classify and identify dam cracks.

[0084] Since there are differences in image quality in the data set, the collected image data set needs to be processed. The purpose of preprocessing is to optimize data information, enhance the valuable features of the image, and improve the accuracy of recognition.

[0085] In the process of processing the dam disease images, the CBDNet network is first used to remove noise. The convolutional blind denoising network (CBDNet) is a real denoising network that can perform blind denoising, that is, the noise level of the image to be denoised does not affect the network's denoising of the image. The CBDNet network consists of two parts: a noise estimation network and a non-blind denoising network. To improve the denoising effect, the CBDNet network uses synthetic noise images and real noise images as inputs to the model. The CBDNet network also uses an asymmetric learning loss function to interactively adjust the denoising results during the network training process to effectively enhance the denoising effect of the CBDNet network.

[0086] Estimated noise level for a given pixel i in the CBDNet network and the true value σ(y i ), when the estimated noise level is smaller than the true value, that is The CBDNet network will increase the penalty for noise level, and the loss function is defined as:

[0087]

[0088] By setting an arbitrary value a∈(0,0.5), the network model introduces total variation (TVL asymm ) Regular term constraints The smoothness of , thus we can get:

[0089]

[0090] Among them, ν h and ν v are the vertical and horizontal gradients of the image respectively. After error reconstruction Defined as:

[0091]

[0092] The objective loss function of the CBDNet network is:

[0093] L=L rec +λ asymm L asymm +λ TV L TV

[0094] Among them, λ asymm and λ TV L asymm and L TV The corresponding weight of the loss function.

[0095] Then, a large amount of data training is carried out, and the model learning ability is enhanced by optimizing the image quality and quantity, so as to learn the deep features of the data and ensure the accuracy of model recognition. However, in actual training, when the amount of data is small or the data distribution is uneven, overfitting will occur during model training, so data enhancement is needed to resolve the overfitting problem. The present invention proposes to use CutMix data enhancement processing to improve image clarity, strengthen the feature differences similar to the dam surface and the images of diseases such as seepage, cracks, settlement, deformation, etc., and improve the quality of the dam surface disease information to improve the recognizability of the enhanced image.

[0096] The CutMix data enhancement operation mixes and superimposes two randomly selected images, randomly crops a part of one of the randomly selected images, and then fills it with the pixel information of the same position area of ​​the other image. The calculation formula of CutMix data enhancement is as follows:

[0097]

[0098] Among them, the enhancement weight λ cut is a random number that satisfies the Beta distribution; (x 1 ,y 1 )、(x 2 ,y2 ) represents two randomly selected samples and their labels from the training set; (x, y) represents the generated data result; M represents the cropped area.

[0099] Assuming that the size of the image to be augmented is H×W, the center point (r, c) is randomly generated, and the cropping area (r 1 ,r 2 ) and (c 1 ,c 2 ) is calculated as follows:

[0100]

[0101]

[0102]

[0103] At this time, the image cropping matrix is ​​M[r 1 :r 2 ,c 1 :c 2 ], the adjusted enhancement weight λ' cut The calculation formula is as follows:

[0104]

[0105] CutMix data enhancement uses cropping to fill in images, reducing the unnatural appearance of mixed images, and finally saves the enhanced image.

[0106] For the enhanced dam surface disease image data, it is necessary to segment and extract the initial disease area, and then simplify it by compressing and calculating the image disease area, so that the useful disease features in the image are further displayed, thereby enhancing the recognition of the image and ensuring the effective classification of the dam image disease area. The image quality and the size of the area to be processed depend on the image segmentation in the image processing process. The present invention uses a multi-view segmentation model based on NeRF to perform feature extraction and segmentation on the image.

[0107] Neural Radiance Field (NeRF) encodes an image as a function that maps image coordinates x and view direction d to color c and density σ. The function f is modeled. For a ray r in the function f model, the expected color is estimated by volume rendering using the quadrature method; the ray is divided into and (near boundary t n and far boundary t f ) between N parts, where t i To sample from the i-th part to render the estimated color The color estimation formula is:

[0108]

[0109] in, is the image transmittance, j is any image area in the i-th part; δ i =t i+1 -t i is the distance between two adjacent pixels; t i and σ i t i The color and density of i is the target relevance.

[0110] For rays passing through pixels of the training view, the actual color C is used. GT (r) to process, and use the reconstruction loss to optimize the function f model:

[0111]

[0112] Where R is the number of rays sampled in the image;

[0113] The classification loss in the image segmentation process is determined by the objective probability of the ray Class determination, L clf The calculation formula is as follows:

[0114]

[0115] Among them, the objective probability Ι represents the indicator function; BCE is the binary cross entropy loss; R masked To pass through The set of rays for the masked pixels in , is the initial mask;

[0116] The total loss function calculation formula of the NeRF-based multi-view segmentation model is as follows:

[0117] L mv =L rec +λ clf L clf

[0118] Among them, λ clf is the hyperparameter of the classification weight. clf Threshold the objects and mask out pixels with probability greater than 0.5.

[0119] For the color missing part of the dam shape, the present invention adopts the restoration view method to perceive the RGB loss function in NeRF. Use the patcher INP to patch it, so as to obtain the patched RGB image The calculation formula of INP patcher is: in, is the mask of the i-th image, and finally through the RGB loss function L LPIPS The masked part of the image is optimized and repaired. The unmasked part is optimized using the mean square error function MSE, and this area does not need to be repaired. RGB loss function L LPIPS The calculation formula is as follows:

[0120]

[0121] Where β is the index between image numbers [1, n]; The i-th image rendered by the NeRF algorithm;

[0122] In order to speed up the image processing speed of the multi-view segmentation model based on NeRF, the present invention adopts the improved zebra algorithm (IZOA) to adjust the hyperparameter λ of the classification weight of the NeRF model. clf Conduct optimization.

[0123] Divide 1000 dam crack images into a test set and training set in, Input set for test set; Output set for the test set; N TE is the number of test set images; Input set for training set; Output set for training set; N TR is the number of images in the training set;

[0124] The mathematical model of the GoogLeNet network is constructed to classify and identify dam diseases. The model structure is as follows:

[0125] (1) Input-output relationship:

[0126]

[0127] Among them, X is the abstract feature or hierarchical representation feature of the input image; W is the convolution kernel; b is the bias;

[0128] (2) Classifier parameter settings:

[0129]

[0130] in,

[0131] The final output category is y=argmax{Y(k)}.

[0132] (3) On the training set, the objective function constructed using cross entropy is:

[0133]

[0134] Among them, R(W) and R(θ) are regularization terms, which make the parameters sparse and prevent overfitting.

[0135] (4) The optimization formula for updating parameters is:

[0136]

[0137] Step 4: The image quality evaluation standard is a key factor in determining the image quality. The present invention proposes to use image evaluation indicators such as image peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

[0138] PSNR is used to evaluate the closeness of images. The calculation formula of PSNR is as follows:

[0139]

[0140] Among them, the Max value depends on the maximum value in the image; MSE is the mean square error, which is a measure of the difference between the estimator and the estimated value. The MSE calculation formula is as follows:

[0141]

[0142] Where n is the length of the predicted value; is the image prediction value; y i is the true value.

[0143] PSNR can reflect the image quality more clearly. The PSNR value represents the quality of the image. The larger the value, the better the image quality.

[0144] SSIM is a commonly used image quality evaluation method.

[0145] The SSIM method is often used to evaluate the structural similarity between images. The calculation formula of SSIM is as follows:

[0146]

[0147] Among them, μ x and μ y Represents the average gray value of image x and y; δ x and δ y represents the standard deviation of images x and y; δ x,y Represents the correlation coefficient between images x and y. Constant C 1 and C 2 The calculation formula is:

[0148] C 1 =(K 1 L)2 ,C 2 =(K 2 L) 2

[0149] Among them, K 1 =0.01, K 2 =0.02; L is the gray value range.

[0150] The accuracy is the ratio of the number of samples correctly classified to the total number of samples. It is used to evaluate the monitoring results of network training. The calculation formula is:

[0151]

[0152] Among them, T P is the number of correctly classified samples; F P The wrong label is a positive sample; T N is the number of negative samples correctly classified; F N Wrongly labeled samples are negative samples.

[0153] The present invention takes into account that in the actual environment, the drone needs to constantly adapt to the dam environment. Therefore, in the process of drone track planning, the optimal path will appear to be more complex and contain many different features. Considering the complex real environment, the present invention adopts a more complex track evaluation function to plan the drone track. The drone track is constrained to improve the safety of drone flight.

[0154] A method for identifying surface defects of dams has high reliability, accuracy and efficiency. The method mainly includes: using drones to acquire dam surface image data and establish a high-definition digital image library; using image recognition, computer vision and deep learning algorithms to process and analyze dam surface images, automatically determine the type of disease, severity of disease, location of disease and spread of disease, and make predictions and early warnings of disease conditions; using artificial intelligence technology to propose effective and practical solutions to improve the accuracy and intelligence of dam surface disease detection. This method can not only effectively reduce manpower and material costs, but also prevent and reduce the dangers and hidden dangers caused by human errors. In addition, this method can be applied to dams of various materials and forms, has certain universality and controllability, and provides a strong guarantee for improving the safety management and production system of dams.

[0155] The present invention proposes to use a novel CCNG algorithm to identify and classify dam surface disease information. The CBDNet network removes noise interference in the dam surface disease image, and the CutMix data enhancement process improves image clarity, eliminates the intact information of the dam surface, and strengthens the feature differences between the dam surface and the image information of diseases such as seepage, cracks, settlement, and deformation. By improving the quality of the dam surface disease information, the recognizability of the enhanced image is improved. The multi-view segmentation model based on NeRF performs feature extraction and segmentation on the image, and finally the mathematical model of the GoogLeNet network classifies and identifies the image information of dam seepage, cracks, settlement, deformation and other diseases.

Claims

1. A method for identifying dam surface defects based on drones, characterized in that: The following steps are involved: Step 1: The millimeter-wave radar carried by the drone collects point cloud data of the dam environment, builds an environmental model, and plans the trajectory of the drone; Step 2: In the process of UAV trajectory planning, a multi-factor constraint function is used to constrain the UAV trajectory to obtain the optimal path; Step 3: During the flight of the drone, the camera carried by the drone collects image information of the dam surface defects, and uses the novel CCNG algorithm to identify and classify the dam surface defect information; In step 2, the multi-factor constraint function is used to constrain the UAV track. Combining the shortest track constraint function F1, the minimum threat constraint function F2, the flight altitude constraint function F3 and the flight angle constraint function F4 of the UAV flight track, the multi-factor constraint function is: Among them, b l is the weight coefficient of each factor, X i for the entire flight path; The shortest track constraint function F1 is specifically: Define the flight track point of the drone as W ij =(x ij ,y ij ,z ij ), represents the three-dimensional spatial position of the UAV at the jth path point in the i-th flight path, then the entire flight path X i is a three-dimensional array containing n path points; the Euclidean distance between two track points is recorded as the track segment || W ij W i,j+1 ||, then the calculation formula of the shortest track constraint function F1 is: The minimum threat constraint function F2 is specifically: Setting the obstacle threat area, the obstacle avoidance threat cost of the drone and its track segment || W ij W i,j+1 || and obstacle threat area center C k The distance d k Inversely proportional, the calculation formula of the minimum threat constraint function F2 is: Among them, K is the obstacle threat area set in the flight environment; and W is the current track point of the UAV. ij The relevant obstacle constraint function T ij The calculation formula is: Among them, the radius of the obstacle threat area is R k , the outer periphery of the obstacle threat area is a circular collision threat warning area D; γ c is the obstacle threat cost penalty coefficient; The flight altitude constraint function F3 is specifically: The flight altitude of the drone during the process of collecting dam disease information is subject to the minimum altitude h min and the maximum height h max Constraints; Move the UAV to the track point W ij The height from the reference terrain is denoted as h ij , then the calculation formula of the flight altitude constraint function F3 is: Among them, the current track point W of the UAV ij The related height constraint function H ij The calculation formula is: Among them, γ h The penalty coefficient for the drone’s flight altitude exceeding the constraint limit; The flight angle constraint function F4 is specifically: The flight angle control parameters of the UAV include the horizontal turning angle and the vertical pitch angle. The calculation formula of the flight angle constraint function F4 is: Among them, a1 and a2 are the penalty coefficients for the horizontal turning angle and vertical pitch angle of the UAV exceeding the constraint conditions; α ij and β i,j are the horizontal turning angle and vertical pitch angle of the jth path point in the UAV’s trajectory, β i,j-1 represents the vertical pitch angle of the j-1th path, β i,j+1 It represents the vertical pitch angle of the j+1th path point. The relevant calculation formula is as follows: Among them, ||W ij W i,j+1 || and ||W i,j+1 W i,j+2 || represents two consecutive track segments in the UAV flight path, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 ' is the projection of two consecutive track points on the three-dimensional plane, z i,j+1 Indicates the vertical depth of the j+1th path point, z i,j represents the vertical depth of the jth path point, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 The calculation formula of ' is: IN ij 'IN i,j+1 '=k×(W ij IN i,j+1 ×k) IN ij+1 'IN i,j+2 '=k×(W ij+1 IN i,j+2 ×k) Where k is the unit vector in the positive direction of the coordinate axis; The novel CCNG algorithm includes CBDNet network to perform denoising on the collected dam surface disease images, CutMix data enhancement to obtain enhanced image data by mixing and superimposing the denoised image data, NeRF multi-view segmentation model to perform feature extraction and segmentation on the enhanced image data, and GoogLeNet network to classify and identify the dam surface disease features.

2. The method for identifying dam surface defects based on drones according to claim 1 is characterized in that: The millimeter wave radar in step 1 completes the collection of point cloud data of the dam environment, and the specific method is: The point cloud data set collected by the millimeter wave radar is S = {s i }, where i∈N, s i =[x i ,y i ,z i ] T , i represents the i-th point cloud data, x i ,y i 、z i Represents the height, width, and depth of the i-th point cloud; divides the three-dimensional point set area into pixels with an imaging side length of d s A finite voxel set V with side length i , to ensure the integrity of the surface image captured by the viewpoint, the center point of the voxel is regarded as the voxel coordinate, and the calculation formula for selecting the viewpoint is: p o =v i ,p o ∈P o Among them, p o is the point that the UAV track needs to pass through; P o This is the point cloud set of the entire dam surface collected by the UAV millimeter wave radar; i Represents a three-dimensional pixel point.

3. The method for identifying dam surface defects based on drone according to claim 1 is characterized in that: In step 1, the trajectory of the UAV is planned based on the collected point cloud data, and the BCO algorithm is used to plan the optimal trajectory of the UAV. When the UAV moves through all viewpoint coordinate positions, the dam inspection mission is completed.

4. A dam surface disease identification system based on drones, characterized in that: Including drone trajectory planning system and dam surface disease identification system; The UAV trajectory planning system uses a multi-factor constraint function to constrain the UAV trajectory during the UAV trajectory planning process to obtain the optimal path; During the flight of the UAV, the camera carried by the UAV collects image information of dam surface diseases and uses a novel CCNG algorithm to identify and classify the dam surface disease information. The novel CCNG algorithm includes CBDNet network to perform noise reduction on the collected dam surface disease images, CutMix data enhancement to obtain enhanced image data by mixing and superimposing the image data after noise reduction, NeRF multi-view segmentation model to perform feature extraction and segmentation on the enhanced image data, and GoogLeNet network to classify and identify the features of dam surface diseases; The multi-factor constraint function is used to constrain the UAV track, combining the shortest track constraint function F1, the minimum threat constraint function F2, the flight altitude constraint function F3 and the flight angle constraint function F4 of the UAV flight track, and then the multi-factor constraint function is: Among them, b l is the weight coefficient of each factor, X i for the entire flight path; The shortest track constraint function F1 is specifically: Define the flight track point of the drone as W ij =(x ij ,y ij ,z ij ), represents the three-dimensional spatial position of the UAV at the jth path point in the i-th flight path, then the entire flight path X i is a three-dimensional array containing n path points; the Euclidean distance between two track points is recorded as the track segment || W ij W i,j+1 ||, then the calculation formula of the shortest track constraint function F1 is: The minimum threat constraint function F2 is specifically: Set the obstacle threat area, the obstacle avoidance threat cost of the drone and its track segment ‖W ij W i,j+1 ‖ and obstacle threat area center C k The distance d k Inversely proportional, the calculation formula of the minimum threat constraint function F2 is: Among them, K is the obstacle threat area set in the flight environment; and W is the current track point of the UAV. ij The relevant obstacle constraint function T ij The calculation formula is: Among them, the radius of the obstacle threat area is R k , the outer periphery of the obstacle threat area is a circular collision threat warning area D; γ c is the obstacle threat cost penalty coefficient; The flight altitude constraint function F3 is specifically: The flight altitude of the drone during the process of collecting dam disease information is subject to the minimum altitude h min and the maximum height h max Constraints; Move the UAV to the track point W ij The height from the reference terrain is denoted as h ij , then the calculation formula of the flight altitude constraint function F3 is: Among them, the current track point W of the UAV ij The related height constraint function H ij The calculation formula is: Among them, γ h The penalty coefficient for the drone’s flight altitude exceeding the constraint limit; The flight angle constraint function F4 is specifically: The flight angle control parameters of the UAV include the horizontal turning angle and the vertical pitch angle. The calculation formula of the flight angle constraint function F4 is: Among them, a1 and a2 are the penalty coefficients for the horizontal turning angle and vertical pitch angle of the UAV exceeding the constraint conditions; α ij and β i,j are the horizontal turning angle and vertical pitch angle of the jth path point in the UAV’s trajectory, β i,j-1 represents the vertical pitch angle of the j-1th path, β i,j+1 It represents the vertical pitch angle of the j+1th path point. The relevant calculation formula is as follows: Among them, ‖W ij W i,j+1 ‖ and ‖W i,j+1 W i,j+2 ‖ represents two consecutive track segments in the UAV flight path, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 ' is the projection of two consecutive track points on the three-dimensional plane, z i,j+1 Indicates the vertical depth of the j+1th path point, z i,j represents the vertical depth of the jth path point, W ij 'W i,j+1 ' and W ij+1 'W i,j+2 The calculation formula of ' is: IN ij 'IN i,j+1 '=k×(W ij IN i,j+1 ×k) IN ij+1 'IN i,j+2 '=k×(W ij+1 IN i,j+2 ×k) Where k is the unit vector in the positive direction of the coordinate axis.

Images