Unmanned aerial vehicle flight path planning and identification method for stay cable corrosion detection

Abstract

The invention discloses an unmanned aerial vehicle flight path planning and identification method for stay cable corrosion detection, and belongs to the technical field of unmanned aerial vehicle corrosion identification. The problems that in the prior art, a traditional stay cable corrosion detection method is low in recognition efficiency, and rapid and intelligent recognition of stay cable corrosion through unmanned aerial vehicle image collection is not considered are solved. The shortest flight path of the unmanned aerial vehicle is planned according to parameter information and operation information of the unmanned aerial vehicle; acquiring a stay cable image to generate an image matrix, and performing image quality enhancement on the image matrix to obtain a new image matrix; equally dividing and updating image pixels to obtain an image with updated pixels; constructing a data set, and inputting the data set into the overall model for training to obtain a trained overall model; and according to an intersection-to-union ratio optimization method, taking the trained basic model as a final model, and outputting a final corrosion identification result. The stay cable corrosion detection efficiency is improved, and the method can be applied to unmanned aerial vehicle traffic monitoring.

Description

Technical Field

[0001] The present invention relates to a UAV flight path planning and identification method, in particular to a UAV flight path planning and identification method for corrosion detection of inclined cables, and belongs to the technical field of UAV corrosion identification. Background Art

[0002] The corrosion of the cables of cable-stayed bridges is mainly caused by environmental factors. For example, humidity, salt, pollutants in the air can cause corrosion of metal materials. In addition, damage to the protective layer of the cables (such as anti-corrosion coatings or sealing systems) will expose the cables directly to harsh environments, accelerating the corrosion process. The corrosion of the cables will have a serious impact on the safety and stability of the bridge. The corrosion will reduce the cross-sectional area of ​​the cables, reduce their bearing capacity, and increase the risk of cable breakage. At the same time, the corrosion will cause uneven stress distribution inside the cables, causing or aggravating fatigue damage, thereby affecting the safety of the entire bridge structure. Therefore, the rapid and intelligent identification of cable corrosion can timely discover potential safety hazards, effectively plan maintenance and repair work, avoid major safety accidents caused by cable breakage, ensure the normal operation and service life of the bridge, improve the identification efficiency and accuracy of cable corrosion, and reduce the workload and risks of manual inspections.

[0003] Due to the limitations of high technical complexity, limited research resources, and diverse application scenarios, there has been insufficient research on the optimization of UAV flight paths. The lack of research on the shortest flight path of UAVs will lead to a variety of efficiency and cost issues. First, UAVs that are not designed with the optimal flight path may prolong the mission completion time and consume more electricity, thereby increasing energy costs and operating costs. This inefficient flight not only increases the wear and tear of the equipment, but may also cause the UAV to be unable to complete the mission due to insufficient power. In addition, unreasonable flight paths may also cause UAVs to face complex environments and flight risks more frequently, increase the possibility of accidents, and affect the quality and timeliness of data collection. Therefore, conducting research on the shortest flight path of UAVs has extremely important practical significance and strategic value. By optimizing the flight path, not only can the operating efficiency of UAVs be significantly improved, energy consumption can be reduced, and equipment life can be extended, but the operating range and flexibility can also be expanded while ensuring safety.

[0004] In the prior art, the patent document with the publication (announcement) number CN108562531B discloses a bridge cable corrosion detection device and proposes an intelligent anchoring structure. A conical anchoring area is designed to anchor the steel wire of the cable, and an anode plate with a lower electrode potential is used to contact the steel wire head to form a cathodic protection to slow down the corrosion process of the steel wire head. The patent document with the publication (announcement) number CN105548360A discloses a composite monitoring method for cable corrosion based on stress concentration and ultrasonic guided waves, which can detect the corrosion of the cable through a self-made corrosion sensor. The prestressed self-balancing system composed of elements and auxiliary frames can realize long-term and quantitative monitoring of the corrosion status and stress corrosion cracks of the cable-stayed cables; however, the above method does not take into account the rapid and intelligent identification of the corrosion of the cable-stayed cables, which will result in the inability to timely detect and evaluate the corrosion conditions outside the anchoring area even after anti-corrosion measures are added to the anchoring structure, thereby failing to fully guarantee the health of the cable-stayed cables and the safety of the bridge, increasing the risk of structural failure, and ultimately affecting the operation and life of the entire cable-stayed bridge. At the same time, the shortest flight path design for drone inspection has not yet been considered.

[0005] In summary, a UAV flight path planning and identification method for corrosion detection of cable-stayed bridges is needed. Summary of the invention

[0006] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0007] In view of this, in order to solve the problem that the traditional cable corrosion detection method in the prior art has low recognition efficiency and does not consider the use of drone image acquisition for rapid and intelligent identification of cable corrosion, the present invention provides a drone flight path planning and identification method for cable corrosion detection.

[0008] The technical solution is as follows: The UAV flight path planning and identification method for cable corrosion detection includes the following steps: S1. Plan the shortest flight path of the drone based on the drone parameter information and operation information; S2. Collect the cable images along the shortest flight path of the UAV to generate an image matrix, and enhance the image quality to obtain a new image matrix; S3. Equally divide the image in the new image matrix and update the image pixels to obtain an image after pixel update; S4. construct a data set based on the pixel-updated image combined with the disease information, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss function, obtain the overall loss function, train the three models again, and obtain the trained overall model; S5. According to the intersection-over-union optimization method, the trained overall model is trained, the trained basic model is used as the final model, and the final rust recognition result is output.

[0009] Furthermore, the S1 specifically includes the following steps: S11. Determine the take-off and landing points of the drone and select the observation position of the operator; S12. Determine the flight altitude AVh according to the focal length and sensor size of the camera carried by the drone, obtain the image coverage and resolution, and set the drone flight speed to AVv; S13. Develop a flight path based on the take-off and landing points of the drone, and determine the coordinate information of the cable and the location information of the drone shooting point; In S13, the take-off point coordinates are AVo (AVoX, AVoY, AVoZ). On this basis, all drone shooting points are determined. The number of any drone shooting point is p, and its corresponding drone shooting point coordinates are AVp (AVpX, AVpY, AVpZ). The number set of drone shooting points is expressed as {1, 2, 3, ..., XN}, and the corresponding drone shooting point coordinate set is expressed as {AV1 (AV1X, AV1Y, AV1Z), AV2 (AV2X, AV2Y, AV2Z), ..., AVXN (AVXNX, AVXNY, AVXNZ)}, where XN is the number of drone shooting points. The adjacent distance between any drone shooting point AVp and its adjacent drone shooting point AV (p+1) is calculated. ; Taking the take-off point AVo as the starting point, according to the distance between any drone shooting point and the take-off point , calculate the distance between all drone shooting points and the take-off point, select the three drone shooting points corresponding to the minimum distance, and randomly select them to obtain the first drone shooting point that the drone arrives at after departing from the take-off point; After determining the first shooting point that the drone arrives at, calculate the distance between all shooting points and the first shooting point that is arrived at, screen out the three drone shooting points corresponding to the smallest distance, and randomly select them to obtain the second shooting point that the drone arrives at after departing from the first shooting point it arrives at. Repeat the above steps until all the shooting points that the drone arrives at are obtained. According to the different random selections each time, calculate the total distance TDis1 of the drone's flight under the first flight mode. Repeat the above steps, the number of repetitions is CN, and integrate to obtain the total distance set of the drone's flight under different flight modes {TDis1, TDis2, TDis3, TDis4, …, TDisCN}, arrange them in order from small to large, and select the total distances and corresponding flight routes ranked in the top TN, TN is the preset value, count the intersection information of TN flight routes, and select the flight route with the largest number of intersections to obtain the shortest flight path of the drone.

[0010] Furthermore, the step S2 specifically includes the following steps: S21. Using a high-definition camera onboard the UAV, collect the cable image ph of the cable-stayed bridge along the shortest flight path of the UAV, and the corresponding image matrix is ​​PH; S22. Construct a 5th-order denoising kernel function N, normalize all elements in the kernel function, and obtain the normalized denoising kernel function ; S23. According to the normalized denoising kernel function , perform denoising kernel function processing on the image to obtain a new image matrix P to achieve image quality enhancement; In S22, the fifth-order denoising kernel function N is expressed as: ; ; Where x=1, 2, 3, 4 or 5, x is the row number of the matrix, and y=1, 2, 3, 4 or 5, y is the column number of the matrix. is the standard deviation of all elements in the kernel function, is the base of natural logarithms; Normalized denoising kernel function It is expressed as: ; In S23, the new image matrix P is expressed as: .

[0011] Furthermore, the step S3 specifically includes the following steps: S31. According to the new image matrix P, the pixel grayscale value is divided into 16 grayscale levels, and the image in the new image matrix P is divided into n equal parts to obtain the divided image; S32. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ; S33.According to the gray frequency cumulative value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update; In S31, the pixel gray value is 0-255, and the gray level is , =1, 2, 3, …, 16, is the grayscale number, and the image after equal division is Pnum, num=1, 2, ..., n; In S32, each gray level in the equally divided image Pnum The corresponding gray frequency cumulative value It is expressed as: ; in, is less than or equal to The gray level number, Indicates the gray level number is Gray level; In S33, the pixel update value corresponding to the gray level It is expressed as: ; in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.

[0012] Furthermore, the step S4 specifically includes the following steps: S41. Based on the pixel updated image, select all images containing rust, broken wires, wear and outer covering damage of the inclined cable as input images; S42. Through manual annotation, the cable corrosion and other three diseases are selected in the input image, and a data set is constructed. The data set is divided into a training set and a test set according to a ratio of 8:2; S43. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, the loss functions of the three models are calculated and integrated to obtain an overall loss function, and the three models are trained using the overall loss function to obtain a trained overall model; In the S42, the rusted area is marked as Rust, and the three diseases, namely, wire breakage, wear and outer sheath damage, are marked as other1, other2 and other3 respectively; In the S43, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; The data set is input into the deep learning model A, and the weights and biases of the neural network units of the deep learning model A are calculated through the error back propagation algorithm, and the training of the deep learning model A is completed to obtain the trained deep learning model A; Loss function of the trained deep learning model A It is expressed as: ; in, is the total number of data samples in the dataset of deep learning model A, For the The true labels of samples, For deep learning model A, predict The probability that a sample is of category 1, is the natural logarithm; The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values ​​of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model A are initially converted into the probability values ​​of the first preliminary output value and the first preliminary output value. , and then further optimize and transform it to obtain the first optimized output value and the enhanced probability value of the first optimized output value ; The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , the output value of the first disease They are the output values ​​of cable corrosion respectively. , output value of broken wire , Wear output value and outer cladding damage output value ; Construct an auxiliary model, namely deep learning model B, which consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model B, and the weights and biases of the neural network units of the deep learning model B are calculated through the error back propagation algorithm, and the training of the deep learning model B is completed to obtain the trained deep learning model B; Loss function of the trained deep learning model B It is expressed as: ; in, is the total number of data samples in the dataset of deep learning model B, For the The true labels of samples, For deep learning model B to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values ​​of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output value and the probability value of the second preliminary output value , and then further optimize and transform it to obtain the second optimized output value and the enhanced probability value of the second optimized output value ; The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the output value of the second disease are the corrosion output values ​​of the second inclined cable respectively , Output value of the second broken wire , Second wear output value and the second outer cladding damage output value ; Construct a reinforcement model, namely deep learning model C. The deep learning model C consists of 1 input layer, 12 convolution layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model C, and the weights and biases of the neural network units of the deep learning model C are calculated through the error back propagation algorithm, and the training of the deep learning model C is completed to obtain the trained deep learning model C; Loss function of the trained deep learning model C It is expressed as: ; in, is the total number of data samples in the dataset of the deep learning model C, For the The true labels of samples, For the deep learning model C to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values ​​of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and the probability value of the third preliminary output value , and then further optimize and transform it to obtain the third optimized output value and the enhanced probability value of the third optimized output value ; The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the third disease output value They are the corrosion output values ​​of the third inclined cable respectively. , Output value of the third broken wire , the third wear output value and the third outer cladding damage output value ; According to the second disease output value is and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient; Probability value based on the first preliminary output value The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient; Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ; According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. The relationship loss function between the output layer of deep learning model C and deep learning model A , define the corresponding overall loss function between deep learning model A, deep learning model B and deep learning model C ; According to the overall loss function Based on the error chain derivation rule, the deep learning model A, the deep learning model B and the deep learning model C are summarized as the overall model for training through the error back propagation algorithm to obtain the trained overall model and output the rust recognition result.

[0013] Furthermore, the S5 specifically includes the following steps: S51. For each target prediction bounding box and the corresponding true bounding box of the trained overall model, calculate the intersection area between the two ; S52. For each target predicted bounding box and the corresponding true bounding box of the trained overall model, calculate the union area of ​​the two ; S53. Calculate the intersection area and the area of ​​the union Corresponding intersection-over-union ratio IoU, introducing adjustment coefficient , construct the loss function of the target position , get the optimized intersection-over-union ratio IoU; S54. According to the optimized intersection-over-union ratio IoU, the optimized intersection-over-union ratio calculation method is used for model training. When the overall loss function When it is minimum, the corresponding model parameters are optimal, which is used as the final optimal model to output the final corrosion recognition result; In S53, the intersection-over-combination ratio threshold is set. , according to the intersection ratio Intersection-over-union threshold The relationship between range, when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding adjustment coefficient of the loss function .

[0014] Furthermore, in S43, the layers in the network structure of the deep learning model A are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.

[0015] Furthermore, in S43, the layers in the network structure of the deep learning model B are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.

[0016] Furthermore, in S43, the layers in the network structure of the deep learning model C are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 6: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 8: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 10: LeakyReLU; Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.

[0017] The beneficial effects of the present invention are as follows: when the present invention adopts a drone to detect the corrosion of a cable-stayed cable, the shortest flight path of the drone helps to improve the detection efficiency, reduce the required time and battery energy consumption, thereby increasing the operating endurance of the drone, reducing the operating cost, and being able to obtain the integrity information of key structures in the shortest time. The method for determining the shortest flight path of the drone, which is crucial for ensuring the safe operation of structures such as bridges, improves the detection effect and reduces energy consumption; the present invention proposes an image quality enhancement method to improve image clarity; the present invention proposes an image pixel update method to construct a denoising kernel function, perform normalization operations and denoising kernel function processing on all elements in the kernel function, thereby enhancing the corrosion features in the image and making the rusted area more obvious; the present invention adopts a constructed overall model to realize the intelligent recognition method for the corrosion of the cable-stayed cable, and improves the accuracy of the corrosion recognition by integrating the loss functions of the three models and combining the advantages of the three models; in order to further obtain more accurate corrosion recognition results, the present invention optimizes the overall model according to the intersection-over-union ratio of the image, accelerates the model training speed, and improves the model training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 The figure is a flow chart of the UAV flight path planning and identification method for cable-stay corrosion detection. DETAILED DESCRIPTION

[0019] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0020] refer to Figure 1 The present embodiment is described in detail. The UAV flight path planning and identification method for cable corrosion detection specifically includes the following steps: S1. Plan the shortest flight path of the drone based on the drone parameter information and operation information; S2. Collect the cable images along the shortest flight path of the UAV to generate an image matrix, and enhance the image quality to obtain a new image matrix; S3. Equally divide the image in the new image matrix and update the image pixels to obtain an image after pixel update; S4. construct a data set based on the pixel-updated image combined with the disease information, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss function, obtain the overall loss function, train the three models again, and obtain the trained overall model; S5. According to the intersection-over-union optimization method, the trained overall model is trained, the trained basic model is used as the final model, and the final rust recognition result is output.

[0021] Furthermore, the S1 specifically includes the following steps: S11. Determine the take-off and landing points of the drone and select the observation position of the operator; Specifically, before the drone takes off, it is necessary to choose a location where the drone can take off and land safely, away from people and traffic, and avoid high-voltage wires and large metal structures that may interfere with the drone's navigation system. At the same time, choose an observation position with a good field of view so that the operator can maintain a good line of sight and facilitate operation; S12. Determine the appropriate flight altitude AVh according to the focal length and sensor size of the camera carried by the drone, obtain sufficient image coverage and resolution, and set the drone flight speed to AVv; S13. Develop a flight path based on the take-off and landing points of the UAV. Based on the drawing parameters of the cable-stayed bridge and the field of view and clarity of the camera carried by the UAV, determine the coordinate information of the cable-stayed cable and the shooting point information of the UAV. Ensure that the images of all point information can fully and clearly cover the cable-stayed cable, and design the shortest flight path for the UAV.

[0022] In S13, the take-off point coordinates are AVo (AVoX, AVoY, AVoZ). On this basis, all drone shooting points are determined. The number of any drone shooting point is p, and its corresponding drone shooting point coordinates are AVp (AVpX, AVpY, AVpZ). The number set of drone shooting points is expressed as {1, 2, 3, ..., XN}, and the corresponding drone shooting point coordinate set is expressed as {AV1 (AV1X, AV1Y, AV1Z), AV2 (AV2X, AV2Y, AV2Z), ..., AVXN (AVXNX, AVXNY, AVXNZ)}, where XN is the number of drone shooting points. The adjacent distance between any drone shooting point AVp and its adjacent drone shooting point AV (p+1) is calculated. ; Adjacent distance It is expressed as: ; Among them, (AVpX, AVpY, AVpZ) are the coordinates of any drone shooting point, and (AV(p+1)X, AV(p+1)Y, AV(p+1)Z) are the coordinates of the drone shooting point adjacent to any drone shooting point; Taking the take-off point AVo as the starting point, according to the distance between any drone shooting point and the take-off point , calculate the distance between all drone shooting points and the take-off point, select the three drone shooting points corresponding to the minimum distance, and randomly select them to obtain the first drone shooting point that the drone arrives at after departing from the take-off point; distance It is expressed as: ; After determining the first shooting point that the drone arrives at, calculate the distance between all shooting points and the first shooting point that is arrived at, screen out the three drone shooting points corresponding to the smallest distance, and randomly select them to obtain the second shooting point that the drone arrives at after departing from the first shooting point that it arrives at. It should be noted that the route of the drone will not return, that is, the drone will not fly to the same shooting point twice. Repeat the above steps until all the shooting points arrived at by the drone are obtained. According to the difference in each random selection, calculate the total flight distance TDis1 of the drone under the first flight mode, repeat the above steps, the number of repetitions is CN, and integrate to obtain the total flight distance set of the drone under different flight modes {TDis1, TDis2, TDis3, TDis4, …, TDisCN}, arrange them in order from small to large, and select the total distance and corresponding flight route ranked in the top TN, TN is a preset value, count the intersection information of TN flight routes, and select the flight route with the largest number of intersections to obtain the shortest flight path of the drone.

[0023] Furthermore, the step S2 specifically includes the following steps: S21. Using a high-definition camera onboard the UAV, collect the cable image ph of the cable-stayed bridge along the shortest flight path of the UAV, and the corresponding image matrix is ​​PH; S22. Construct a 5th-order denoising kernel function N, normalize all elements in the kernel function, and obtain the normalized denoising kernel function ; S23. According to the normalized denoising kernel function Perform denoising kernel function processing on the image to obtain a new image matrix P to achieve image quality enhancement; In S22, the fifth-order denoising kernel function N is expressed as: ; ; Where x=1, 2, 3, 4 or 5, x is the row number of the matrix, and y=1, 2, 3, 4 or 5, y is the column number of the matrix. is the standard deviation of all elements in the kernel function, is the base of natural logarithms; Normalized denoising kernel function It is expressed as: ; In S23, the new image matrix P is expressed as: ; Specifically, a drone is used to collect images of the cables of a cable-stayed bridge. Due to the influence of external environmental factors and the operating characteristics of the drone itself, there is noise in the image. The environmental factors include lighting conditions, weather conditions (such as wind, haze), background interference, etc., and the vibration, flight speed and altitude of the drone, camera resolution and stability, etc. will also affect the image quality. The noise will cover or blur the signs of corrosion on the cables and reduce the visibility of the corrosion features, thereby causing difficulties for the corrosion recognition algorithm and increasing the risk of false detection and missed detection. Therefore, the present invention performs an image quality enhancement operation.

[0024] Furthermore, the step S3 specifically includes the following steps: S31. According to the new image matrix P, the pixel grayscale value is divided into 16 grayscale levels, and the image in the new image matrix P is divided into n equal parts to obtain the divided image; S32. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ; S33.According to the gray frequency cumulative value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update; In S31, the pixel gray value is 0-255, and the gray level is , =1, 2, 3, …, 16, is the grayscale number, and the image after equal division is Pnum, num=1, 2, ..., n; In S32, each gray level in the equally divided image Pnum The corresponding gray frequency cumulative value It is expressed as: ; in, is less than or equal to The gray level number, Indicates the gray level number is Gray level; In S33, the pixel update value corresponding to the gray level It is expressed as: ; in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.

[0025] Specifically, in the collected images of the cables of the cable-stayed bridge, due to the influence of environmental factors and acquisition equipment, the images may have low contrast, uneven lighting, and changes in color and brightness, making it difficult to accurately distinguish the rusted area from the background. Therefore, the pixels of the images of the rusted cables are updated.

[0026] Furthermore, the step S4 specifically includes the following steps: S41. Based on the pixel updated image, select all images containing rust, broken wires, wear and outer covering damage of the inclined cable as input images; S42. Through manual annotation, the cable corrosion and other three diseases are selected in the input image, and a data set is constructed. The data set is divided into a training set and a test set according to a ratio of 8:2; S43. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, the loss functions of the three models are calculated and integrated to obtain an overall loss function, and the three models are trained using the overall loss function to obtain a trained overall model; In the S42, the rusted area is marked as Rust, and the three diseases, namely, wire breakage, wear and outer sheath damage, are marked as other1, other2 and other3 respectively; In the S43, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; The data set is input into the deep learning model A, and the weights and biases of the neural network units of the deep learning model A are calculated through the error back propagation algorithm, and the training of the deep learning model A is completed to obtain the trained deep learning model A; Loss function of the trained deep learning model A It is expressed as: ; in, is the total number of data samples in the dataset of deep learning model A, For the The true labels of samples, For deep learning model A, predict The probability that a sample is of category 1, is the natural logarithm; The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values ​​of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node In order to further clarify the differences between the output values, the model output is converted into a probability distribution to improve the model's ability to distinguish. According to the exponential function , firstly, the output values ​​of the four output unit nodes corresponding to the deep learning model A are initially converted into the first preliminary output value and its probability value, and then further optimized and converted to obtain the first optimized output value and its enhanced probability value in order to amplify the probability difference between the output values ​​and enhance the distinguishing ability of the model; Probability value of the first preliminary output value It is expressed as: ; The enhanced probability value of the first optimized output value It is expressed as: ; Where G is the enhancement coefficient; The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , the output value of the first disease They are the output values ​​of cable corrosion respectively. , output value of broken wire , Wear output value and outer cladding damage output value ; Construct an auxiliary model, namely deep learning model B, which consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model B, and the weights and biases of the neural network units of the deep learning model B are calculated through the error back propagation algorithm, and the training of the deep learning model B is completed to obtain the trained deep learning model B; Loss function of the trained deep learning model B It is expressed as: ; in, is the total number of data samples in the dataset of deep learning model B, For the The true labels of samples, For deep learning model B to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values ​​of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node In order to further clarify the differences between the output values, the model output is converted into a probability distribution to improve the model's ability to distinguish. According to the exponential function , firstly, the output values ​​of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output values ​​and their probability values, and then further optimized and converted to obtain the second optimized output values ​​and their enhanced probability values ​​in order to amplify the probability difference between the output values ​​and enhance the distinguishing ability of the model; Probability value of the second preliminary output value It is expressed as: ; The enhanced probability value of the second optimized output value It is expressed as: ; The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the output value of the second disease are the corrosion output values ​​of the second inclined cable respectively , Output value of the second broken wire , Second wear output value and the second outer cladding damage output value ; Construct a reinforcement model, namely deep learning model C. The deep learning model C consists of 1 input layer, 12 convolution layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model C, and the weights and biases of the neural network units of the deep learning model C are calculated through the error back propagation algorithm, and the training of the deep learning model C is completed to obtain the trained deep learning model C; Loss function of the trained deep learning model C It is expressed as: ; in, is the total number of data samples in the dataset of the deep learning model C, For the The true labels of samples, For the deep learning model C to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values ​​of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node In order to further clarify the differences between the output values, the model output is converted into a probability distribution to improve the model's ability to distinguish. According to the exponential function , firstly, the output values ​​of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and its probability value, and then further optimized and converted to obtain the third optimized output value and its enhanced probability value in order to amplify the probability difference between the output values ​​and enhance the distinguishing ability of the model; The probability value of the third preliminary output value It is expressed as: ; The enhanced probability value of the third optimized output value It is expressed as: ; The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model C , the third disease output value They are the corrosion output values ​​of the third inclined cable respectively. , Output value of the third broken wire , the third wear output value and the third outer cladding damage output value ; According to the second disease output value is and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient; Loss function between deep learning model B and deep learning model A It is expressed as: ; Probability value based on the first preliminary output value The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient; The loss function of the deep learning model A considering the probability value of the first preliminary output value It is expressed as: ; Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ; The relationship loss function between the output layer of deep learning model B and deep learning model A It is expressed as: ; in, is the loss function between deep learning model B and deep learning model A The weight coefficient of is the probability value considering the first preliminary output value The loss function of the deep learning model A is The weight coefficient of Output value according to the third disease and the enhanced probability value of the second optimized output value , construct the loss function between deep learning model C and deep learning model B , get the corresponding weight coefficient; Loss function between deep learning model C and deep learning model B It is expressed as: ; Probability value based on the second preliminary output value And the second actual disease mark , construct the probability value considering the second preliminary output value The loss function of the deep learning model B is , get the corresponding weight coefficient; Consider the probability value of the second preliminary output value The loss function of the deep learning model B is It is expressed as: ; Output value according to the third disease and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model C and deep learning model A , get the corresponding weight coefficient; Loss function between deep learning model C and deep learning model A It is expressed as: ; According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. The relationship loss function between the output layer of deep learning model C and deep learning model A , define the corresponding overall loss function between deep learning model A, deep learning model B and deep learning model C ; The relationship loss function between the output layers of deep learning model C and deep learning model B It is expressed as: ; in, is the loss function between deep learning model C and deep learning model B The weight coefficient of is the probability value considering the second preliminary output value The loss function of the deep learning model B is The weight coefficient of The relationship loss function between the output layer of deep learning model C and deep learning model A It is expressed as: ; in, is the loss function between deep learning model C and deep learning model A The weight coefficient of Overall loss function It is expressed as: ; According to the overall loss function Based on the error chain derivation rule, the deep learning model A, the deep learning model B and the deep learning model C are summarized as the overall model for training through the error back propagation algorithm to obtain the trained overall model and output the rust recognition result.

[0027] Specifically, the corrosion images of the cable usually have complex textures and local features, so a model that can capture these details and has good distinguishing ability is needed. The three models designed in this invention are adapted to the characteristics of drone images and effectively process image information through various levels of design: The input layer is responsible for receiving the image data of the cable collected by the drone; the convolution layer extracts the rust features in the image, including edges, corners, textures, etc., through multiple filters; the activation layer introduces nonlinearity to the network, enabling the model to learn more complex features; the pooling layer reduces the spatial dimension of the feature map, reduces the amount of calculation, and improves the model's invariance to small-scale changes; the merging layer is used to fuse different feature maps from the convolution layer to enhance the model's ability to recognize features; the upsampling layer increases the resolution of the feature map to help the model better locate the rust area; the PANet layer enhances the transfer of features through the feature pyramid and improves the model's ability to learn multi-scale features; the pooling splicing layer combines the features of different pooling areas, further enhancing the model's capture of features of different scales; the SPP layer (spatial pyramid pooling) allows the network to maintain robustness to object size and shape by performing multi-scale pooling on features; the output layer maps all learned features into the final rust recognition results and outputs the probability of each category through the activation function.

[0028] Furthermore, the S5 specifically includes the following steps: S51. For each target predicted bounding box and the corresponding true bounding box of the trained overall model, calculate the intersection area between the two ; S52. For each target predicted bounding box and the corresponding true bounding box of the trained overall model, calculate the union area of ​​the two ; S53. Calculate the intersection area and the area of ​​the union Corresponding intersection-over-union ratio IoU, introducing adjustment coefficient , construct the loss function of the target position , get the optimized intersection-over-union ratio IoU; S54. According to the optimized intersection-over-union ratio IoU, the optimized intersection-over-union ratio calculation method is used for model training. When the overall loss function When it is minimum, the corresponding model parameters are optimal, which is used as the final optimal model to output the final corrosion recognition result; In S53, the intersection over union (IoU) is expressed as: ; Loss function for target location It is expressed as: ; Set the Intersection-over-Union Ratio threshold , according to the intersection ratio Intersection-over-union threshold The relationship between range, when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding adjustment coefficient of the loss function .

[0029] Specifically, during the model training process, the image intersection over union (IoU) is used as a key performance evaluation indicator. IoU measures the degree of overlap between the predicted bounding box and the true bounding box, which reflects the accuracy and reliability of the model prediction. The value of IoU ranges from 0 (no overlap) to 1 (complete overlap). Using IoU helps to evaluate and optimize performance during model training. By setting a threshold to determine the acceptance of the prediction, IoU can be used as part of the loss function to guide the model to more accurately locate and identify the target object during learning. However, the model In the initial stage of the model training process, the value of the intersection over union (IoU) is relatively small, and the optimization speed of the predicted bounding box is slow. In the later stage of model training, the value of the intersection over union (IoU) is relatively large, and the predicted bounding box needs to be fine-tuned to get close to the real bounding box, but the speed is still very slow. Therefore, in order to improve the overall model training speed and model performance, the overall model is optimized according to the intersection over union (IoU) threshold, and the optimized deep learning model A is selected as the final optimal model. The optimized deep learning model A has the highest lightweight level and can be easily deployed in end devices such as edge devices to achieve fast and accurate identification of cable corrosion.

[0030] Furthermore, in S43, the layers in the network structure of the deep learning model A are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.

[0031] Furthermore, in S43, the layers in the network structure of the deep learning model B are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.

[0032] Furthermore, in S43, the layers in the network structure of the deep learning model C are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 6: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 8: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 10: LeakyReLU; Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.

[0033] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.

Claims

1. UAV flight path planning and identification method for cable corrosion detection, It is characterized in that The following steps are involved: S1. Plan the shortest flight path of the drone based on the drone parameter information and operation information; S2. Collect the cable images along the shortest flight path of the UAV to generate an image matrix, and enhance the image quality to obtain a new image matrix; S3. Equally divide the image in the new image matrix and update the image pixels to obtain an image after pixel update; S4. construct a data set based on the pixel-updated image combined with the disease information, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss function, obtain the overall loss function, train the three models again, and obtain the trained overall model; S5. According to the intersection-over-union optimization method, the trained overall model is trained, the trained basic model is used as the final model, and the final rust recognition result is output.

2. The UAV flight path planning and identification method for cable corrosion detection according to claim 1, It is characterized in that The S1 specifically includes the following steps: S11. Determine the take-off and landing points of the drone and select the observation position of the operator; S12. Determine the flight altitude AVh according to the focal length and sensor size of the camera carried by the drone, obtain the image coverage and resolution, and set the drone flight speed to AVv; S13. Develop the shortest flight path based on the take-off and landing points of the UAV, and determine the coordinate information of the cable and the location information of the UAV shooting points; In S13, the take-off point coordinates are AVo (AVoX, AVoY, AVoZ). On this basis, all drone shooting points are determined. The number of any drone shooting point is p, and its corresponding drone shooting point coordinates are AVp (AVpX, AVpY, AVpZ). The number set of drone shooting points is expressed as {1, 2, 3, ..., XN}, and the corresponding drone shooting point coordinate set is expressed as {AV1 (AV1X, AV1Y, AV1Z), AV2 (AV2X, AV2Y, AV2Z), ..., AVXN (AVXNX, AVXNY, AVXNZ)}, where XN is the number of drone shooting points. The adjacent distance between any drone shooting point AVp and its adjacent drone shooting point AV (p+1) is calculated. ; Taking the take-off point AVo as the starting point, according to the distance between any drone shooting point and the take-off point , calculate the distance between all drone shooting points and the take-off point, select the three drone shooting points corresponding to the minimum distance, and randomly select them to obtain the first drone shooting point that the drone arrives at after departing from the take-off point; After determining the first shooting point that the drone arrives at, calculate the distance between all shooting points and the first shooting point that is arrived at, screen out the three drone shooting points corresponding to the smallest distance, and randomly select them to obtain the second shooting point that the drone arrives at after departing from the first shooting point it arrives at. Repeat the above steps until all the shooting points that the drone arrives at are obtained. According to the different random selections each time, calculate the total distance TDis1 of the drone's flight under the first flight mode. Repeat the above steps, the number of repetitions is CN, and integrate to obtain the total distance set of the drone's flight under different flight modes {TDis1, TDis2, TDis3, TDis4, …, TDisCN}, arrange them in order from small to large, and select the total distances and corresponding flight routes ranked in the top TN, TN is the preset value, count the intersection information of TN flight routes, and select the flight route with the largest number of intersections to obtain the shortest flight path of the drone.

3. The UAV flight path planning and identification method for cable corrosion detection according to claim 2, It is characterized in that The S2 specifically includes the following steps: S21. Using a high-definition camera onboard the UAV, collect the cable image ph of the cable-stayed bridge along the shortest flight path of the UAV, and the corresponding image matrix is ​​PH; S22. Construct a 5th-order denoising kernel function N, normalize all elements in the kernel function, and obtain the normalized denoising kernel function ; S23. According to the normalized denoising kernel function , perform denoising kernel function processing on the image to obtain a new image matrix P to achieve image quality enhancement; In S22, the fifth-order denoising kernel function N is expressed as: ； ； Where x=1, 2, 3, 4 or 5, x is the row number of the matrix, and y=1, 2, 3, 4 or 5, y is the column number of the matrix. is the standard deviation of all elements in the kernel function, is the base of natural logarithms; Normalized denoising kernel function It is expressed as: ； In S23, the new image matrix P is expressed as: 。 4. The UAV flight path planning and identification method for cable corrosion detection according to claim 3, It is characterized in that The S3 specifically includes the following steps: S31. According to the new image matrix P, the pixel grayscale value is divided into 16 grayscale levels, and the image in the new image matrix P is divided into n equal parts to obtain the divided image; S32. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ; S33.According to the gray frequency cumulative value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update; In S31, the pixel gray value is 0-255, and the gray level is , =1, 2, 3, …, 16, is the grayscale number, and the image after equal division is Pnum, num=1, 2, ..., n; In S32, each gray level in the equally divided image Pnum The corresponding grayscale frequency cumulative value It is expressed as: ； in, is less than or equal to The gray level number, Indicates the gray level number is Gray level; In S33, the pixel update value corresponding to the gray level It is expressed as: ； in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.

5. The method for UAV flight path planning and identification for cable corrosion detection according to claim 4, It is characterized in that The S4 specifically includes the following steps: S41. Based on the pixel updated image, select all images containing rust, broken wires, wear and outer covering damage of the inclined cable as input images; S42. Through manual annotation, the cable corrosion and other three diseases are selected in the input image, and a data set is constructed. The data set is divided into a training set and a test set according to a ratio of 8:2; S43. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, the loss functions of the three models are calculated and integrated to obtain an overall loss function, and the three models are trained using the overall loss function to obtain a trained overall model; In the S42, the rusted area is marked as Rust, and the three diseases, namely, wire breakage, wear and outer sheath damage, are marked as other1, other2 and other3 respectively; In the S43, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; The data set is input into the deep learning model A, and the weights and biases of the neural network units of the deep learning model A are calculated through the error back propagation algorithm, and the training of the deep learning model A is completed to obtain the trained deep learning model A; Loss function of the trained deep learning model A It is expressed as: ； in, is the total number of data samples in the dataset of deep learning model A, For the The true labels of samples, For deep learning model A, predict The probability that a sample is of category 1, is the natural logarithm; The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values ​​of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model A are initially converted into the probability values ​​of the first preliminary output value and the first preliminary output value. , and then further optimize and transform it to obtain the first optimized output value and the enhanced probability value of the first optimized output value ; The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , the output value of the first disease They are the output values ​​of cable corrosion respectively. , output value of broken wire , Wear output value and outer cladding damage output value ; Construct an auxiliary model, namely deep learning model B, which consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model B, and the weights and biases of the neural network units of the deep learning model B are calculated through the error back propagation algorithm, and the training of the deep learning model B is completed to obtain the trained deep learning model B; Loss function of the trained deep learning model B It is expressed as: ； in, is the total number of data samples in the dataset of deep learning model B, For the The true labels of samples, For deep learning model B to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values ​​of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output value and the probability value of the second preliminary output value , and then further optimize and transform it to obtain the second optimized output value and the enhanced probability value of the second optimized output value ; The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the output value of the second disease are the corrosion output values ​​of the second inclined cable respectively , Output value of the second broken wire , Second wear output value and the second outer cladding damage output value ; Construct a reinforcement model, namely deep learning model C. The deep learning model C consists of 1 input layer, 12 convolution layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model C, and the weights and biases of the neural network units of the deep learning model C are calculated through the error back propagation algorithm, and the training of the deep learning model C is completed to obtain the trained deep learning model C; Loss function of the trained deep learning model C It is expressed as: ； in, is the total number of data samples in the dataset of the deep learning model C, For the The true labels of samples, For the deep learning model C to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values ​​of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node , according to the exponential function First, the output values ​​of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and the probability value of the third preliminary output value , and then further optimize and transform it to obtain the third optimized output value and the enhanced probability value of the third optimized output value ; The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model C , the third disease output value They are the corrosion output values ​​of the third inclined cable respectively. , Output value of the third broken wire , the third wear output value and the third outer cladding damage output value ; According to the second disease output value is and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient; Probability value based on the first preliminary output value The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient; Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ; According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. The relationship loss function between the output layer of deep learning model C and deep learning model A , define the corresponding overall loss function between deep learning model A, deep learning model B and deep learning model C ; According to the overall loss function Based on the error chain derivation rule, the deep learning model A, the deep learning model B and the deep learning model C are summarized as the overall model for training through the error back propagation algorithm to obtain the trained overall model and output the rust recognition result.

6. The UAV flight path planning and identification method for cable corrosion detection according to claim 5, It is characterized in that The S5 specifically includes the following steps: S51. For each target prediction bounding box and the corresponding true bounding box of the trained overall model, calculate the intersection area between the two ; S52. For each target predicted bounding box and the corresponding true bounding box of the trained overall model, calculate the union area of ​​the two ; S53. Calculate the intersection area and the area of ​​the union Corresponding intersection-over-union ratio IoU, introducing adjustment coefficient , construct the loss function of the target position , get the optimized intersection-over-union ratio IoU; S54. According to the optimized intersection-over-union ratio IoU, the trained overall model is trained. When the overall loss function When it is minimum, the corresponding model parameters are optimal, which is used as the final optimal model to output the final corrosion recognition result; In S53, the intersection-over-combination ratio threshold is set. , according to the intersection ratio Intersection-over-union threshold The relationship between range, when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding loss function of the target position Adjustment factor ,when When , the corresponding adjustment coefficient of the loss function .

7. The method for UAV flight path planning and identification for cable corrosion detection according to claim 6, It is characterized in that In S43, the layers in the network structure of the deep learning model A are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 9×9, stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons:

4.

8. The method for UAV flight path planning and identification for cable corrosion detection according to claim 7, It is characterized in that In S43, the layers in the network structure of the deep learning model B are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons:

4.

9. The method for UAV flight path planning and identification for cable corrosion detection according to claim 8, It is characterized in that In S43, the layers in the network structure of the deep learning model C are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 6: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 8: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 10: LeakyReLU; Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.

Images