Stay cable broken wire prediction and classification method based on unmanned aerial vehicle intelligent identification
Through the intelligent drone identification technology, the shortest flight path is planned and combined with multi-model training, the problem of low wire break recognition efficiency of cable-stayed bridge cable cables is solved, efficient and safe wire break detection and prediction are achieved, and the bridge is extended Service life.
Patent Information
- Application Number
- CN202411594986.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, the cable-stayed bridge cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-stayed cable-sta , resulting in low operating efficiency and high safety risks.
Using the cable-stayed cable wire break prediction and classification method based on intelligent recognition of drone, the cable-stayed cable image is collected and updated, and the loss function of the basic model, auxiliary model and reinforcement model is trained to achieve the cable-stayed cable Identification and prediction of wire breakage.
It improves the efficiency and accuracy of cable-stayed cable wire break detection, reduces energy consumption, ensures detection safety, and improves recognition capabilities through multi-model loss function optimization, extends the service life of cable-stayed bridges.
Smart Images

Figure CN119091339A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for predicting and classifying broken wires of a stay cable, and in particular to a method for predicting and classifying broken wires of a stay cable based on intelligent identification by unmanned aerial vehicles, and belongs to the technical field of prediction of disease of stay cables. Background Art
[0002] During the service of the cables of cable-stayed bridges, major defects such as cable wire breakage may occur. The main causes of the defects include long-term exposure to alternating stress and fatigue, corrosion (such as rust), environmental factors (such as temperature difference, humidity, and chemical erosion), and construction or design defects. The above factors may cause the steel wires in the cables to break over time, that is, wire breakage. Cable wire breakage will have a serious impact on the safety and stability of the bridge. Wire breakage reduces the bearing capacity of the cables, which may cause uneven deformation of the bridge and even cause structural failure in extreme cases. Cable wire breakage will also accelerate the fatigue process of steel wires in other cables, increasing the risk of the overall structure. Rapid and intelligent identification of cable wire breakage can timely discover potential structural problems, avoid safety accidents, ensure the normal operation of the bridge and the safety of users, and at the same time help reduce maintenance costs and extend the service life of the bridge.
[0003] In addition, the design method of the shortest flight path for UAVs involves many challenges. The lack of research on the shortest flight path will take longer time and more electricity to complete the same inspection task, which not only reduces operational efficiency but also increases costs. When UAVs operate according to non-shortest flight paths, they may cause frequent excessive flights or unnecessary maneuvers, which increases equipment wear and reduces the service life of the UAVs. They may also fail to complete the scheduled tasks due to insufficient power. Most importantly, unoptimized flight paths may increase safety risks, such as accidental collisions or loss of control of UAVs, which seriously endangers traffic safety. Research on methods to determine the shortest flight path for UAVs has important practical significance and long-term value, which can significantly improve mission efficiency, reduce flight time and energy consumption, thereby reducing operating costs and improving economic benefits. At the same time, accurate path planning can ensure that UAVs complete their tasks safely and efficiently, reduce accidental risks, and improve the quality and accuracy of data collection.
[0004] On the other hand, at present, there is a lack of prediction and classification methods for cable-stayed cable breakage, which leads to inaccurate timing for cable-stay replacement, increases maintenance costs, and shortens the service life of cable-stayed bridges.
[0005] Prior art 1, a patent document with publication (announcement) number CN109682460A discloses a method for identifying broken wires in cables based on vibration signals. The method collects lateral vibration signals by installing an acceleration sensor in the cable or the hanger. The pulse vibration amplitude of the cable body generated when the wire is broken is used, combined with the signal energy or power as a characteristic indicator, and then the energy ratio is calculated as a judgment parameter for broken wires to distinguish the difference between broken wire signals and conventional vibrations. This method helps to achieve long-term monitoring of cable damage; Prior art 2, a patent document with publication (announcement) number CN114595733A discloses a bridge cable broken wire signal recognition method and system based on a long short-term memory network, which comprehensively utilizes time domain, frequency domain and time-frequency analysis to extract signal features, and constructs a comprehensive feature vector to characterize the acoustic emission signal, and then uses the LSTM network to establish a high-performance broken wire signal recognition model. Compared with traditional machine learning algorithms, it can more accurately distinguish broken wire signals from non-broken wire signals, showing better recognition effect; However, the above methods do not consider the rapid identification and prediction of broken wires in the cable-stayed bridge. If the broken wires cannot be identified in time, the best time for bridge maintenance will be delayed, the risk of structural failure will be increased, the safe operation of the bridge will be affected, and higher maintenance costs and potential safety accidents will be caused. At the same time, it has not yet considered designing the shortest flight path of the drone for cable-stayed cable monitoring to shorten the operation time.
[0006] Therefore, a method for predicting and classifying broken wires of cable-stayed bridges based on intelligent identification of UAVs is needed. Summary of the invention
[0007] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.
[0008] In view of this, in order to solve the problem of low recognition efficiency of the traditional method for identifying broken wires of inclined cables based on signal features in the prior art, the present invention provides a method for predicting and classifying broken wires of inclined cables based on intelligent recognition of unmanned aerial vehicles.
[0009] The technical solution is as follows: A method for predicting and classifying broken wires of inclined cables based on intelligent identification by unmanned aerial vehicles, specifically including the following steps: S1. Plan the shortest flight path of the drone based on the drone parameter information and operation information; S2. Collect the cable image along the shortest flight path of the UAV, divide it into equal parts and update the image pixels to obtain an image with updated pixels; S3. construct a data set based on the pixel updated image combined with the disease information, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss functions of the three models, and obtain the fused loss function; S4. Establish an inter-layer loss function from the enhanced model to the basic model, combine the fused loss function to obtain the total loss function, train the three models, use the trained basic model as the final model, and output the final broken wire recognition result; S5. According to the final broken wire identification result, confirm that there are broken wires in the inclined cable, obtain the first broken wire status result based on the total number of broken wires, obtain the second broken wire status result based on the cable force value borne by a single unbroken steel wire in the inclined cable, and obtain the final broken wire status by combining the broken wire status results.
[0010] Furthermore, the S1 specifically includes the following steps: S11. Determine the take-off and landing points of the drone and select the observation position of the operator; S12. Determine the flight altitude AVh according to the focal length and sensor size of the camera carried by the drone, obtain the image coverage and resolution, and set the drone flight speed to AVv; S13. Develop the shortest flight path based on the take-off and landing points of the UAV, and determine the coordinate information of the cable and the location information of the UAV shooting points; In S13, the take-off point coordinates are AVo (AVoX, AVoY, AVoZ). On this basis, all drone shooting points are determined. The number of any drone shooting point is p, and its corresponding drone shooting point coordinates are AVp (AVpX, AVpY, AVpZ). The number set of drone shooting points is expressed as {1, 2, 3, ..., XN}, and the corresponding drone shooting point coordinate set is expressed as {AV1 (AV1X, AV1Y, AV1Z), AV2 (AV2X, AV2Y, AV2Z), ..., AVXN (AVXNX, AVXNY, AVXNZ)}, where XN is the number of drone shooting points. The adjacent distance between any drone shooting point AVp and its adjacent drone shooting point AV (p+1) is calculated. ; Taking the take-off point AVo as the starting point, according to the distance between any drone shooting point and the take-off point , calculate the distance between all drone shooting points and the take-off point, select the three drone shooting points corresponding to the minimum distance, and randomly select them to obtain the first drone shooting point that the drone arrives at after departing from the take-off point; After determining the first shooting point that the drone arrives at, calculate the distance between all shooting points and the first shooting point that is arrived at, screen out the three drone shooting points corresponding to the smallest distance, and randomly select them to obtain the second shooting point that the drone arrives at after departing from the first shooting point it arrives at. Repeat the above steps until all the shooting points that the drone arrives at are obtained. According to the different random selections each time, calculate the total distance TDis1 of the drone's flight under the first flight mode. Repeat the above steps, the number of repetitions is CN, and integrate to obtain the total distance set of the drone's flight under different flight modes {TDis1, TDis2, TDis3, TDis4, …, TDisCN}, arrange them in order from small to large, and select the total distances and corresponding flight routes ranked in the top TN, TN is the preset value, count the intersection information of TN flight routes, and select the flight route with the largest number of intersections to obtain the shortest flight path of the drone.
[0011] Furthermore, the step S2 specifically includes the following steps: S21. Use a high-definition camera on a drone to collect images of the cables of the cable-stayed bridge along the shortest flight path that has been determined; S22. Divide the pixel grayscale values of the cable image into 16 grayscale levels, divide the cable image into n equal parts, and obtain an image after being divided; S23. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ; S24.According to the gray frequency accumulation value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update; In S22, the pixel gray value is 0-255, and the gray level is , =1, 2, 3, …, 16, is the grayscale number, and the image after equal division is Pnum, num=1, 2, ..., n; In S23, each gray level in the equally divided image Pnum The corresponding gray frequency cumulative value It is expressed as: ; in, is less than or equal to The gray level number, Indicates the gray level number is Gray level; In S24, the pixel update value corresponding to the gray level It is expressed as: ; in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.
[0012] Furthermore, the step S3 specifically includes the following steps: S31. Based on the pixel updated image, select all images containing broken wires, wear, outer covering damage and corrosion of the inclined cable as input images; S32. Through manual annotation, the cable broken wire area and other three types of defects are selected in the input image, and a data set is constructed. The data set is divided into a training set and a test set according to a ratio of 8:2; S33. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, and the loss functions of the three models are calculated and integrated to obtain a fused loss function; In S32, the broken wire area is marked as Wire, and the three types of diseases, namely wear, outer layer damage and rust, are marked as other1, other2 and other3 respectively; In the S33, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; Input the data set into the deep learning model A, calculate the weights and biases of the neural network units of the deep learning model A through the error back propagation algorithm, complete the training of the deep learning model A, and obtain the trained deep learning model A and its loss function ; The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node , according to the exponential function First, the output values of the four output unit nodes corresponding to the deep learning model A are initially converted into the probability values of the first preliminary output value and the first preliminary output value. , and then further optimize and transform it to obtain the first optimized output value and the enhanced probability value of the first optimized output value ; The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , the output value of the first disease They are the output values of the first inclined cable broken wire respectively , the output value of the first wear , the first outer layer damage output value and the first rust output value ; Construct an auxiliary model, namely deep learning model B, which consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; Input the data set into the deep learning model B, calculate the weights and biases of the neural network units of the deep learning model B through the error back propagation algorithm, complete the training of the deep learning model B, and obtain the trained deep learning model B and its loss function ; The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node , according to the exponential function First, the output values of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output value and the probability value of the second preliminary output value , and then further optimize and transform it to obtain the second optimized output value and the enhanced probability value of the second optimized output value ; The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the output value of the second disease They are the output values of the second inclined cable broken wire respectively , the output value of the second wear , the second outer layer damage output value and the second rust output value ; Construct a reinforcement model, namely a deep learning model C. The deep learning model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; Input the data set into the deep learning model C, calculate the weights and biases of the neural network units of the deep learning model C through the error back propagation algorithm, complete the training of the deep learning model C, and obtain the trained deep learning model C and its loss function ; The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node , according to the exponential function First, the output values of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and the probability value of the third preliminary output value , and then further optimize and transform it to obtain the third optimized output value and the enhanced probability value of the third optimized output value ; The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the third disease output value They are the output values of the third inclined cable broken wire respectively. , Output value of the third wear , the third outer layer damage output value and the third rust output value ; According to the second disease output value is and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient; Probability value based on the first preliminary output value The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient; Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ; According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. The relationship loss function between the output layer of deep learning model C and deep learning model A .
[0013] Furthermore, the step S4 specifically includes the following steps: S41. Establish an inter-layer loss function from the enhanced model to the basic model, and combine the fused loss function to obtain the total loss function; S42. Based on the chain derivation rule, the three models are trained by error back propagation to obtain the trained deep learning model A, i.e., the optimal model; In the S41, the selected layers of the deep learning model A are connected to the selected layers of the deep learning model C, the selected layers in the deep learning model A include the second layer: convolution layer 1, the third layer: activation layer 1, the fourth layer: convolution layer 2, and the twenty-eighth layer: output layer, and the selected layers of the deep learning model C include the fourth layer: convolution layer 2, the sixth layer: convolution layer 3, the tenth layer: convolution layer 5, and the thirty-sixth layer: output layer; The selected layers of the deep learning model A are , the corresponding size is ,in, =2, 3, 4 and 28, is the selected number of layers of the deep learning model A, is the selected layer width of the deep learning model A, is the selected layer length of the deep learning model A, is the number of channels of the selected layer of deep learning model A, and the selected layer of deep learning model C is , the corresponding size is ,in, =4, 6, 10 and 36, is the selected number of layers of the deep learning model C, is the selected layer width of the deep learning model C, is the selected layer length of the deep learning model C, is the number of selected layer channels of the deep learning model C; when When , a convolution operation is added to the selected layer output of the deep learning model A, with a convolution kernel size of 1×1 and a number of convolution kernels. , get the selected layer output of deep learning model A , and the corresponding output value of each channel is , =1, 2, 3, …, , Output channel number for the selected layer of deep learning model A and the selected layer of deep learning model C The corresponding output values of each channel are , =1, 2, 3, …, , Output channel number for the selected layer of deep learning model C, and establish the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C ; when When , a convolution operation is added to the selected layer output of the deep learning model C, with a convolution kernel size of 1×1 and a number of convolution kernels. , the selected layer output of the deep learning model C is , and the corresponding output value of each channel is , the selected layers of the deep learning model C The corresponding output values of each channel are , establish the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C ; According to the above calculation, the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C is obtained: , the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C And the loss function between the fourth layer of deep learning model A and the tenth layer of deep learning model C , combining the relationship loss function between the output layers of deep learning model B and deep learning model A , the relationship loss function between the output layers of deep learning model C and deep learning model B The relationship loss function between the output layer of deep learning model C and deep learning model A , define the total loss function ; Total loss function It is expressed as: ; in, is the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C The corresponding weight coefficient is, is the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C The corresponding weight coefficient is, is the loss function between the fourth layer of deep learning model A and the tenth layer of deep learning model C The corresponding weight coefficient.
[0014] Furthermore, the S5 specifically includes the following steps: S51. Use the optimal model to identify the broken wires of the inclined cables, and conduct on-site review to confirm that the inclined cables have broken wires in their steel wires; S52. According to the presence of broken wires in the inclined cable, predict the total number of broken wires in the inclined cable at any time, and obtain the first broken wire state result according to the first broken wire state determination condition; S53. Measure the cable force value of the inclined cable where the wire is broken, calculate the cable force value borne by the single steel wire that is not broken after the wire is broken, and obtain the second wire breaking state result according to the second wire breaking state determination condition; S54. Combining the first broken wire state result and the second broken wire state result, when the results are consistent, directly obtain the final broken wire state, when the results are inconsistent, the broken wire state result with a more severe broken wire degree is used as the final broken wire state; In the above S52, for the inclined cable with broken wires in the steel wire , predict the total number of broken wires in the inclined cable at any time , according to the total number of inclined cables , the number of unbroken wires is ; The total number of broken wires in the cable It is expressed as: ; in, The time when the wire breaks first occur. is the number of initial broken wires, is the adjustment factor, is the current time; According to the total number of broken wires in the inclined cable , the broken wire status is divided into four categories, namely, slight broken wire, moderate broken wire, severe broken wire and serious broken wire. The first broken wire status judgment condition is as follows: (1) , judged as slight wire breakage, This is the quantity limit corresponding to slight wire breakage; (2) , judged as moderate wire breakage, The quantity limit corresponding to moderate wire breakage; (3) , judged as severe wire breakage, This is the quantity limit corresponding to severe wire breakage; (4) , it is judged as severe wire breakage; After judging the current time of the stay cable, the first broken wire state result is obtained; In S53, the cable force value borne by the single steel wire that is not broken after the wire breaks It is expressed as: ; Based on the cable force value borne by a single steel wire that is not broken in the inclined cable , the broken wire status is divided into four categories, namely, slight broken wire, moderate broken wire, severe broken wire and serious broken wire. The first broken wire status judgment condition is as follows: (1) , judged as slight wire breakage, This is the quantity limit corresponding to slight wire breakage; (2) , judged as moderate wire breakage, The quantity limit corresponding to moderate wire breakage; (3) , judged as severe wire breakage, This is the quantity limit corresponding to severe wire breakage; (4) , it is judged as severe wire breakage; After judging the current time of the stay cable, the second broken wire state result is obtained.
[0015] Furthermore, in S33, the layers in the network structure of the deep learning model A are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.
[0016] Furthermore, in S33, the layers in the network structure of the deep learning model B are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.
[0017] Furthermore, in S33, the layers in the network structure of the deep learning model C are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 6: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 8: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 10: LeakyReLU; Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.
[0018] The beneficial effects of the present invention are as follows: the present invention constructs a method for determining the shortest flight path of an unmanned aerial vehicle, improves the subsequent broken wire detection effect of the inclined cable, reduces energy consumption, and ensures the safety of the detection process; the present invention updates the image pixels of the collected inclined cable image, enhances the visibility of the broken wire area in the image, and supports the accuracy of the subsequent output results; the present invention constructs the output layer loss function of the basic model, the auxiliary model and the reinforcement model, so that the basic model can learn complex data faster during the training process; the present invention jointly optimizes the three models based on the multi-model loss function to obtain the optimal model, realizes the broken wire identification of the inclined cable of the cable-stayed bridge, and improves the recognition ability and generalization ability of the model; the present invention proposes to predict and classify the broken wires in the inclined cable according to the total number of broken wires in the inclined cable and the cable force value borne by the single steel wire of the unbroken wire in the inclined cable, thereby improving the service safety of the cable-stayed bridge and extending the service life. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 The figure is a flow chart of the broken wire prediction and classification method of the inclined cable based on UAV intelligent identification. DETAILED DESCRIPTION
[0020] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.
[0021] refer to Figure 1 The present embodiment is described in detail. The method for predicting and classifying broken wires of a stay cable based on intelligent identification of a drone specifically includes the following steps: S1. Plan the shortest flight path of the drone based on the drone parameter information and operation information; S2. Collect the cable image along the shortest flight path of the UAV, divide it into equal parts and update the image pixels to obtain an image with updated pixels; S3. construct a data set based on the pixel updated image combined with the disease information, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss functions of the three models, and obtain the fused loss function; S4. Establish an inter-layer loss function from the enhanced model to the basic model, combine the fused loss function to obtain the total loss function, train the three models, use the trained basic model as the final model, and output the final broken wire recognition result; S5. According to the final broken wire identification result, confirm that there are broken wires in the inclined cable, obtain the first broken wire status result based on the total number of broken wires in the inclined cable, obtain the second broken wire status result based on the cable force value borne by a single unbroken steel wire in the inclined cable, and obtain the final broken wire status by combining the broken wire status results.
[0022] Furthermore, the S1 specifically includes the following steps: S11. Determine the take-off and landing points of the drone and select the observation position of the operator; Specifically, before the drone takes off, it is necessary to choose a location where the drone can take off and land safely, away from people and traffic, and avoid high-voltage wires and large metal structures that may interfere with the drone's navigation system. At the same time, choose an observation position with a good field of view so that the operator can maintain a good line of sight and facilitate operation; S12. Determine the appropriate flight altitude AVh according to the focal length and sensor size of the camera carried by the drone, obtain sufficient image coverage and resolution, and set the drone flight speed to AVv; S13. Develop a flight path based on the take-off and landing points of the UAV. Based on the drawing parameters of the cable-stayed bridge and the field of view and clarity of the camera carried by the UAV, determine the coordinate information of the cable-stayed cables and the location information of the UAV shooting points. Ensure that the images of all location information can fully and clearly cover the cable-stayed cables, and design the shortest flight path for the UAV. In S13, the take-off point coordinates are AVo (AVoX, AVoY, AVoZ). On this basis, all drone shooting points are determined. The number of any drone shooting point is p, and its corresponding drone shooting point coordinates are AVp (AVpX, AVpY, AVpZ). The number set of drone shooting points is expressed as {1, 2, 3, ..., XN}, and the corresponding drone shooting point coordinate set is expressed as {AV1 (AV1X, AV1Y, AV1Z), AV2 (AV2X, AV2Y, AV2Z), ..., AVXN (AVXNX, AVXNY, AVXNZ)}, where XN is the number of drone shooting points. The adjacent distance between any drone shooting point AVp and its adjacent drone shooting point AV (p+1) is calculated. ; Adjacent distance It is expressed as: ; Among them, (AVpX, AVpY, AVpZ) are the coordinates of any drone shooting point, and (AV(p+1)X, AV(p+1)Y, AV(p+1)Z) are the coordinates of the drone shooting point adjacent to any drone shooting point; Taking the take-off point AVo as the starting point, according to the distance between any drone shooting point and the take-off point , calculate the distance between all drone shooting points and the take-off point, select the three drone shooting points corresponding to the minimum distance, and randomly select them to obtain the first drone shooting point that the drone arrives at after departing from the take-off point; distance It is expressed as: ; After determining the first shooting point that the drone arrives at, calculate the distance between all shooting points and the first shooting point that is arrived at, screen out the three drone shooting points corresponding to the smallest distance, and randomly select them to obtain the second shooting point that the drone arrives at after departing from the first shooting point that it arrives at. It should be noted that the route of the drone will not return, that is, the drone will not fly to the same shooting point twice. Repeat the above steps until all the shooting points arrived at by the drone are obtained. According to the difference in each random selection, calculate the total flight distance TDis1 of the drone under the first flight mode, repeat the above steps, the number of repetitions is CN, and integrate to obtain the total flight distance set of the drone under different flight modes {TDis1, TDis2, TDis3, TDis4, …, TDisCN}, arrange them in order from small to large, and select the total distance and corresponding flight route ranked in the top TN, TN is a preset value, count the intersection information of TN flight routes, and select the flight route with the largest number of intersections to obtain the shortest flight path of the drone.
[0023] Furthermore, the step S2 specifically includes the following steps: S21. Use a high-definition camera on a drone to collect images of the cables of the cable-stayed bridge along the shortest flight path that has been determined; S22. Divide the pixel grayscale values of the cable image into 16 grayscale levels, divide the cable image into n equal parts, and obtain an image after being divided; S23. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ; S24.According to the gray frequency accumulation value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update; In S22, the pixel gray value is 0-255, and the gray level is , =1, 2, 3, …, 16, is the grayscale number, and the image after equal division is Pnum, num=1, 2, ..., n; In S23, each gray level in the equally divided image Pnum The corresponding grayscale frequency cumulative value It is expressed as: ; in, is less than or equal to The gray level number, Indicates the gray level number is Gray level; In S24, the pixel update value corresponding to the gray level It is expressed as:
[0024] in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of gray level; Specifically, in the captured images of the cables of the cable-stayed bridge, due to the influence of environmental factors and acquisition equipment, the images may have low contrast, uneven lighting, and changes in color and brightness, making it difficult to accurately distinguish the broken wire area from the background. Therefore, the image of the broken wire of the cable-stayed cable is updated with pixels.
[0025] Furthermore, the step S3 specifically includes the following steps: S31. Based on the pixel updated image, select all images containing broken wires, wear, outer covering damage and corrosion of the inclined cable as input images; S32. Through manual annotation, the cable broken wire area and other three types of defects are selected in the input image, and a data set is constructed. The data set is divided into a training set and a test set according to a ratio of 8:2; S33. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, and the loss functions of the three models are calculated and integrated to obtain a fused loss function; In S32, the broken wire area is marked as Wire, and the three types of diseases, namely wear, outer layer damage and rust, are marked as other1, other2 and other3 respectively; In the S33, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; The data set is input into the deep learning model A, and the weights and biases of the neural network units of the deep learning model A are calculated through the error back propagation algorithm, and the training of the deep learning model A is completed to obtain the trained deep learning model A; Loss function of the trained deep learning model A It is expressed as: ; in, is the total number of data samples in the dataset of deep learning model A, For the The true labels of samples, For deep learning model A, predict The probability that a sample is of category 1, is the natural logarithm; The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node In order to further clarify the differences between the output values, the model output is converted into a probability distribution to improve the model's ability to distinguish. According to the exponential function , firstly, the output values of the four output unit nodes corresponding to the deep learning model A are initially converted into the first preliminary output value and its probability value, and then further optimized and converted to obtain the first optimized output value and its enhanced probability value in order to amplify the probability difference between the output values and enhance the distinguishing ability of the model; Probability value of the first preliminary output value It is expressed as: ; The enhanced probability value of the first optimized output value It is expressed as: ; in, G is the enhancement coefficient; The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , the output value of the first disease They are the output values of the first inclined cable broken wire respectively. , the output value of the first wear , the first outer layer damage output value and the first rust output value ; Construct an auxiliary model, namely deep learning model B, which consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model B, and the weights and biases of the neural network units of the deep learning model B are calculated through the error back propagation algorithm, and the training of the deep learning model B is completed to obtain the trained deep learning model B; Loss function of the trained deep learning model B It is expressed as: ; in, is the total number of data samples in the dataset of deep learning model B, For the The true labels of samples, For deep learning model B to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node In order to further clarify the differences between the output values, the model output is converted into a probability distribution to improve the model's ability to distinguish. According to the exponential function , firstly, the output values of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output values and their probability values, and then further optimized and converted to obtain the second optimized output values and their enhanced probability values in order to amplify the probability difference between the output values and enhance the distinguishing ability of the model; Probability value of the second preliminary output value It is expressed as: ; The enhanced probability value of the second optimized output value It is expressed as: ; The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the output value of the second disease They are the output values of the second inclined cable broken wire respectively , the output value of the second wear , the second outer layer damage output value and the second rust output value ; Construct a reinforcement model, namely a deep learning model C. The deep learning model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; The data set is input into the deep learning model C, and the weights and biases of the neural network units of the deep learning model C are calculated through the error back propagation algorithm, and the training of the deep learning model C is completed to obtain the trained deep learning model C; Loss function of the trained deep learning model C It is expressed as: ; in, is the total number of data samples in the dataset of the deep learning model C, For the The true labels of samples, For the deep learning model C to predict The probability that a sample is class 1; The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node In order to further clarify the differences between the output values, the model output is converted into a probability distribution to improve the model's ability to distinguish. According to the exponential function , firstly, the output values of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and its probability value, and then further optimized and converted to obtain the third optimized output value and its enhanced probability value in order to amplify the probability difference between the output values and enhance the distinguishing ability of the model; The probability value of the third preliminary output value It is expressed as: ; The enhanced probability value of the third optimized output value It is expressed as: ; The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the third disease output value They are the output values of the third inclined cable broken wire respectively , Output value of the third wear , the third outer layer damage output value and the third rust output value ; According to the second disease output value is and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient; Loss function between deep learning model B and deep learning model A It is expressed as: ; Probability value based on the first preliminary output value The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient; Consider the probability value of the first preliminary output value The loss function of the deep learning model A is It is expressed as: ; Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ; The relationship loss function between the output layer of deep learning model B and deep learning model A It is expressed as: ; in, is the loss function between deep learning model B and deep learning model A The weight coefficient of is the probability value considering the first preliminary output value The loss function of the deep learning model A is The weight coefficient of Output value according to the third disease and the enhanced probability value of the second optimized output value , construct the loss function between deep learning model C and deep learning model B , get the corresponding weight coefficient; Loss function between deep learning model C and deep learning model B It is expressed as: ; Probability value based on the second preliminary output value And the second actual disease mark , construct the probability value considering the second preliminary output value The loss function of the deep learning model B is , get the corresponding weight coefficient; Consider the probability value of the second preliminary output value The loss function of the deep learning model B is It is expressed as: ; Output value according to the third disease and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model C and deep learning model A , get the corresponding weight coefficient; Loss function between deep learning model C and deep learning model A It is expressed as: ; According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. The relationship loss function between the output layer of deep learning model C and deep learning model A , define the corresponding overall loss function between deep learning model A, deep learning model B and deep learning model C ; The relationship loss function between the output layers of deep learning model C and deep learning model B It is expressed as: ; in, is the loss function between deep learning model C and deep learning model B The weight coefficient of is the probability value considering the second preliminary output value The loss function of the deep learning model B is The weight coefficient of The relationship loss function between the output layer of deep learning model C and deep learning model A It is expressed as: ; in, is the loss function between deep learning model C and deep learning model A The weight coefficient of Specifically, the image characteristics of the broken wire of the inclined cable usually include small and irregular broken wire shapes, complex background noise, and illumination changes, which requires the model to have high-resolution feature extraction capabilities, strong background noise suppression capabilities, and robustness to illumination changes. The three models designed in the present invention can effectively achieve the above requirements through various levels of design. The specific functions of each layer structure are as follows: Input layer: receives the image data of the cable collected by the drone; Convolution layer: uses filters to extract the features of the image, including the local features of the broken wire shape. Multiple convolution layers can extract complex features from the bottom layer to the top layer; Activation layer: introduces nonlinearity to the network to help the network learn complex feature representations; Pooling layer: reduces the spatial size of the feature map, reduces the number of parameters and the amount of calculation, and at the same time improves the robustness of feature extraction, making the model less sensitive to small displacements and deformations (such as the subtle differences of broken wires); Merging layer: fuses feature information at different levels to help restore the details of the broken wire; Upsampling layer: improves the spatial resolution of the feature map, making the network It can locate the specific position of the broken wire more accurately; PANet layer: improves the information flow between low-level and high-level features, enhances the representation ability of features, and helps capture the fine features of broken wires; Pooling and splicing layer: refers to splicing the features of different pooling areas to further enhance the model's ability to capture features of different scales; SPP layer: through spatial pyramid pooling, the network can learn broken wire features of different sizes and enhance robustness to size changes; Output layer: maps the learned high-level features to the final recognition results, and outputs the probability of each category through the activation function, so that the model can predict whether there are broken wires in the image.
[0026] Furthermore, the step S4 specifically includes the following steps: S41. Establish an inter-layer loss function from the enhanced model to the basic model, and combine the fused loss function to obtain the total loss function; S42. Based on the chain derivation rule, the three models are trained by error back propagation to obtain the trained deep learning model A, i.e., the optimal model; In the S41, the selected layers of the deep learning model A are connected to the selected layers of the deep learning model C, the selected layers in the deep learning model A include the second layer: convolution layer 1, the third layer: activation layer 1, the fourth layer: convolution layer 2, and the twenty-eighth layer: output layer, and the selected layers of the deep learning model C include the fourth layer: convolution layer 2, the sixth layer: convolution layer 3, the tenth layer: convolution layer 5, and the thirty-sixth layer: output layer; The selected layers of the deep learning model A are , the corresponding size is ,in, =2, 3, 4 and 28, is the selected number of layers of the deep learning model A, is the selected layer width of the deep learning model A, is the selected layer length of the deep learning model A, is the number of channels of the selected layer of deep learning model A, and the selected layer of deep learning model C is , the corresponding size is ,in, =4, 6, 10 and 36, is the selected number of layers of the deep learning model C, is the selected layer width of the deep learning model C, is the selected layer length of the deep learning model C, is the number of selected layer channels of the deep learning model C; when When , a convolution operation is added to the selected layer output of the deep learning model A, with a convolution kernel size of 1×1 and a number of convolution kernels. , get the selected layer output of deep learning model A , and the corresponding output value of each channel is , =1, 2, 3, ..., , Output channel number for the selected layer of deep learning model A and the selected layer of deep learning model C The corresponding output values of each channel are , =1, 2, 3, ..., , Output channel number for the selected layer of deep learning model C, and establish the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C ; when When , the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C is It is expressed as: ; when When , a convolution operation is added to the selected layer output of the deep learning model C, with a convolution kernel size of 1×1 and a number of convolution kernels. , the selected layer output of the deep learning model C is , and the corresponding output value of each channel is , the selected layers of the deep learning model C The corresponding output values of each channel are , establish the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C ; when When , the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C is It is expressed as: ; According to the above calculation, the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C is obtained: , the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C And the loss function between the fourth layer of deep learning model A and the tenth layer of deep learning model C , combining the relationship loss function between the output layers of deep learning model B and deep learning model A , the relationship loss function between the output layers of deep learning model C and deep learning model B The relationship loss function between the output layer of deep learning model C and deep learning model A , define the total loss function ; Total loss function It is expressed as: ; in, is the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C The corresponding weight coefficient is, is the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C The corresponding weight coefficient is, is the loss function between the fourth layer of deep learning model A and the tenth layer of deep learning model C The corresponding weight coefficient; Specifically, deep learning model B is an auxiliary model. In order to enrich the feature representation capability of the basic deep learning model A, an inter-layer association relationship is established between the basic model (deep learning model A) and the enhanced model (deep learning model C), so that the features of different levels in the enhanced model can be transferred to the basic model, thereby further improving the performance improvement effect of the basic model. The trained deep learning model A is the optimal model. The deep learning model A has the highest level of lightweight and can be easily deployed in end devices such as edge devices to achieve fast and accurate identification of broken wires in inclined cables.
[0027] Furthermore, the S5 specifically includes the following steps: S51. Use the optimal model to identify the broken wires of the inclined cables, and conduct on-site review to confirm that the inclined cables have broken wires in their steel wires; S52. According to the presence of broken wires in the inclined cable, predict the total number of broken wires in the inclined cable at any time, and obtain the first broken wire state result according to the first broken wire state determination condition; S53. Measure the cable force value of the inclined cable where the wire is broken, calculate the cable force value borne by the single steel wire that is not broken after the wire is broken, and obtain the second wire breaking state result according to the second wire breaking state determination condition; S54. Combining the first broken wire state result and the second broken wire state result, when the results are consistent, directly obtain the final broken wire state, when the results are inconsistent, the broken wire state result with a more severe broken wire degree is used as the final broken wire state; In the above S52, for the inclined cable with broken wires in the steel wire , predict the total number of broken wires in the inclined cable at any time , according to the total number of inclined cables , the number of unbroken wires is ; The total number of broken wires in the cable It is expressed as: ; in, The time when the wire breaks first occur. is the number of initial broken wires, is the adjustment factor, is the current time; According to the total number of broken wires in the inclined cable , the broken wire status is divided into four categories, namely, slight broken wire, moderate broken wire, severe broken wire and serious broken wire. The first broken wire status judgment condition is as follows: (1) , judged as slight wire breakage, This is the quantity limit corresponding to slight wire breakage; (2) , judged as moderate wire breakage, The quantity limit corresponding to moderate wire breakage; (3) , judged as severe wire breakage, This is the quantity limit corresponding to severe wire breakage; (4) , it is judged as severe wire breakage; After judging the current time of the stay cable, the first broken wire state result is obtained; In S53, the cable force value borne by the single steel wire that is not broken after the wire breaks It is expressed as: ; Based on the cable force value borne by a single steel wire that is not broken in the inclined cable , the broken wire status is divided into four categories, namely, slight broken wire, moderate broken wire, severe broken wire and serious broken wire. The first broken wire status judgment condition is as follows: (1) , judged as slight wire breakage, This is the quantity limit corresponding to slight wire breakage; (2) , judged as moderate wire breakage, The quantity limit corresponding to moderate wire breakage; (3) , judged as severe wire breakage, This is the quantity limit corresponding to severe wire breakage; (4) , it is judged as severe wire breakage; After judging the current time (which is consistent with the judgment time based on the number of broken wires in the stay cable), the second broken wire state result is obtained; Specifically, a cable-stayed bridge includes multiple cables, and different cables have different lengths. The longer the length, the greater the force. Therefore, for cables of different lengths, when the same number of wires are broken, the influence on the force of other wires is also different. When analyzing the influence of wire breakage, the influence of force needs to be considered. Compare the first broken wire status result obtained based on the total number of broken wires and the second broken wire status result obtained by the cable force value borne by a single steel wire in the inclined cable that is not broken. When the broken wire status results obtained by the two are inconsistent, the result with a more serious broken wire degree is used as the final evaluation result. For example, if the two evaluation results are severe broken wire and serious broken wire respectively, the final evaluation result is severe broken wire.
[0028] Furthermore, in S33, the layers in the network structure of the deep learning model A are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.
[0029] Furthermore, in S33, the layers in the network structure of the deep learning model B are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.
[0030] Furthermore, in S33, the layers in the network structure of the deep learning model C are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 6: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 8: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 10: LeakyReLU; Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.
[0031] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.
Claims
1. A method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles, characterized in that: The following steps are involved: S1. Plan the shortest flight path of the drone based on the drone parameter information and operation information; S2. Collect the cable image along the shortest flight path of the UAV, divide it into equal parts and update the image pixels to obtain an image with updated pixels; S3. construct a data set based on the pixel updated image combined with the disease information, input the data set into the established basic model, auxiliary model and enhanced model for training, fuse the loss functions of the three models, and obtain the fused loss function; S4. Establish an inter-layer loss function from the enhanced model to the basic model, combine the fused loss function to obtain the total loss function, train the three models, use the trained basic model as the final model, and output the final broken wire recognition result; S5. According to the final broken wire identification result, confirm that there are broken wires in the inclined cable, obtain the first broken wire status result based on the total number of broken wires in the inclined cable, obtain the second broken wire status result based on the cable force value borne by a single unbroken steel wire in the inclined cable, and obtain the final broken wire status by combining the broken wire status results.
2. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 1 is characterized in that: The S1 specifically includes the following steps: S11. Determine the take-off and landing points of the drone and select the observation position of the operator; S12. Determine the flight altitude AVh according to the focal length and sensor size of the camera carried by the drone, obtain the image coverage and resolution, and set the drone flight speed to AVv; S13. Develop the shortest flight path based on the take-off and landing points of the UAV, and determine the coordinate information of the cable and the location information of the UAV shooting points; In S13, the take-off point coordinates are AVo (AVoX, AVoY, AVoZ). On this basis, all drone shooting points are determined. The number of any drone shooting point is p, and its corresponding drone shooting point coordinates are AVp (AVpX, AVpY, AVpZ). The number set of drone shooting points is expressed as {1, 2, 3, ..., XN}, and the corresponding drone shooting point coordinate set is expressed as {AV1 (AV1X, AV1Y, AV1Z), AV2 (AV2X, AV2Y, AV2Z), ..., AVXN (AVXNX, AVXNY, AVXNZ)}, where XN is the number of drone shooting points. The adjacent distance between any drone shooting point AVp and its adjacent drone shooting point AV (p+1) is calculated. ; Taking the take-off point AVo as the starting point, according to the distance between any drone shooting point and the take-off point , calculate the distance between all drone shooting points and the take-off point, select the three drone shooting points corresponding to the minimum distance, and randomly select them to obtain the first drone shooting point that the drone arrives at after departing from the take-off point; After determining the first shooting point that the drone arrives at, calculate the distance between all shooting points and the first shooting point that is arrived at, screen out the three drone shooting points corresponding to the smallest distance, and randomly select them to obtain the second shooting point that the drone arrives at after departing from the first shooting point it arrives at. Repeat the above steps until all the shooting points that the drone arrives at are obtained. According to the different random selections each time, calculate the total distance TDis1 of the drone's flight under the first flight mode. Repeat the above steps, the number of repetitions is CN, and integrate to obtain the total distance set of the drone's flight under different flight modes {TDis1, TDis2, TDis3, TDis4, …, TDisCN}, arrange them in order from small to large, and select the total distances and corresponding flight routes ranked in the top TN, TN is the preset value, count the intersection information of TN flight routes, and select the flight route with the largest number of intersections to obtain the shortest flight path of the drone.
3. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 2 is characterized in that: The S2 specifically includes the following steps: S21. Use a high-definition camera on a drone to collect images of the cables of the cable-stayed bridge along the shortest flight path that has been determined; S22. Divide the pixel grayscale values of the cable image into 16 grayscale levels, divide the cable image into n equal parts, and obtain an image after being divided; S23. Count the pixel frequencies corresponding to each gray level in the equally divided image , calculate the grayscale frequency cumulative value corresponding to each grayscale level in the equally divided image ; S24.According to the gray frequency accumulation value , construct a pixel value update method and calculate the pixel update value corresponding to the gray level , get the image after pixel update; In S22, the pixel gray value is 0-255, and the gray level is , =1, 2, 3, …, 16, is the grayscale number, and the image after equal division is Pnum, num=1, 2, ..., n; In S23, each gray level in the equally divided image Pnum The corresponding gray frequency cumulative value It is expressed as: ; in, is less than or equal to The gray level number, Indicates the gray level number is Gray level; In S24, the pixel update value corresponding to the gray level It is expressed as: ; in, is the minimum non-zero value among all pixel frequency accumulation values, is the total number of pixels corresponding to the equally divided image Pnum, is the maximum value of the gray level.
4. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 3 is characterized in that: The S3 specifically includes the following steps: S31. Based on the pixel updated image, select all images containing broken wires, wear, outer covering damage and corrosion of the inclined cable as input images; S32. Through manual annotation, the cable broken wire area and other three types of defects are selected in the input image, and a data set is constructed. The data set is divided into a training set and a test set according to a ratio of 8:2; S33. According to the data set, a basic model, an auxiliary model and an enhanced model are constructed, and the loss functions of the three models are calculated and integrated to obtain a fused loss function; In S32, the broken wire area is marked as Wire, and the three types of diseases, namely wear, outer layer damage and rust, are marked as other1, other2 and other3 respectively; In the S33, a network structure of a basic model, i.e., a deep learning model A, is constructed, and the deep learning model A consists of 1 input layer, 9 convolutional layers, 9 activation layers, 3 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer, and 1 output layer; Input the data set into the deep learning model A, calculate the weights and biases of the neural network units of the deep learning model A through the error back propagation algorithm, complete the training of the deep learning model A, and obtain the trained deep learning model A and its loss function ; The output value of the output unit node corresponding to the deep learning model A is , respectively, are the output values of the first output unit node , the output value of the second output unit node , the output value of the third output unit node and the output value of the fourth output unit node , according to the exponential function First, the output values of the four output unit nodes corresponding to the deep learning model A are initially converted into the probability values of the first preliminary output value and the first preliminary output value. , and then further optimize and transform it to obtain the first optimized output value and the enhanced probability value of the first optimized output value ; The enhanced probability value using the first optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model A. After the training is completed, the output value of the first disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , the output value of the first disease They are the output values of the first inclined cable broken wire respectively. , the output value of the first wear , the first outer layer damage output value and the first rust output value ; Construct an auxiliary model, namely deep learning model B, which consists of 1 input layer, 9 convolutional layers, 9 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; Input the data set into the deep learning model B, calculate the weights and biases of the neural network units of the deep learning model B through the error back propagation algorithm, complete the training of the deep learning model B, and obtain the trained deep learning model B and its loss function ; The output value of the output unit node corresponding to the deep learning model B is , respectively, are the output values of the fifth output unit node , the output value of the sixth output unit node , the output value of the seventh output unit node and the output value of the eighth output unit node , according to the exponential function First, the output values of the four output unit nodes corresponding to the deep learning model B are initially converted into the second preliminary output value and the probability value of the second preliminary output value , and then further optimize and transform it to obtain the second optimized output value and the enhanced probability value of the second optimized output value ; The enhanced probability value using the second optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model B. After the training is completed, the output value of the second disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the output value of the second disease They are the output values of the second inclined cable broken wire respectively , the output value of the second wear , the second outer layer damage output value and the second rust output value ; Construct a reinforcement model, namely a deep learning model C. The deep learning model C consists of 1 input layer, 12 convolutional layers, 12 activation layers, 5 pooling layers, 1 merging layer, 1 upsampling layer, 1 PANet layer, 1 pooling splicing layer, 1 SPP layer and 1 output layer; Input the data set into the deep learning model C, calculate the weights and biases of the neural network units of the deep learning model C through the error back propagation algorithm, complete the training of the deep learning model C, and obtain the trained deep learning model C and its loss function ; The output value of the output unit node corresponding to the deep learning model C is , respectively, are the output values of the ninth output unit node , the output value of the tenth output unit node , the output value of the eleventh output unit node and the output value of the twelfth output unit node , according to the exponential function First, the output values of the four output unit nodes corresponding to the deep learning model C are initially converted into the third preliminary output value and the probability value of the third preliminary output value , and then further optimize and transform it to obtain the third optimized output value and the enhanced probability value of the third optimized output value ; The enhanced probability value using the third optimized output value And the data set, combined with the error chain derivative rule, continue to train the deep learning model C. After the training is completed, the output value of the third disease is obtained The actual disease annotations corresponding to the nodes in the output layer of the deep learning model B , the third disease output value They are the output values of the third inclined cable broken wire respectively. , Output value of the third wear , the third outer layer damage output value and the third rust output value ; According to the second disease output value is and the enhanced probability value of the first optimized output value , construct the loss function between deep learning model B and deep learning model A , get the corresponding weight coefficient; Probability value based on the first preliminary output value The actual disease annotations corresponding to the nodes in the output layer of the deep learning model A , construct the probability value considering the first preliminary output value The loss function of the deep learning model A is , get the corresponding weight coefficient; Based on the loss function between deep learning model B and deep learning model A and the probability value considering the first preliminary output value The loss function of the deep learning model A is , establish the relationship loss function between the output layer of deep learning model B and deep learning model A ; According to the above operations, the relationship loss function between the output layers of deep learning model C and deep learning model B is established. The relationship loss function between the output layer of deep learning model C and deep learning model A .
5. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 4 is characterized in that: The S4 specifically includes the following steps: S41. Establish an inter-layer loss function from the enhanced model to the basic model, and combine the fused loss function to obtain the total loss function; S42. Based on the chain derivation rule, the three models are trained by error back propagation to obtain the trained deep learning model A, i.e., the optimal model; In the S41, the selected layers of the deep learning model A are connected to the selected layers of the deep learning model C, the selected layers in the deep learning model A include the second layer: convolution layer 1, the third layer: activation layer 1, the fourth layer: convolution layer 2, and the twenty-eighth layer: output layer, and the selected layers of the deep learning model C include the fourth layer: convolution layer 2, the sixth layer: convolution layer 3, the tenth layer: convolution layer 5, and the thirty-sixth layer: output layer; The selected layers of the deep learning model A are , the corresponding size is ,in, =2, 3, 4 and 28, is the selected number of layers of the deep learning model A, is the selected layer width of the deep learning model A, is the selected layer length of the deep learning model A, is the number of channels of the selected layer of deep learning model A, and the selected layer of deep learning model C is , the corresponding size is ,in, =4, 6, 10 and 36, is the selected number of layers of the deep learning model C, is the selected layer width of the deep learning model C, is the selected layer length of the deep learning model C, is the number of selected layer channels of the deep learning model C; when When , a convolution operation is added to the selected layer output of the deep learning model A, with a convolution kernel size of 1×1 and a number of convolution kernels. , get the selected layer output of deep learning model A , and the corresponding output value of each channel is , =1, 2, 3, …, , Output channel number for the selected layer of deep learning model A and the selected layer of deep learning model C The corresponding output values of each channel are , =1, 2, 3, …, , Output channel number for the selected layer of deep learning model C, and establish the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C ; when When , a convolution operation is added to the selected layer output of the deep learning model C, with a convolution kernel size of 1×1 and a number of convolution kernels. , the selected layer output of the deep learning model C is , and the corresponding output value of each channel is , the selected layers of the deep learning model C The corresponding output values of each channel are , establish the loss function between the selected layer of deep learning model A and the selected layer of deep learning model C ; According to the above calculation, the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C is obtained: , the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C And the loss function between the fourth layer of deep learning model A and the tenth layer of deep learning model C , combining the relationship loss function between the output layers of deep learning model B and deep learning model A , the relationship loss function between the output layers of deep learning model C and deep learning model B The relationship loss function between the output layer of deep learning model C and deep learning model A , define the total loss function ; Total loss function It is expressed as: ; in, is the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C The corresponding weight coefficient is, is the loss function between the third layer of deep learning model A and the sixth layer of deep learning model C The corresponding weight coefficient is, is the loss function between the fourth layer of deep learning model A and the tenth layer of deep learning model C The corresponding weight coefficient.
6. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 5 is characterized in that: The S5 specifically includes the following steps: S51. Use the optimal model to identify the broken wires of the inclined cables, and conduct on-site review to confirm that the inclined cables have broken wires in their steel wires; S52. According to the presence of broken wires in the inclined cable, predict the total number of broken wires in the inclined cable at any time, and obtain the first broken wire state result according to the first broken wire state determination condition; S53. Measure the cable force value of the inclined cable where the wire is broken, calculate the cable force value borne by the single steel wire that is not broken after the wire is broken, and obtain the second wire breaking state result according to the second wire breaking state determination condition; S54. Combining the first broken wire state result and the second broken wire state result, when the results are consistent, directly obtain the final broken wire state, when the results are inconsistent, the broken wire state result with a more severe broken wire degree is used as the final broken wire state; In the above S52, for the inclined cable with broken wires in the steel wire , predict the total number of broken wires in the inclined cable at any time , according to the total number of inclined cables , the number of unbroken wires is ; The total number of broken wires in the cable It is expressed as: ; in, The time when the wire breaks first occur. is the number of initial broken wires, is the adjustment factor, is the current time; According to the total number of broken wires in the inclined cable , the broken wire status is divided into four categories, namely, slight broken wire, moderate broken wire, severe broken wire and serious broken wire. The first broken wire status judgment condition is as follows: (1) , judged as slight wire breakage, This is the quantity limit corresponding to slight wire breakage; (2) , judged as moderate wire breakage, The quantity limit corresponding to moderate wire breakage; (3) , judged as severe wire breakage, This is the quantity limit corresponding to severe wire breakage; (4) , it is judged as severe wire breakage; After judging the current time of the stay cable, the first broken wire state result is obtained; In S53, the cable force value borne by the single steel wire that is not broken after the wire breaks It is expressed as: ; Based on the cable force value borne by a single steel wire that is not broken in the inclined cable , the broken wire status is divided into four categories, namely, slight broken wire, moderate broken wire, severe broken wire and serious broken wire. The first broken wire status judgment condition is as follows: (1) , judged as slight wire breakage, This is the quantity limit corresponding to slight wire breakage; (2) , judged as moderate wire breakage, The quantity limit corresponding to moderate wire breakage; (3) , judged as severe wire breakage, This is the quantity limit corresponding to severe wire breakage; (4) , it is judged as severe wire breakage; After judging the current time of the stay cable, the second broken wire state result is obtained.
7. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 6 is characterized in that: In S33, the layers in the network structure of the deep learning model A are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 9×9, stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 13×13, Stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 5: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 768, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons:
4.
8. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 7 is characterized in that: In S33, the layers in the network structure of the deep learning model B are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 4: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the five pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolution layer 5: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 6: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 7: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 7: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 8: LeakyReLU; Convolutional layer 9: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Output layer: Type: Softmax, Number of neurons:
4.
9. The method for predicting and classifying broken wires of inclined cables based on intelligent identification of unmanned aerial vehicles according to claim 8 is characterized in that: In S33, the layers in the network structure of the deep learning model C are connected in the following order: Input layer: image size 227×227×3; Convolutional layer 1: kernel size 3×3, number of kernels 32, stride: 1, batch normalization; Activation layer 1: LeakyReLU; Convolutional layer 2: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 2: LeakyReLU; Convolution layer 3: convolution kernel size 3×3, number of convolution kernels 128, stride: 1, batch normalization; Activation layer 3: LeakyReLU; Convolutional layer 4: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 4: LeakyReLU; Convolutional layer 5: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 5: LeakyReLU; Convolutional layer 6: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 6: LeakyReLU; SPP layer: extracts features of different scales and fuses them to increase the receptive field of the model; Pooling layer 1: Type: global average pooling, pooling size: 5×5, stride: 1; Pooling layer 2: Type: global average pooling, pooling size: 7×7, Stride: 1; Pooling layer 3: Type: global average pooling, pooling size: 9×9, Stride: 1; Pooling layer 4: Type: global average pooling, pooling size: 11×11, Stride: 1; Pooling layer 5: Type: global average pooling, pooling size: 13×13, stride: 1; Pooling concatenation layer: connect the outputs of the three pooling layers; PANet layer: aggregates features at different levels and implements feature pyramids through upsampling and downsampling operations; Convolutional layer 7: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 7: LeakyReLU; Convolutional layer 8: kernel size 3×3, number of kernels 1280, stride: 2, batch normalization; Activation layer 8: LeakyReLU; Upsampling layer: uses the nearest neighbor interpolation method to upsample to the same size as the pooling splicing layer; Merging layer: The output feature map of the downsampling layer is merged with the output feature map of the upsampling layer; Convolutional layer 9: kernel size 3×3, number of kernels 512, stride: 1, batch normalization; Activation layer 9: LeakyReLU; Convolution layer 10: kernel size 3×3, number of kernels 256, stride: 1, batch normalization; Activation layer 10: LeakyReLU; Convolutional layer 11: kernel size 3×3, number of kernels 128, stride: 1, batch normalization; Activation layer 11: LeakyReLU; Convolutional layer 12: kernel size 3×3, number of kernels 64, stride: 1, batch normalization; Activation layer 12: LeakyReLU; Output layer: Type: Softmax, Number of neurons: 4.
Citation Information
Patent Citations
Recognition method for inhaul cable wire breakage based on vibration signals
CN109682460A
Bridge cable wire breakage signal identification method and system based on long short-term memory network
CN114595733A
Bridge disease real-time detection method based on model pruning
CN111832607A
Multi-model combined unmanned aerial vehicle garbage classification method for smart garden
CN113627481A
YOLOv6-based bridge disease inspection method and system
CN117151680A
