A bridge crack identification and detection system and method based on drone

Abstract

The invention discloses a bridge crack recognition and detection system and method based on an unmanned aerial vehicle. The detection system comprises an unmanned aerial vehicle, an unmanned aerial vehicle route design and route planning unit, an unmanned aerial vehicle safe flight unit, an image acquisition and processing unit, a crack recognition and analysis unit, and a crack information data output unit. When in use, the system can formulate reasonable flight routes for different bridge crack recognition and detection tasks and recognition and detection parts. At the same time, when flying in a sunny area, it can absorb solar energy and increase the working time of the unmanned aerial vehicle. The cracks in the image are recognized and detected by a convolutional neural network, and the width parameter information of the cracks can be calculated. The future development trend of the cracks can be predicted through microscopic observation. Finally, a 3D stereoscopic model view of the bridge is generated, which fully displays the position information and actual parameter information of the cracks, and the cracks are maintained and maintained in a targeted manner. The system has the characteristics of easy use, comprehensive detection and recognition, and high efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge detection, and in particular to a bridge crack identification and detection system and method based on an unmanned aerial vehicle. Background Art

[0002] Bridge construction is one of the important national infrastructure projects. Bridge engineering is the lifeline project related to the coordinated development of society and economy. The rapid development of bridge construction, huge capital investment, and prominent role in the economic society have made people pay more and more attention to the safety and durability of bridges. At the same time, my country's bridges are facing the same problems as bridges in many countries in the world, that is, a large number of bridges have entered the aging stage, and some bridges have already suffered from various diseases, but they have been operating all year round, posing huge major application safety hazards.

[0003] In the current use of bridges, reports of bridge damage due to cracks affecting engineering quality are common, and nearly half of my country's bridges have entered the disease exposure period. Therefore, regular identification, inspection, evaluation and maintenance of bridges are crucial for traffic safety;

[0004] Traditional bridge inspection mainly relies on manpower and related inspection equipment for direct inspection, which has the disadvantages of low efficiency, failure to inspect inaccessible parts, and even danger to inspectors. In recent years, in response to the demand for inspection of inaccessible parts of conventional bridge inspections, and the rapid development of drone technology and its advantages of miniaturization, flexibility, and speed, with the help of drones as carriers, combined with big data analysis, deep learning algorithms, and convolutional neural network model analysis, a method for accurately identifying and inspecting structural defects in inaccessible parts of conventional bridge inspections has emerged.

[0005] Based on this, it is a very necessary detection means to design a bridge crack identification and detection system and detection method based on drones to solve the problems existing in the above-mentioned prior art. Summary of the invention

[0006] In view of the above-mentioned problems, the present invention aims to provide a bridge crack identification and detection system and method based on drones. The system can formulate reasonable drone flight routes for specific bridge crack identification and detection tasks and identification and detection parts; at the same time, when flying in sunny areas, the solar panels it carries can absorb solar energy and increase the working time of the drone; the cracks in the image are identified and detected through a convolutional neural network, and the width parameter information of the cracks can be calculated through an algorithm, and the future development trend of the cracks can be predicted through microscopic observation. Finally, a 3D stereo model view of the bridge is generated through modeling software, which fully displays the location information and actual parameter information of the cracks, and performs targeted maintenance and care for the cracks. The system is easy to use, comprehensive detection and identification, and high efficiency.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A bridge crack identification and detection system based on a drone, comprising a drone, a drone route design and route planning unit, a drone safe flight unit, an image acquisition and processing unit, a crack identification and analysis unit, and a crack information data output unit;

[0009] The UAV route design and route planning unit is used to generate a UAV flight route according to the bridge inspection task, the bridge inspection location and the UAV parameter information;

[0010] The UAV safe flight unit is used to ensure the flight safety of the UAV according to the UAV obstacle avoidance module and the energy module, and at the same time obtain the UAV location information according to the GPS positioning module to determine the crack location;

[0011] The image acquisition and processing unit is used to acquire bridge surface crack images and extract crack features;

[0012] The crack identification and analysis unit is used to detect and identify cracks frame by frame on the PC side according to the crack video information, calculate the crack width parameter information, and predict the crack development trend;

[0013] The crack information data output unit is used to export effective crack information images, store actual crack parameter information and position information in a crack information database, establish a 3D stereoscopic model view of the bridge, and extract crack parameter information from the crack information database and display it at the node position corresponding to the bridge.

[0014] Preferably, the image acquisition and processing unit includes a camera anti-shake module, a light supplement module, an industrial camera module, an image denoising module and a feature extraction module.

[0015] The camera anti-shake module is used to reduce the blur of the video image caused by external environmental interference;

[0016] The light supplement module is used to ensure that there is sufficient light when collecting pictures of the shaded side of the bridge, and to balance the shooting brightness;

[0017] The industrial camera module is used for collecting high-resolution video recordings;

[0018] The image denoising module is used to perform denoising on the collected crack image information;

[0019] The feature extraction module is used to extract feature information from the collected crack image information.

[0020] Preferably, the crack identification and analysis unit includes a crack identification module, a crack analysis module, a crack prediction module and a PC terminal.

[0021] The crack recognition module is used to recognize and detect crack image information collected by the industrial camera on the PC side according to the convolutional neural network;

[0022] The crack analysis module is used to calculate the width of the crack according to the identified crack image through the MATLAB algorithm program;

[0023] The crack prediction module is used to magnify the cracks based on the identified crack images using a MATLAB program to observe the crack growth trends at subtle locations and predict future development trends.

[0024] A method for detecting a bridge crack using an unmanned aerial vehicle (UAV)-based bridge crack detection system, the detection method comprising:

[0025] Step 1: Crack information collection process

[0026] Step 1.1 The drone collects crack information on the bridge surface through the industrial camera carried according to the planned flight route, and stores all the video recordings of the cracks collected in the SD card and transmits them remotely to the PC;

[0027] Step 2: Data Transfer Process

[0028] Step 2.1 After receiving the video of the cracks collected by the drone, the PC first uses the image denoising module to remove the noise in the image;

[0029] Step 2.2 After denoising the image, the feature extraction module is used to extract crack feature information from the collected crack image using the SIFT algorithm;

[0030] Step 2.3 After the feature extraction module extracts the crack feature information from the collected crack image with the help of SIFT algorithm, the crack recognition module uses the crack image information to identify and detect;

[0031] Step 2.4 calculates the crack width using the crack analysis module;

[0032] Step 2.5: Use the crack prediction module to magnify the cracks, observe the crack growth trend and predict the possible future development trend;

[0033] Step 3: Generate 3D stereoscopic view.

[0034] Preferably, in step S2.1, the image denoising module uses an optimized genetic algorithm to remove noise points in the image, and the specific process includes:

[0035] Step 2.11 Based on the recorded video image, first set the nonlinear filtering denoising data ε s (m,n) is

[0036]

[0037] Step 2.12: Based on the nonlinear filtering denoising data, the improved filtering denoising data is obtained as follows:

[0038]

[0039] Step 2.13: Based on the nonlinear filtering denoising data and the improved filtering denoising data, the denoised image data is obtained as follows:

[0040] ε(m,n)=ε s (m,n)·ε l (m,n)(3)

[0041] Step 2.14 calculates the pixel value of the image after filtering:

[0042]

[0043] In formulas (1) to (4): ε s (m,n) nonlinear filtering denoising data; m, n—image location information to be denoised; s—neighborhood value; λ—filter coefficient; x, y—image edge information; A—specific weight; ε l (m,n)—improved filtering denoising data; l—improved neighborhood value; g(x,y)—image pixel value; B—improved weight; ε(m,n)—denoised image data; f(x,y)—pixel value of the image after filtering; V(x,y)—other neighborhood values ​​of the center point of the image;

[0044] Step 2.15 When f(x,y)=g(x,y), the denoised image is obtained.

[0045] Preferably, in step S2.2, the process of using the feature extraction module to extract crack feature information from the collected crack image by using the SIFT algorithm to enhance the features of interest and suppress the features of no interest includes:

[0046] (1) Use Gaussian differential function to extract potential key points of cracks, and screen these key points again at different scales to ensure that the extracted crack points are scale invariant and are not affected by factors such as lighting, scale, and rotation;

[0047] (2) Determine the direction of the crack feature point by calculating through different spatial operators;

[0048] (3) Match the crack feature points to obtain the corresponding relationship.

[0049] Preferably, in step S2.3, the crack recognition module recognizes and detects the crack image information according to the convolutional neural network, and the specific process includes:

[0050] Step 2.31 trains the YOLO crack detection algorithm model based on the existing crack dataset;

[0051] Step 2.32: The crack video collected by the drone is processed into frames and then input into the crack dataset to be detected;

[0052] Step 2.33 first performs crack feature extraction including convolution, upsampling, downsampling, feature fusion and other operations on the crack data set to be detected to obtain high-level features;

[0053] Step 2.34: The high-level features are convolved once to form convolution module 1 and convolution module 2. Convolution module 1 undergoes two 3*3 and one 1*1 convolution operations to obtain convolution module 11. Convolution module 2 undergoes two 3*3 and two 1*1 convolution operations to obtain convolution module 22. Finally, convolution modules 11 and 22 are stacked to obtain a new feature layer.

[0054] Step 2.35 divides the new feature layer into regions and confuses the domain information through the gradient reversal layer so that the model learns the invariant features of the crack domain;

[0055] Step 2.36 finally predicts the target category and location information, learns the crack target features through back propagation of the loss function, and performs bounding box regression to obtain the crack detection box position;

[0056] Step 2.37 sends the crack detection and identification results to the crack analysis module for further processing.

[0057] Preferably, in step S2.4, the crack analysis module is used to calculate the crack width through an algorithm program according to the identified crack image, and the specific process includes:

[0058] Step 2.41 Grayscale processing of crack image

[0059] Y=0.301×R+0.586×G+0.113×B(5)

[0060] In formula (5), Y represents the final brightness value of the grayscale image, R, G, and B correspond to the color values ​​of the red, green, and blue channels in the color image, respectively, and the range of Y, R, G, and B is 0 to 255;

[0061] Step 2.42 Crack image binarization processing

[0062]

[0063] In formula (6), (x, y) is the pixel coordinate point, T is the threshold, and P(x, y) is the pixel value after binarization;

[0064] Step 2.43 Calculate the percentage of pixels with a grayscale value of 255 in the statistical binary image to the total number of pixels, and then multiply it by the area of ​​the image to calculate the crack area:

[0065]

[0066] In formula (7), S0 is the area of ​​the crack, S is the area of ​​the image, n is the number of crack pixels, and N is the total number of pixels;

[0067] Step 2.44 calculates the ratio of the crack area to the crack length, and then multiplies it by the unit pixel width to get the crack width:

[0068]

[0069] In formula (8), a is the unit pixel width, L is the crack length, and W is the crack width.

[0070] Preferably, the process of predicting cracks using the crack prediction module in step S2.5 is to use a MATLAB program to magnify the cracks to observe the growth trend of the cracks in subtle places and predict possible future development trends.

[0071] Preferably, the process of generating a 3D stereoscopic view in step S3 includes:

[0072] The 3D stereoscopic model view of the bridge is established through the SolidWorks modeling software. At the same time, the crack information is retrieved from the crack information database and embedded in the corresponding node position of the bridge to obtain the 3D stereoscopic view of the bridge.

[0073] The beneficial effects of the present invention are as follows: the present invention discloses a bridge crack identification and detection system and method based on an unmanned aerial vehicle. Compared with the prior art, the improvements of the present invention are as follows:

[0074] 1. The present invention designs a bridge crack identification and detection system based on drones. When in use, the system can plan a reasonable drone flight route according to the bridge crack identification and detection tasks, the bridge parts to be inspected, and the drone parameters; the drone is equipped with an industrial camera to perform aerial operations according to the specified flight route, and the industrial camera records the entire process to collect crack information; when the drone flies in a sunny area, the solar panels it carries can absorb solar energy to increase the working time of the drone; when flying in a shady area, the light supplement module is used to balance the brightness of the insufficient area to achieve uniform light and dark; the collected images are subjected to image denoising and feature extraction processing to improve the subsequent crack identification and detection accuracy; the system identifies and detects cracks in the image through a convolutional neural network, calculates the width parameter information of the cracks using an algorithm program, and predicts the future development trend of the cracks through a deep learning algorithm, and finally generates a 3D stereoscopic model view of the bridge through a modeling software, which fully displays the location information and actual parameter information of the cracks, and can perform targeted maintenance and care on the cracks;

[0075] 2. The present invention designs a detection method for a bridge crack identification and detection system based on an unmanned aerial vehicle. When used, the method can design a reasonable flight route and give full play to the advantages of the unmanned aerial vehicle being flexible, efficient and controllable. According to the different roles played by the unmanned aerial vehicle when working in different sunny and shady flight areas, when flying in the sunny area, the solar panels are used to fully absorb solar energy and convert it to power the unmanned aerial vehicle battery module. When flying in the shady area, in order to ensure that the collected image is clear and visible, the insufficient light module automatically adjusts the brightness according to the brightness to keep the image brightness balanced and the light and dark uniform. On the PC side, a convolutional neural network is used to detect and identify cracks in the video frame by frame to reduce the occurrence of missed detection and false detection. The algorithm program is used to calculate the width parameter information of the crack according to the crack parameters of the video screen, which can be used to quantitatively describe the degree of crack danger. A deep learning algorithm is used to predict the development trend of the crack, which is convenient for subsequent targeted maintenance. Finally, the crack parameter information is embedded in the corresponding node position of the 3D stereo model view of the bridge, which is convenient for browsing, viewing and maintenance. It has the advantages of easy use, comprehensive detection and identification, and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 The present invention is a flowchart of the bridge crack identification and detection system and method based on unmanned aerial vehicle.

[0077] Figure 2 This is a system module diagram of the drone-based bridge crack identification and detection system and method of the present invention.

[0078] Among them: 1. UAV, 2. Obstacle avoidance module, 3. UAV battery module, 4. Solar panel, 5. SD card, 6. Time of flight sensor, 7. GPS positioning module, 8. Camera anti-shake module, 9. Light supplement module, 10. Industrial camera module, 11. Crack, 12. Bridge, 13. Image denoising module, 14. Feature extraction module, 15. Crack recognition module, 16. Crack analysis module, 17. Crack prediction module, 18. Crack information database, 19. PC terminal, 20. Crack information unit, 21. 3D stereo model view, 22. Crack information collection process, 23. Data transmission process, 24. Generate 3D stereo view process. DETAILED DESCRIPTION

[0079] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0080] Embodiment 1: refer to the attached Figure 1-2 The bridge crack identification and detection system based on UAV shown in the figure comprises a UAV 1, a UAV route design and route planning unit, a UAV safe flight unit, an image acquisition and processing unit, a crack identification and analysis unit and a crack information data output unit arranged on the UAV 1, wherein

[0081] The UAV route design and route planning unit is used to generate a reasonable UAV flight route according to the specific bridge detection task, the bridge detection part and the UAV parameter information; the specific UAV route design and route planning unit includes an obstacle avoidance module 2 and a flight time sensor 6. When in use, during the flight of the UAV 1 in the air, the UAV obstacle avoidance module 2 thereon calculates the distance between the UAV 1 and the bridge 12 through the flight time sensor 6 to plan the UAV flight route. At the same time, the flight time sensor 6 emits infrared rays of a certain frequency to the bridge 12, calculates the distance according to the phase difference between the reflected signal and the source signal, and plans the heading route to prevent the UAV from colliding.

[0082] (1) Route design, which is used to preliminarily design the flight route of the UAV according to the bridge structure design drawing and the flight parameter information of the UAV 1;

[0083] (2) Route planning, which is used to optimize the route design based on the specific geographical location and environmental information of the bridge and the weather conditions during operation, and then divide the bridge into different flight areas based on the sunny and shady sides, and plan the final UAV flight route;

[0084] The drone safe flight unit includes an energy module and a GPS positioning module 7, which are used to ensure the flight safety of the drone according to the drone obstacle avoidance module 2 and the energy module, and to obtain the drone position information according to the GPS positioning module 7 to determine the bridge position where the crack is located, wherein the energy module includes a battery module 3 and a solar panel 4, and when the solar panel 4 works in a sunny area, it fully absorbs solar energy to supply the drone battery module 3, and then distributes it to the electrical equipment of the drone 1; wherein the GPS positioning module 7 is used to collect the flight position of the drone 1 in the air in real time to determine the actual position of the bridge 12 where the crack is located;

[0085] The image acquisition and processing unit includes a camera anti-shake module 8, a light compensation module 9, an industrial camera module 10, an image denoising module 13 and a feature extraction module 14. When in use, it is used to ensure that the image of the surface crack 11 of the bridge 12 obtained by the industrial camera module 10, the camera anti-shake module 8 and the light compensation module 9 is clear and visible, and to improve the crack detection accuracy according to the image denoising module 13 and the feature extraction module 14; when the gyroscope in the camera anti-shake module 8 embedded in the industrial camera 10 detects a small shake, it will transmit a signal to the industrial camera microprocessor and calculate the displacement to be compensated, and then compensate it according to the camera shake direction and displacement through the compensation lens group; when the drone 1 is working in a shaded area, the camera 10 in the industrial camera detects that the brightness of the shooting environment is reduced, and sends a command to the light compensation module 9, and the brightness of the low beam group and the high beam group on it ensures that the brightness of the collected image is balanced;

[0086] The crack identification and analysis unit includes a crack identification module 15, a crack analysis module 16, a crack prediction module 17 and a PC terminal 19. When in use, the crack identification module 15 is used to detect and identify cracks frame by frame on the PC terminal 19 according to crack video information, the crack analysis module 16 is used to calculate the width parameter information of the crack 11 according to the MATLAB program algorithm, and the crack prediction module 17 is used to predict the possible development trend of the crack 11;

[0087] The crack information data output unit includes a crack information database 18 and a crack information unit 20, which is used to export effective crack information images based on the video, store the actual crack parameter information and position information in the crack information unit 20 in the crack information database 18, and establish a 3D stereoscopic model view 21 of the bridge through modeling software. At the same time, the crack parameter information is extracted from the crack information database 18 and displayed at the corresponding node position of the bridge, so as to facilitate subsequent repair and maintenance work.

[0088] Preferably, the image acquisition and processing unit further includes an SD card 5 , and the SD card 5 is used to store all video recordings of the crack 11 acquired by the drone 1 , and remotely transmit the video recordings to the PC 19 through a data transmission process 23 .

[0089] The detection method and detection process of the above-mentioned bridge crack identification and detection system based on drone described in this embodiment include:

[0090] Step 1: Crack information collection process 22

[0091] Step 1.1 The drone 1 collects information about the cracks 11 on the surface of the bridge 12 through the industrial camera 8 carried by the drone according to the planned flight route, and stores all the video recordings of the cracks 11 collected in the SD card 5 and transmits them remotely to the PC 19;

[0092] Step 2: Data Transfer Process 23

[0093] Step 2.1 After receiving the video of the crack 11 collected by the drone 1, the PC 19 first uses the image denoising module 13 to use an optimized genetic algorithm to remove noise points in the image to improve the image quality. The denoising process includes:

[0094] Step 2.11 Based on the recorded video image, first set the nonlinear filtering denoising data ε s (m,n) is

[0095]

[0096] Step 2.12: Based on the nonlinear filtering denoising data, the improved filtering denoising data is obtained as follows:

[0097]

[0098] Step 2.13: Based on the nonlinear filtering denoising data and the improved filtering denoising data, the denoised image data is obtained as follows:

[0099] ε(m,n)=ε s (m,n)·ε l (m,n)(3)

[0100] Step 2.14 then calculates the pixel value of the image after filtering:

[0101]

[0102] In formulas (1) to (4): ε s (m,n) nonlinear filtering denoising data; m, n—image location information to be denoised; s—neighborhood value; λ—filter coefficient; x, y—image edge information; A—specific weight; ε l(m,n)—improved filtering denoising data; l—improved neighborhood value; g(x,y)—image pixel value; B—improved weight; ε(m,n)—denoised image data; f(x,y)—pixel value of the image after filtering; V(x,y)—other neighborhood values ​​of the center point of the image;

[0103] Therefore, when f(x, y) = g(x, y), the image denoising effect is stronger.

[0104] Step 2.2 After the image is denoised, the feature extraction module 14 is used to extract crack feature information from the collected crack image using the SIFT algorithm to enhance the features of interest and suppress the features of no interest:

[0105] (1) Use Gaussian differential function to extract potential key points of cracks, and screen these key points again at different scales to ensure that the extracted crack points are scale invariant and are not affected by factors such as lighting, scale, and rotation;

[0106] (2) Determine the direction of the crack feature point and obtain it through calculation using different spatial operators;

[0107] (3) Match the crack feature points to obtain the corresponding relationship;

[0108] Step 2.3: After the feature extraction module 14 extracts the crack feature information from the collected crack image by using the SIFT algorithm, the crack recognition module 15 is used to recognize and detect the crack image information collected by the industrial camera according to the convolutional neural network, including the following steps:

[0109] Step 2.31 trains the YOLO crack detection algorithm model based on the existing crack dataset;

[0110] Step 2.32: The crack video collected by the drone is processed into frames and then input into the crack dataset to be detected;

[0111] Step 2.33 first performs crack feature extraction including convolution, upsampling, downsampling, feature fusion and other operations on the crack data set to be detected to obtain high-level features;

[0112] Step 2.34: The high-level features are convolved once to form convolution module 1 and convolution module 2. Convolution module 1 undergoes two 3*3 and one 1*1 convolution operations to obtain convolution module 11. Convolution module 2 undergoes two 3*3 and two 1*1 convolution operations to obtain convolution module 22. Finally, convolution modules 11 and 22 are stacked to obtain a new feature layer.

[0113] Step 2.35 divides the new feature layer into regions and confuses the domain information through the gradient reversal layer (GRL) so that the model learns the invariant features of the crack domain;

[0114] Step 2.36 finally predicts the target category and location information, learns the crack target features through back propagation of the loss function, and performs bounding box regression to obtain the crack detection box position;

[0115] Step 2.37 sends the crack detection and identification results to the crack analysis module 16 for further processing;

[0116] The crack analysis module 16 described in step 2.4 is used to calculate the crack width through an algorithm program according to the identified crack image, and includes the following steps:

[0117] Step 2.41 Grayscale processing of crack image

[0118] Y=0.301×R+0.586×G+0.113×B(5)

[0119] In formula (5), Y represents the final brightness value of the grayscale image, R, G, and B correspond to the color values ​​of the red, green, and blue channels in the color image, respectively, and the range of Y, R, G, and B is 0 to 255;

[0120] Step 2.42 Crack image binarization processing

[0121]

[0122] In formula (6), (x, y) is the pixel coordinate point, T is the threshold, and P(x, y) is the pixel value after binarization;

[0123] Step 2.43 calculates the percentage of pixels with a gray value of 255 in the statistical binary image to the total number of pixels, and then multiplies it by the area of ​​the image to calculate the crack area. The specific formula is as follows:

[0124]

[0125] In formula (7), S0 is the area of ​​the crack, S is the area of ​​the image, n is the number of crack pixels, and N is the total number of pixels;

[0126] Step 2.44 calculates the ratio of the crack area to the crack length, and then multiplies it by the unit pixel width to get the crack width:

[0127]

[0128] In formula (8), a is the unit pixel width, L is the crack length, and W is the crack width;

[0129] Step 2.5 The crack prediction module 17 is used to magnify the cracks based on the identified crack images using the MATLAB program to observe the crack growth trend in the subtle parts and predict the possible future development trend;

[0130] Step 3: Generate 3D stereoscopic view process 24

[0131] A 3D stereoscopic model view 21 of the bridge is established by using the modeling software SolidWorks, and crack information is retrieved from the crack information database 18 and embedded in the corresponding node position of the bridge to facilitate subsequent repair and maintenance work.

[0132] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A detection method for a bridge crack identification and detection system based on an unmanned aerial vehicle, characterized in that: The UAV-based bridge crack identification and detection system includes a UAV, a UAV route design and route planning unit, a UAV safe flight unit, an image acquisition and processing unit, a crack identification and analysis unit, and a crack information data output unit; The UAV route design and route planning unit is used to generate a UAV flight route according to the bridge inspection task, the bridge inspection location and the UAV parameter information; The UAV safe flight unit is used to ensure the flight safety of the UAV according to the UAV obstacle avoidance module and the energy module, and at the same time obtain the UAV location information according to the GPS positioning module to determine the crack location; The image acquisition and processing unit is used to acquire bridge surface crack images and extract crack features; The crack identification and analysis unit is used to detect and identify cracks frame by frame on the PC side according to the crack video information, calculate the crack width parameter information, and predict the crack development trend; The crack information data output unit is used to export effective crack information pictures, store actual crack parameter information and position information in a crack information database, establish a 3D model view of the bridge, and extract crack parameter information from the crack information database and display it at the node position corresponding to the bridge; The detection method comprises: Step 1: Crack information collection process Step 1.1 The drone collects crack information on the bridge surface through the industrial camera carried according to the planned flight route, and stores all the video recordings of the cracks collected in the SD card and transmits them remotely to the PC; Step 2: Data Transfer Process Step 2.1 After receiving the video of the cracks collected by the drone, the PC first uses the image denoising module to remove the noise in the image; In step S2.1, the image denoising module uses an optimized genetic algorithm to remove noise points in the image. The specific process includes: Step 2.11 Based on the recorded video image, first set the nonlinear filtering denoising data ε s (m,n) is Step 2.12: Based on the nonlinear filtering denoising data, the improved filtering denoising data is obtained as follows: Step 2.13: Based on the nonlinear filtering denoising data and the improved filtering denoising data, the denoised image data is obtained as follows: ε(m,n)=ε s (m,n)·e l (m,n)(3) Step 2.14 calculates the pixel value of the image after filtering: In formulas (1) to (4): ε s (m,n) nonlinear filtering denoising data; m, n—image location information to be denoised; s—neighborhood value; λ—filter coefficient; x, y—image edge information; A—specific weight; ε l (m,n)—improved filtering denoising data; l—improved neighborhood value; g(x,y)—image pixel value; B—improved weight; ε(m,n)—denoised image data; f(x,y)—pixel value of the image after filtering; V(x,y)—other neighborhood values ​​of the center point of the image; Step 2.15 When f(x,y) = g(x,y), the denoised image is obtained; Step 2.2 After denoising the image, the feature extraction module is used to extract crack feature information from the collected crack image using the SIFT algorithm; Step 2.3 After the feature extraction module extracts the crack feature information from the collected crack image with the help of SIFT algorithm, the crack recognition module uses the crack image information to identify and detect; In step S2.3, the crack recognition module recognizes and detects the crack image information based on the convolutional neural network. The specific process includes: Step 2.31 trains the YOLO crack detection algorithm model based on the existing crack dataset; Step 2.32: The crack video collected by the drone is processed into frames and then input into the crack dataset to be detected; Step 2.33 first performs crack feature extraction including convolution, upsampling, downsampling, feature fusion and other operations on the crack data set to be detected to obtain high-level features; Step 2.34: The high-level features are convolved once to form convolution module 1 and convolution module 2. Convolution module 1 undergoes two 3*3 and one 1*1 convolution operations to obtain convolution module 11. Convolution module 2 undergoes two 3*3 and two 1*1 convolution operations to obtain convolution module 22. Finally, convolution modules 11 and 22 are stacked to obtain a new feature layer. Step 2.35 divides the new feature layer into regions and confuses the domain information through the gradient reversal layer so that the model learns the invariant features of the crack domain; Step 2.36 finally predicts the target category and location information, learns the crack target features through back propagation of the loss function, and performs bounding box regression to obtain the crack detection box position; Step 2.37 sends the crack detection and identification results to the crack analysis module for further processing; Step 2.4 uses the crack analysis module to calculate the crack width; the crack analysis module is used to calculate the crack width through an algorithm program based on the identified crack image, and the specific process includes: Step 2.41 Grayscale processing of crack image Y=0.301×R+0.586×G+0.113×B(5) In formula (5), Y represents the final brightness value of the grayscale image, R, G, and B correspond to the color values ​​of the red, green, and blue channels in the color image, respectively, and the range of Y, R, G, and B is 0 to 255; Step 2.42 Crack image binarization processing In formula (6), (x, y) is the pixel coordinate point, T is the threshold, and P(x, y) is the pixel value after binarization; Step 2.43 Calculate the percentage of pixels with a grayscale value of 255 in the statistical binary image to the total number of pixels, and then multiply it by the area of ​​the image to calculate the crack area: In formula (7), S0 is the area of ​​the crack, S is the area of ​​the image, n is the number of crack pixels, and N is the total number of pixels; Step 2.44 calculates the ratio of the crack area to the crack length, and then multiplies it by the unit pixel width to get the crack width: In formula (8), a is the unit pixel width, L is the crack length, and W is the crack width; Step 2.5: Use the crack prediction module to magnify the cracks, observe the crack growth trend and predict the possible future development trend; Step 3: Generate 3D stereoscopic view.

2. The detection method of a bridge crack identification and detection system based on an unmanned aerial vehicle according to claim 1 is characterized in that: The image acquisition and processing unit includes a camera anti-shake module, a light supplement module, an industrial camera module, an image denoising module and a feature extraction module. The camera anti-shake module is used to reduce the blur of the video image caused by external environmental interference; The light supplement module is used to ensure that there is sufficient light when collecting pictures of the shaded side of the bridge, and to balance the shooting brightness; The industrial camera module is used for collecting high-resolution video recordings; The image denoising module is used to perform denoising on the collected crack image information; The feature extraction module is used to extract feature information from the collected crack image information.

3. The detection method of a bridge crack identification and detection system based on an unmanned aerial vehicle according to claim 2 is characterized in that: The crack identification and analysis unit includes a crack identification module, a crack analysis module, a crack prediction module and a PC terminal. The crack recognition module is used to recognize and detect crack image information collected by the industrial camera on the PC side according to the convolutional neural network; The crack analysis module is used to calculate the width of the crack according to the identified crack image through the MATLAB algorithm program; The crack prediction module is used to magnify the cracks based on the identified crack images using a MATLAB program to observe the crack growth trends at subtle locations and predict future development trends.

4. The detection method of the bridge crack identification and detection system based on an unmanned aerial vehicle according to claim 3 is characterized in that: In step S2.2, the feature extraction module is used to extract crack feature information from the collected crack image with the help of SIFT algorithm, and the process of enhancing the features of interest and suppressing the features of no interest includes: (1) Use Gaussian differential function to extract potential key points of cracks, and screen these key points again at different scales to ensure that the extracted crack points are scale invariant and are not affected by factors such as lighting, scale, and rotation; (2) Determine the direction of the crack feature point by calculating through different spatial operators; (3) Match the crack feature points to obtain the corresponding relationship.

5. The detection method of the bridge crack identification and detection system based on an unmanned aerial vehicle according to claim 4 is characterized in that: The process of predicting cracks using the crack prediction module described in step S2.5 is to use the MATLAB program to magnify the cracks to observe the growth trend of the cracks in the subtle parts and predict the possible development trend in the future.

6. The detection method of the bridge crack identification and detection system based on an unmanned aerial vehicle according to claim 5 is characterized in that: The process of generating a 3D stereoscopic view described in step S3 includes establishing a 3D stereoscopic model view of the bridge through SolidWorks modeling software, and at the same time retrieving crack information from a crack information database and embedding it in the corresponding node position of the bridge to obtain a 3D stereoscopic view of the bridge.

Images