A Smart Detection Method for Dam Cracks Based on UAV Visual Perception and Deep Learning

Abstract

This invention discloses an intelligent method for detecting dam cracks based on UAV visual perception and deep learning, comprising the following steps: Step 1, using a UAV to photograph the dam and acquire images of the dam surface area; Step 2, identifying and locating dam cracks; Step 3, calculating dam crack size information; Step 4, calculating crack size information. The method disclosed in this invention utilizes a UAV to acquire image data of the dam surface, and uses deep learning and image processing technology to identify, locate, and quantify dam cracks, enabling rapid, automatic, and intelligent dam crack detection. The Mask R-CNN algorithm based on an improved FPN is used for dam crack identification, which enhances the ability to identify small cracks and improves crack detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of health monitoring and damage identification of dam structures, and specifically relates to an intelligent detection method for dam cracks based on UAV visual perception and deep learning. Background Technology

[0002] Cracks are a common defect in large-volume concrete structures such as reservoirs and dams in my country. Their presence affects the integrity, durability, and seismic performance of the dam structure, significantly reducing its quality. Therefore, regular inspection of dam defects, especially cracks, is necessary. However, due to the characteristics of dam cracks—small width, poor visibility, and indistinct identification features—achieving rapid, stable, and accurate dam crack detection remains a highly challenging problem.

[0003] The earliest and most common method for detecting cracks in dams relied on manual observation. However, this method is time-consuming, inefficient, and easily affected by subjective human factors, making rapid large-scale dam inspection impossible. With technological advancements, the emergence of unmanned aerial vehicle (UAV) intelligent inspection technology has provided a promising approach for detecting surface cracks in dams. As an emerging data acquisition method, UAVs can be equipped with various types of sensors to meet different types of crack detection requirements. They can easily acquire data that is difficult for surveyors to obtain and can transmit captured images to ground or cloud systems. Combined with advanced digital image processing algorithms, this enables efficient and accurate location and identification of dam crack information.

[0004] Currently, the use of drones as carriers and image recognition technology for rapid identification of dam cracks has begun to enter the engineering application stage. However, interference from factors such as human-machine shake, external interference, and complex backgrounds of the target object often results in problems such as image blurring, low regional resolution, and insufficient crack pixels, leading to low accuracy in small target identification and difficulty in accurately identifying, locating, and quantifying crack locations. Although researchers have applied deep learning (DNN, CNN, Faster RCNN networks) to the field of dam crack detection and identification, achieving significant breakthroughs, the accuracy in detecting small targets remains unsatisfactory. Moreover, automation and intelligence have not been achieved in crack localization and quantification. Therefore, there is an urgent need to research an intelligent drone detection method suitable for rapid dam crack detection. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide an intelligent detection method for dam cracks based on UAV visual perception and deep learning, which can improve the recognition accuracy of small cracks in dams and calculate the size information of cracks, thereby achieving fast, accurate and intelligent dam crack detection.

[0006] The present invention adopts the following technical solution:

[0007] An intelligent detection method for dam cracks based on UAV visual perception and deep learning is improved by including the following steps:

[0008] Step 1: Use a drone to photograph the dam and collect images of the dam's surface area;

[0009] Step 2, Identification and location of cracks in the dam:

[0010] First, preprocessing is performed: the acquired images are enhanced and data annotation and mask creation are performed using the LabelMe annotation tool;

[0011] Secondly, the Mask R-CNN algorithm based on the improved FPN is used for dam crack identification:

[0012] Input the image to be detected, and use the ResNet-101 network to extract the depth features {C2,C3,C4,C5} of the image;

[0013] An improved FPN feature pyramid network is adopted to fuse the feature map generated by ResNet-101 with the low-level feature information, and output the top-down fused feature map {P2,P3,P4,P5} and the bottom-up fused feature map {T2,T3,T4,T5}.

[0014] The top-down fused feature maps {P2,P3,P4,P5} and the bottom-up fused feature maps {T2,T3,T4,T5} are fused to obtain multi-scale multi-information feature maps {Q2,Q3,Q4,Q5}, which are then input into the RPN region generation network to generate proposal boxes about the detected targets.

[0015] The RoI Align network is used to perform alignment operations on the generated suggestion boxes;

[0016] A fully convolutional network is used to perform classification, boundary regression, and mask segmentation operations on the aligned proposal boxes;

[0017] Output the crack detection results, target detection boxes, and target masks for the acquired images;

[0018] Finally, the exact location of the crack on the dam surface was determined.

[0019] Step 3, Calculation of dam crack size information:

[0020] First, the color image containing the crack is converted to a grayscale image to obtain the initial grayscale image of the crack, with the image pixel grayscale value set {x1, x2, ..., x...}. N}, where N is the number of samples;

[0021] Secondly, the K-means clustering algorithm and the region growing algorithm are used to accurately segment the crack grayscale image:

[0022] The crack grayscale image is divided into three categories based on pixel features: background region, noise region, and crack region. Therefore, the K-cluster value is initialized to 3. The distance from each pixel in the crack grayscale image to the preset initial center {u1,u2,…,u} is calculated using the following formula. K The distance between cluster centers is used to dynamically adjust the cluster centers. This process is iterated until the clustering results converge, thus obtaining the optimal clustering result.

[0023]

[0024] In the above formula, x l For the l-th sample data, u k Let J be the cluster center of the k-th cluster, and J represent the x-th cluster in the sample data. l with u k The sum of the distances between them;

[0025] When using the region growing algorithm to accurately segment a crack grayscale image, the point corresponding to the grayscale value of the target cluster center pixel obtained by the K-means clustering algorithm is used as the initial target point. The change in size between the target point and its eight neighboring grayscale pixels is used as the evaluation criterion for growth. Assuming the size of the crack grayscale image is M×N, and G(i,j) represents the pixel grayscale value of the point at coordinate (i,j), a scalar F is assigned to the point (i,j). i,j Then the crack growth criterion S is:

[0026] S(G(i,j),G(i±1,j±1))≤T

[0027] In the above formula, G(i±1,j±1) represents the pixel gray value of any point within the eight neighborhoods of point (i,j), S(G(i,j),G(i±1,j±1)) represents the similarity between the pixel gray values ​​of point (i,j) and its eight neighboring points, and T is a threshold with a value of 10. If the above formula is satisfied, then F i,j =1, indicating that point (i,j) belongs to the crack region; otherwise F i,j =0, indicating that point (i,j) does not belong to the crack region;

[0028] Then, the crack regions in the segmented image are segmented and labeled according to their width;

[0029] Finally, the crack size information is calculated:

[0030] Crack area P a For: P a =λ 2 ·P v In the above formula, P vλ is the pixel size of the crack, and λ is the scaling factor of the crack in the image;

[0031] Crack length: The crack skeleton image is thinned, and the width of the thinned crack is a single pixel. The number of pixels in the crack is obtained by calculating half of the perimeter at this time. The crack length is obtained based on the actual physical size of the pixels.

[0032] The average width of a crack is the ratio of its area to its length, calculated using the following formula:

[0033] d e =2S e / C e ,1≤e≤n

[0034] In the above formula, S e Let C be the area of ​​the e-th crack. e Let d be the perimeter of the e-th crack. e Let be the average width of the e-th crack, and n be the total number of cracks.

[0035] Furthermore, step 1 includes:

[0036] First, design the observation route, including the flight range, number of flights, and the division of the sunlit and shaded areas of the dam;

[0037] Secondly, preset condition parameters are set, including shooting angle, overlap, pixel calibration, offset distance, and flight mode;

[0038] Finally, image data acquisition and transmission.

[0039] Furthermore, the improved FPN feature pyramid network in step 2 employs a TAM module, which has two inputs: the low-level feature map t z and high-level feature map t z+2 If z = 2 and 3, after processing by the module, the outputs correspond to T respectively. z and T z+2 The low-level output feature map T z Defined as:

[0040]

[0041] In the above formula, t d Indicates the lower level t z Enhanced features, t d The definition of is:

[0042] t d =Sigmoid(t) cat ⊙Attf)

[0043] In the above formula, Sigmoid represents the activation function, t catThe cascaded features are obtained by aggregating the outputs of dilated convolutions with different dilation rates, where r = 1, 2, 3. Attf represents the attention score, defined as:

[0044] Attf = Sigmoid(W2×δ(W1×P)) avg (t cat )))

[0045] In the above formula, W1 and W2 represent the weight coefficients of the first and second 1×1 convolutional layers, respectively, and δ(·) and P avg (·) represent the ReLU activation function and the global average pooling operation, respectively;

[0046] t s Represents attention mapping, t s The definition of is:

[0047] t s =Sigmoid(W3×U(t) z+2 ))

[0048] In the above formula, W3 represents the weight coefficient of the 1×1 convolutional layer, and U(·) represents upsampling;

[0049] High-level output feature map T z+2 Defined as:

[0050]

[0051] In the above formula, D represents the downsampling operation.

[0052] The beneficial effects of this invention are:

[0053] The method disclosed in this invention utilizes drones to collect image data of the dam surface, and employs deep learning and image processing technologies to identify, locate, and quantify dam cracks, enabling rapid, automatic, and intelligent dam crack detection. The Mask R-CNN algorithm based on an improved FPN is used for dam crack identification, enhancing the ability to identify small cracks and improving crack detection accuracy. The K-means clustering algorithm is used for crack region segmentation, which can finely characterize the scale information of the cracks, providing a basis for inspectors to assess the severity of dam crack damage. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the method of the present invention;

[0055] Figure 2 This is a schematic diagram of the improved FPN network structure;

[0056] Figure 3 This is a structural diagram of the TAM module;

[0057] Figure 4 This is a network structure diagram of the Mask R-CNN algorithm based on the improved FPN;

[0058] Figure 5 This is a schematic diagram illustrating the transformation relationship between image coordinates and world coordinates;

[0059] Figure 6(a) shows crack identification;

[0060] Figure 6(b) shows the segmentation of the crack region;

[0061] Figure 7(a) is an example of segmentation marking of a crack region;

[0062] Figure 7(b) is another example of segmentation identification of crack regions. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0064] This invention discloses an intelligent detection method for dam cracks based on UAV visual perception and deep learning, such as... Figure 1 As shown, it includes the following steps:

[0065] Step 1: Use a drone to photograph the dam and collect images of the dam's surface area;

[0066] After the drone fuselage is assembled, the first step is to design the observation route. Based on the specific conditions of the site environment, the drone route is designed reasonably, including the flight range, number of flights, and the division of the sunny and shady areas of the embankment.

[0067] Secondly, preset condition parameters are set, including shooting angle, overlap, pixel calibration, offset distance, and flight mode;

[0068] Finally, image data acquisition and transmission. Considering shooting distance, lighting, and overlap, the drone uses video recording and fixed-point reshooting functions to achieve precise imaging of the embankment. After the drone completes its imaging, the data needs to be promptly aggregated into an identification database and transmitted to a computer via the network. Furthermore, the effectiveness of the captured image quality needs to be monitored; the criteria for determination are as follows:

[0069]

[0070]

[0071] In the above formula, E Ip represents the sum of the standard deviations of the intensity of each pixel in the grayscale image, where I represents the grayscale input image. i S represents the probability that gray value i appears in a grayscale image. I This represents the image sharpness of a grayscale image, where P is the total number of pixels in the image, dI is the grayscale change magnitude, and dx is the distance increment between pixels.

[0072] Step 2, Identification and location of cracks in the dam:

[0073] First, preprocessing is performed: the LabelMe annotation tool is used for data annotation and mask creation. In addition, to improve the model's detection accuracy, the training samples need to be augmented. Data augmentation methods include image rotation and translation, translation transformation, scaling transformation, contrast transformation, random cropping, color dithering, and noise perturbation.

[0074] Secondly, the Mask R-CNN algorithm based on the improved FPN is used for dam crack identification:

[0075] To address the challenge of accurate detection of dam cracks due to their diversity and randomness, this invention proposes a Mask R-CNN dam crack recognition algorithm based on an improved Feature Processing Network (FPN). This algorithm adds a bottom-up backward connection path to the traditional FPN network, achieving more comprehensive image information extraction through complete fusion of feature maps from each stage. This provides the Mask R-CNN network with multi-scale, multi-information feature maps possessing stronger semantic and localization information. The improved FPN network is shown below. Figure 2 As shown.

[0076] Furthermore, to improve the model's performance in detecting small-scale objects, a Texture Attention Module (TAM) was designed. This module dynamically enhances texture information by reducing redundant semantics to emphasize small-scale features in lower layers and highlights large-scale features in higher layers by adding believable details. This ensures that the network can accurately detect large-scale objects while improving the model's performance in detecting small-scale objects.

[0077] Specifically, this invention designs a TAM module to fuse t z and t z+2 The feature (z = 2, 3) aims to remove the low-level t z The features of large-scale objects in the model are used to highlight the feature information of small-scale objects; at the same time, at the high-level t z+2 Enhance the believable details of small-scale objects to supplement the features of large-scale objects, such as... Figure 3 As shown, this TAM module has two inputs: the low-level feature map t z and high-level feature map t z+2If z = 2 and 3, after processing by the module, the outputs correspond to T respectively. z and T z+2 The low-level output feature map T z Defined as:

[0078]

[0079] In the above formula, t d Indicates the lower level t z Enhanced features are achieved by dynamically utilizing dilated convolutions with different dilation rates to accurately represent the feature information of small-scale objects. d The definition of is:

[0080] t d =Sigmoid(t) cat ⊙Attf)

[0081] In the above formula, Sigmoid represents the activation function, t cat The cascaded features are obtained by aggregating the outputs of dilated convolutions with different dilation rates, r = 1, 2, 3. Attf represents the attention score, used to measure the importance of channels, and is defined as follows:

[0082] Attf = Sigmoid(W2×δ(W1×P)) avg (t cat )))

[0083] In the above formula, W1 and W2 represent the weight coefficients of the first and second 1×1 convolutional layers, respectively, and δ(·) and P avg (·) represent the ReLU activation function and the global average pooling operation, respectively;

[0084] t s This represents an attention mapping that suppresses distracting low-resolution features within a high-resolution feature set rich in information.

[0085] t s The definition of is:

[0086] t s =Sigmoid(W3×U(t) z+2 ))

[0087] In the above formula, W3 represents the weight coefficient of the 1×1 convolutional layer, and U(·) represents upsampling;

[0088] High-level output feature map T z+2 Defined as:

[0089]

[0090] In the above formula, D represents the downsampling operation, used to maintain the relationship with t.z+2 Consistent spatial resolution. The improved FPN feature pyramid network outputs top-down fused feature maps {P2,P3,P4,P5} and bottom-up fused feature maps {T2,T3,T4,T5} after executing the TAM module.

[0091] The process of identifying dam cracks using the Mask R-CNN algorithm based on improved FPN is as follows: Figure 4 As shown,

[0092] Input the image to be detected, and use the ResNet-101 network to extract the depth features {C2,C3,C4,C5} of the image;

[0093] An improved FPN feature pyramid network is adopted to fuse the feature map generated by ResNet-101 with the low-level feature information, and output the top-down fused feature map {P2,P3,P4,P5} and the bottom-up fused feature map {T2,T3,T4,T5}.

[0094] The top-down fused feature maps {P2,P3,P4,P5} and the bottom-up fused feature maps {T2,T3,T4,T5} are fused to obtain multi-scale multi-information feature maps {Q2,Q3,Q4,Q5}, which are then input into the RPN region generation network to generate proposal boxes about the detected targets.

[0095] The RoI Align network is used to perform alignment operations on the generated suggestion boxes;

[0096] A fully convolutional network is used to perform classification, boundary regression, and mask segmentation operations on the aligned proposal boxes;

[0097] Output the crack detection results of the image acquired for this location area (as shown in Figure 6(a)), the target detection box, and the target mask;

[0098] Finally, the exact location of the crack on the dam surface was determined:

[0099] When using drones for aerial photography, it's necessary to obtain the specific area on the dam in the captured image. Taking a planar dam as an example, a three-dimensional coordinate system is established parallel to the downstream dam surface. The y-axis and z-axis are parallel to the horizontal and vertical directions of the downstream dam surface, respectively; that is, the yoz plane is parallel to the downstream dam surface, and the x-direction points from the downstream of the dam to the upstream of the dam. Figure 5 As shown, during aerial photography by a multi-rotor UAV, high-precision self-positioning coordinates P0(x0,y0,z0) can be acquired in real time, and the coordinate information is embedded into the aerial image in real time. If the vertical distance from the UAV to the downstream dam surface is d, then the center coordinates of the captured image are P1(x0+d,y0,z0), thereby achieving center positioning of the aerial image and determining the specific location of the crack on the dam surface.

[0100] Step 3: Calculate the crack size information of the dam. Digital image processing technology is used to post-process the image containing the cracks to calculate the crack size information. This includes image transformation, crack region segmentation, secondary crack identification, and crack size calculation.

[0101] First, the color image containing the crack is converted to a grayscale image to obtain the initial grayscale image of the crack, with the image pixel grayscale value set {x1, x2, ..., x...}. N}, where N is the number of samples;

[0102] Secondly, as shown in Figure 6(b), the K-means clustering algorithm and the region growing algorithm are used to accurately segment the crack grayscale image:

[0103] The crack grayscale image is divided into three categories based on pixel features: background region, noise region, and crack region. Therefore, the K-cluster value is initialized to 3. The distance from each pixel in the crack grayscale image to the preset initial center {u1,u2,…,u} is calculated using the following formula. K The distance between cluster centers is used to dynamically adjust the cluster centers. This process is iterated until the clustering results converge, thus obtaining the optimal clustering result.

[0104]

[0105] In the above formula, x l For the l-th sample data, u k Let J be the cluster center of the k-th cluster, and J represent the x-th cluster in the sample data. l with u k The cumulative distance between them; K-means clustering initially divides the crack class and obtains the gray value of the cluster center pixel, which has a very high gray value similarity with the crack features, providing target points for the region growing algorithm.

[0106] When using the region growing algorithm to accurately segment a crack grayscale image, the point corresponding to the grayscale value of the target cluster center pixel obtained by the K-means clustering algorithm is used as the initial target point. The change in size between the target point and its eight neighboring grayscale pixels is used as the evaluation criterion for growth. Assuming the size of the crack grayscale image is M×N, and G(i,j) represents the pixel grayscale value of the point at coordinate (i,j), a scalar F is assigned to the point (i,j). i,j Then the crack growth criterion S is:

[0107] S(G(i,j),G(i±1,j±1))≤T

[0108] In the above formula, G(i±1,j±1) represents the pixel gray value of any point within the eight neighborhoods of point (i,j), S(G(i,j),G(i±1,j±1)) represents the similarity between the pixel gray values ​​of point (i,j) and its eight neighboring points, and T is a threshold with a value of 10. This algorithm refers to the gray value difference between two pixels. If the above formula is satisfied, then F i,j =1, indicating that point (i,j) belongs to the crack region; otherwise F i,j =0, indicating that point (i,j) does not belong to the crack region;

[0109] Then, to facilitate the calculation of crack size, the crack image is re-labeled. Based on the crack identification in step 2, and considering that the width of each crack segment is different, the crack region in the segmented image is divided into segments according to the width.

[0110] Finally, the crack size information is calculated:

[0111] The crack area needs to be calculated by determining the number of pixels contained in each connected region, then obtaining the area of ​​each connected region based on the actual physical size corresponding to each pixel, and finally summing the areas of all connected regions.

[0112] Crack area P a For: P a =λ 2 ·P v In the above formula, P a The unit is mm. 2 P v The pixel size of the crack, in pixels. 2 λ is the scaling factor of the crack in the image, in mm·pixel. -1 ;

[0113] Crack length: The crack skeleton image is thinned, and the width of the thinned crack is a single pixel. The number of pixels in the crack is obtained by calculating half of the perimeter at this time. The crack length is obtained based on the actual physical size of the pixels.

[0114] The average width of a crack is the ratio of its area to its length, calculated using the following formula:

[0115] d e =2S e / C e ,1≤e≤n

[0116] In the above formula, S e Let C be the area of ​​the e-th crack. e Let d be the perimeter of the e-th crack. e Let be the average width of the e-th crack, and n be the total number of cracks.

[0117] Example 1 discloses an example of an intelligent detection method for dam cracks based on UAV visual perception and deep learning, including the following steps:

[0118] Step 1: Use a drone to photograph the dam, collecting images of its surface area. The drone photography process includes steps such as drone assembly, observation flight path design, setting preset parameters, and image data acquisition and transmission.

[0119] (1) UAV fuselage assembly. This embodiment uses a DJI M210-RTK UAV, equipped with a Zenmuse X5S gimbal high-definition camera. In addition, it is also equipped with a gyroscope sensor, GPS positioning system, intelligent navigation system and ultrasonic rangefinder, etc.

[0120] (2) Observation route design. First, the exploration area of ​​the dike was determined through dike drawings and on-site reconnaissance. Then, the dike was divided into equal areas, and the center coordinates of each area were marked. Finally, these coordinates were transmitted to the UAV for path planning so that it could take pictures at that point.

[0121] (3) Setting Preset Conditions and Parameters. When capturing image data, the UAV needs to maintain a perpendicular position between the camera lens and the surface of the embankment. Therefore, the camera's shooting angle should be adjusted appropriately based on the slope angle of the embankment. To ensure that the shooting resolution and crack identification meet sub-millimeter requirements, the offset distance between the UAV and the embankment is fixed at 5–10 m, and the focal length is set to 15 mm. The UAV uses GPS mode and cruises along an "S"-shaped flight path for photography, with an overlap greater than 60%.

[0122] (4) Image data acquisition and transmission. Taking into account shooting distance, lighting, and overlap, the UAV achieves accurate imaging of the embankment through functions such as video recording and fixed-point reshooting. After the UAV completes the shooting, it needs to be promptly aggregated into an identification database via the network and transmitted to the computer.

[0123] Step 2, Identification and location of cracks in the dam:

[0124] (1) Image Classification. The acquired crack images included four categories: crack-free, transverse cracks, longitudinal cracks, and network cracks, totaling 12,860 images. Among them, there were 5,000 crack-free images, 2,210 images of transverse cracks, 2,560 images of longitudinal cracks, and 3,090 images of network cracks. Each image was 1280×960 pixels. The LabelMe annotation tool was used to annotate the images.

[0125] (2) Data preprocessing. To improve the detection accuracy of the model, the acquired image samples are enhanced. The enhancement methods include image rotation and translation, translation transformation, scaling transformation, contrast transformation, random cropping, color dithering, and noise perturbation.

[0126] (3) Training the model. First, the labeled images are used as the training set and input into the model. Second, the initial parameters of the model are set. The weights and biases adopt the default values ​​of the network. The initial learning rate is set to 0.004, the weight decay coefficient is set to 0.0001, and the batch size is set to 10. Then, the Adam optimizer is used to train the Mask R-CNN model based on the improved FPN. The optimal weights of the model are obtained by continuously updating the weights and biases.

[0127] (4) Crack Identification. The trained model is used to identify cracks in the test images. To intuitively evaluate the crack identification performance of the Mask R-CNN model based on the improved FPN, this invention uses precision and recall as evaluation metrics, defined as follows:

[0128]

[0129]

[0130] The threshold for the Intersection over Union (IoU) ratio between the mask and the actual crack region is set to 0.5. If the IoU exceeds the threshold, the identification result is considered a True Positive (TP); otherwise, the result is considered a False Negative (FN). If a mask is generated even though the image does not contain a crack, the result is considered a False Positive (FP); if the image does not contain a crack and no mask is generated, the result is considered a True Negative (TN).

[0131] The crack identification results of different methods are shown in the table below:

[0132] Crack identification results of different algorithms

[0133] algorithm Precision / % Recall / % Faster R-CNN 93.65 92.16 Mask R-CNN 97.52 96.39 Mask R-CNN with Improved FPN 98.86 98.27

[0134] As can be seen from the table above, this embodiment improves the FPN to ensure that the model extracts candidate regions on feature maps of different scales with more positioning information, reduces the introduction of quantization operation errors, and thus improves the model's ability to identify cracks.

[0135] (5) Crack location. Extract the center coordinates P2(x2,y2,z2) of the crack recognition model output image. Then, based on the vertical distance d between the UAV and the surface, the specific coordinates P3(x2-d,y2,z2) of the crack location area can be located.

[0136] Step 3, Calculation of dam crack size information:

[0137] (1) First, convert the color crack image to a grayscale image.

[0138] (2) Secondly, the K-means clustering algorithm and the region growing algorithm are used to accurately segment the crack grayscale image.

[0139] (3) Then, the image after the crack region is segmented is labeled.

[0140] (4) Finally, the crack size information is calculated. In this embodiment, an Olympus Zuiko digital 15mm fixed-focus lens is used. The zoom ratio of this camera in image capture is 0.21mm·pixel-1. Figure 7(a) , 7(b) Two crack examples are given. Due to the varying widths of each crack segment, the cracks are segmented. Crack I in Figure 7(a) contains 3 segment markers, and crack II in Figure 7(b) contains 8 segment markers. The widths of crack I in segments 1, 2, and 3 are 5, 5, and 4 pixels respectively, with an average width of 4.67 pixels. The lengths are 328, 907, and 156 pixels, with a total length of 1391 pixels. The widths of crack II in the 8 segments are 4, 4, 4, 5, 6, 6, 5, and 3 pixels respectively, with an average width of 4.63 pixels. The lengths are 108, 65, 96, 152, 166, 287, 255, and 71 pixels, with a total length of 1200 pixels. Based on the pixel dimensions of each segment of crack I and crack II and the image scaling ratio, the actual crack dimensions of crack I and crack II can be calculated, and the results are shown in the table below. As can be seen from the table below, the crack size calculated by this invention is not much different from the actual size, and it has extremely high practicality.

[0141] Comparison of calculated and actual values ​​for cracks I and II

[0142]

Claims

1. A method for intelligent detection of dam cracks based on UAV visual perception and deep learning, characterized in that, Includes the following steps: Step 1: Use a drone to photograph the dam and collect images of the dam's surface area; Step 2, Identification and location of cracks in the dam: First, preprocessing is performed: the acquired images are enhanced and data annotation and mask creation are performed using the LabelMe annotation tool; Secondly, the Mask R-CNN algorithm based on the improved FPN is used for dam crack identification: Input the image to be detected, and use the ResNet-101 network to extract the depth features {C2,C3,C4,C5} of the image; An improved FPN feature pyramid network is adopted to fuse the feature map generated by ResNet-101 with the low-level feature information, and output the top-down fused feature map {P2,P3,P4,P5} and the bottom-up fused feature map {T2,T3,T4,T5}. The top-down fused feature maps {P2,P3,P4,P5} and the bottom-up fused feature maps {T2,T3,T4,T5} are fused to obtain multi-scale multi-information feature maps {Q2,Q3,Q4,Q5}, which are then input into the RPN region generation network to generate proposal boxes about the detected targets. The RoI Align network is used to perform alignment operations on the generated suggestion boxes; A fully convolutional network is used to perform classification, boundary regression, and mask segmentation operations on the aligned proposal boxes; Output the crack detection results, target detection boxes, and target masks for the acquired images; Finally, the exact location of the crack on the dam surface was determined. Step 3, Calculation of dam crack size information: First, the color image containing the crack is converted to a grayscale image to obtain the initial grayscale image of the crack, with the image pixel grayscale value set {x1, x2, ..., x...}. N }, where N is the number of samples; Secondly, the K-means clustering algorithm and the region growing algorithm are used to accurately segment the crack grayscale image: The crack grayscale image is divided into three categories based on pixel features: background region, noise region, and crack region. Therefore, the K-cluster value is initialized to 3. The distance from each pixel in the crack grayscale image to the preset initial center {u1,u2,…,u} is calculated using the following formula. K The distance between cluster centers is used to dynamically adjust the cluster centers. This process is iterated until the clustering results converge, thus obtaining the optimal clustering result. In the above formula, x l For the l-th sample data, u k Let J be the cluster center of the k-th cluster, and J represent the x-th cluster in the sample data. l with u k The sum of the distances between them; When using the region growing algorithm to accurately segment a crack grayscale image, the point corresponding to the grayscale value of the target cluster center pixel obtained by the K-means clustering algorithm is used as the initial target point. The change in size between the target point and its eight neighboring grayscale pixels is used as the evaluation criterion for growth. Assuming the size of the crack grayscale image is M×N, and G(i,j) represents the pixel grayscale value of the point at coordinate (i,j), a scalar F is assigned to the point (i,j). i,j Then the crack growth criterion S is: S(G(i,j),G(i±1,j±1))≤T In the above formula, G(i±1,j±1) represents the pixel gray value of any point within the eight neighborhoods of point (i,j), S(G(i,j),G(i±1,j±1)) represents the similarity between the pixel gray values ​​of point (i,j) and its eight neighboring points, and T is a threshold with a value of 10. If the above formula is satisfied, then F i,j =1, indicating that point (i,j) belongs to the crack region; otherwise F i,j =0, indicating that point (i,j) does not belong to the crack region; Then, the crack regions in the segmented image are segmented and labeled according to their width; Finally, the crack size information is calculated: Crack area P a For: P a =λ 2 ·P v In the above formula, P v λ is the pixel size of the crack, and λ is the scaling factor of the crack in the image; Crack length: The crack skeleton image is thinned, and the width of the thinned crack is a single pixel. The number of pixels in the crack is obtained by calculating half of the perimeter at this time. The crack length is obtained based on the actual physical size of the pixels. The average width of a crack is the ratio of its area to its length, calculated using the following formula: d e <2S e / C e ,1≤e≤n In the above formula, S e Let C be the area of ​​the e-th crack. e Let d be the perimeter of the e-th crack. e Let be the average width of the e-th crack, and n be the total number of cracks.

2. The intelligent detection method for dam cracks based on UAV visual perception and deep learning according to claim 1, characterized in that, Step 1 includes: First, design the observation route, including the flight range, number of flights, and the division of the sunlit and shaded areas of the dam; Secondly, preset condition parameters are set, including shooting angle, overlap, pixel calibration, offset distance, and flight mode; Finally, image data acquisition and transmission.

3. The intelligent detection method for dam cracks based on UAV visual perception and deep learning according to claim 1, characterized in that, The improved FPN feature pyramid network in step 2 employs a TAM module, which has two inputs: the low-level feature map t z and high-level feature map t z+2 If z = 2 and 3, after processing by the module, the outputs correspond to T respectively. z and T z+2 The low-level output feature map T z Defined as: In the above formula, t d Indicates the lower level t z Enhanced features, t d The definition of is: t d =Sigmoid(t cat ⊙Attf) In the above formula, Sigmoid represents the activation function, t cat The cascaded features are obtained by aggregating the outputs of dilated convolutions with different dilation rates, where r = 1, 2, 3. Attf represents the attention score, defined as: Attf=Sigmoid(W2×δ(W1×P avg (t cat ))) In the above formula, W1 and W2 represent the weight coefficients of the first and second 1×1 convolutional layers, respectively, and δ(·) and P avg (·) represent the ReLU activation function and the global average pooling operation, respectively; t s Represents attention mapping, t s The definition of is: t s =Sigmoid(W3×U(t z+2 )) In the above formula, W3 represents the weight coefficient of the 1×1 convolutional layer, and U(·) represents upsampling; High-level output feature map T z+2 Defined as: In the above formula, D represents the downsampling operation.

Images