A method and system for measuring geometric parameters of building surface cracks based on drones

Abstract

The present invention discloses a method and system for measuring geometric parameters of building surface cracks based on an unmanned aerial vehicle (UAV). The UAV is used to obtain images of the surface of the building to be detected and the distance information between the UAV and the building to be detected. A crack detection model based on multi-feature region attention is used to detect the images of the surface of the building to be detected, and a crack detection result map is output. A crack geometric parameter measurement algorithm is constructed, and the crack geometric parameter measurement algorithm is used to calculate the crack size of the crack detection result map to obtain the crack pixel length information and the crack pixel maximum width information. According to the crack pixel length information, the crack pixel maximum width information, and the distance information between the UAV and the building to be detected, the actual length and the actual maximum width of the building surface cracks are obtained. The present invention realizes non-contact crack measurement and provides a basis for whether to take subsequent repair measures.

Description

Technical Field

[0001] The present invention belongs to the technical field of building health detection, and more specifically, to a method and system for measuring geometric parameters of building surface cracks based on an unmanned aerial vehicle (UAV). Background Technique

[0002] During the construction and use of civil infrastructure such as bridges, roads, and houses, crack problems will inevitably occur. The continuous expansion of cracks will cause damage to the structure, thus affecting the safety and durability of the building. Facing the huge number of existing buildings, traditional crack detection methods relying on the naked eye and magnifying glasses are inefficient and have poor detection accuracy; while the embedded and contact-type sensor detection methods are costly and cause certain damage to the building; on the other hand, the ultrasonic detection method is mainly applicable to cracks inside the structure and has limited applicability to surface cracks. The above detection technologies can no longer meet the current detection needs. Therefore, non-destructive detection of building surfaces has become an urgent research topic. Among them, the building surface crack detection technology based on computer vision can effectively solve this problem and has become a major research hotspot at present.

[0003] With the development of non-contact crack measurement technology, more and more technicians have carried out research on measuring geometric parameters of cracks based on mathematical morphology and achieved certain results. The existing technology measures the pixel width of cracks using the horizontal distance edge point method and the tangent perpendicular method based on the crack edge. By using the Euclidean distance transformation on the binary image of the crack, the crack center line and the crack edge are calculated to obtain the pixel width of the crack. The existing technology has good measurement results for cracks with simple shapes. However, cracks often have complex shapes and severely curved edges. It is difficult to determine the crack direction using the existing technology, and it is impossible to accurately match the pixel points of the crack edge, resulting in large errors in the measurement results. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a method and system for measuring geometric parameters of building surface cracks based on an unmanned aerial vehicle. The method uses the unmanned aerial vehicle to obtain pictures of the building to be detected, uses a crack detection model to obtain the crack detection result, and finally calculates the crack size according to the crack detection result to achieve non-contact crack measurement, providing a basis for whether to take repair measures in the future.

[0005] To achieve the above object, the present invention provides the following technical solution: A method for measuring geometric parameters of building surface cracks based on an unmanned aerial vehicle, the specific steps are as follows:

[0006] S1 Use the unmanned aerial vehicle to obtain pictures of the surface of the building to be detected and the distance information between the unmanned aerial vehicle and the building to be detected;

[0007] S2 uses a crack detection model based on multi-feature region attention to detect the surface image of the building to be detected and outputs a crack detection result map;

[0008] S3 constructs a crack geometric parameter measurement algorithm, and uses the crack geometric parameter measurement algorithm to calculate the crack size of the crack detection result image to obtain crack pixel length information and crack pixel maximum width information;

[0009] S4 obtains the actual length and actual maximum width of the cracks on the building surface according to the crack pixel length information, the crack pixel maximum width information, and the distance information between the drone and the building to be inspected.

[0010] Furthermore, in step S3, the crack geometric parameter measurement algorithm specifically includes the following steps:

[0011] Step S3.1: After the crack detection result image is corrected for geometric distortion, the maximum connected area in the crack detection result image after distortion correction is extracted as a crack for evaluation to obtain the main crack key information image F. 1 ;

[0012] Step S3.2: main crack key information image F 1 Extract the crack skeleton to obtain the crack skeleton map, traverse each pixel point in the crack skeleton map, and then add the distances between adjacent pixels along the skeleton map to obtain the pixel length information of the crack;

[0013] Step S3.3: main crack key information image F 1 Perform edge detection to obtain the crack edge map F 4 , the crack direction is determined according to the crack edge map, the spacing between the crack edge pixels is measured, and the pixel maximum width information of the crack is obtained.

[0014] Furthermore, in step S3.2, the calculation process of the pixel length information of the crack is as follows:

[0015] 1) Traverse the crack skeleton graph and obtain the coordinates of n groups of target points between the starting point and the ending point, (x i ,y i ),i=1,…,n;

[0016] 2) Calculate two consecutive pixel points (x i ,y i ) and (x i+1 ,y i+1 ) i , when two consecutive pixels are in the horizontal and vertical directions, d i Take 1; if the previous pixel is in the diagonal direction of the next pixel, d i Pick

[0017] 3) Accumulate the distances d between two consecutive pixel points i to obtain the pixel length information l of the crack by summation.

[0018] Furthermore, in step S3.3, the specific steps of the edge detection are as follows:

[0019] 1) Perform convolution calculation on the key information image F of the main crack 1 to obtain the gradient and gradient direction of the key information image F of the main crack 1 of the main crack, and calculate the maximum value of the gradient modulus as the edge of the key information image F 1 of the main crack;

[0020] 2) Set a high threshold and a low threshold. When the gradient value of the edge point of the key information image F 1 of the main crack is greater than the high threshold, it is regarded as a strong edge point; when the gradient value of the edge point is less than the high threshold and greater than the low threshold, it is regarded as a weak edge point; when there is no strong edge point in the neighborhood of the weak edge point, it is not regarded as an edge point; when the gradient value of the edge point is less than the low threshold, it is not regarded as an edge point, and the crack edge map F 4 is obtained.

[0021] Furthermore, in step S3.3, the specific steps to obtain the maximum pixel width information of the crack are as follows:

[0022] 1) Determine the crack direction according to the crack edge map, and obtain the two side lines of the crack in the crack edge map based on the crack direction;

[0023] 2) Based on the long side line among the two side lines of the crack, match the crack edge pixels according to the long-short side mapping relationship;

[0024] 3) Obtain the coordinates of two corresponding matching crack edge pixels on the long side line and the short side line, and calculate the maximum pixel width information of the crack. The specific formula is:

[0025]

[0026] where: x l , y l are the coordinates of the matching crack edge pixel on the long side line, and x s , y s are the coordinates of the matching crack edge pixel on the short side line.

[0027] Furthermore, in step 1), the method for judging the crack direction is as follows:

[0028] 1.1) Obtain the coordinates of the pixel points in the crack edge map, and obtain the width W and height H of the crack area;

[0029] 1.2) Establish the coordinate axes of the crack edge map. Take the upper left corner as the origin of the coordinate axes of the crack edge map, the horizontal axis as x, with the rightward direction as the positive direction, and the vertical axis as y, with the downward direction as the positive direction;

[0030] 1.3) If |W - H| < min(W, H) / 2, the crack is inclined. If the ordinate of the first pixel point of the crack edge map is greater than the ordinate of the last pixel point on the right side of the crack edge map, the crack is inclined upward from left to right; otherwise, the crack is inclined downward from left to right;

[0031] 1.4) If |W - H| ≥ min(W, H) / 2 and W > H, the crack is horizontal;

[0032] 1.5) If |W - H| ≥ min(W, H) / 2 and W < H, the crack is vertical.

[0033] Further, in step 1), the method for obtaining the two side lines of the crack in the crack edge map is as follows:

[0034] 1.6) If the crack is horizontal, traverse and search for the side lines from the upper and lower sides of the crack edge map as the starting points towards the center;

[0035] 1.7) If the crack is vertical, traverse and search for the side lines from the left and right sides of the crack edge map as the starting points towards the center;

[0036] 1.8) If the crack is inclined upward from left to right, traverse and search for the side lines from the upper left diagonal and the lower right diagonal as the starting points towards the center;

[0037] 1.9) If the crack is inclined downward from left to right, traverse and search for the side lines from the upper right diagonal and the lower left diagonal as the starting points towards the center;

[0038] Obtain the set of pixel coordinate points of the two side lines of the crack: the set of points on the long side line of the crack is L 1 ={l 1，1 , l 1，2 ,...l 1，n}, and the set of points on the short side line of the crack is L 2 ={l 2，1 , l 2，2 ,...l 2，m}, where n is the number of pixel points on the long side line of the crack, m is the number of pixel points on the short side line of the crack, and n > m.

[0039] Further, in step 2), the mapping relationship f between the pixel point l 1，p in the set of long side lines of the crack and the pixel point l 2，q in the set of short side lines of the crack is specifically as follows:

[0040] f(L 2 (l 2，q ))→L 1 (l 1，p ) (6)

[0041] In the formula, f is the mapping relationship between the pixel points of the long side line of the crack and the pixel points of the short side line of the crack, l 1 is the long side line, l 2 represents the short side line, p ∈ [1, n], q ∈ [1, m], p = [q × rate + 0.5], and [p × rate + 0.5] is the result of rounding down q × rate + 0.5.

[0042] Furthermore, in step S4, according to the distance information between the drone and the building to be detected, the scale factor is calculated, and the pixel length information of the crack and the maximum pixel width information of the crack are multiplied by the scale factor to obtain the actual length and the actual maximum width of the crack on the building surface. The calculation formula of the scale factor is:

[0043]

[0044] In the formula, p 1 = 0.0003434, p 2 = 0.003446, D is the distance between the drone and the building to be detected, and D ∈ {200, 2000}.

[0045] The present invention also provides a geometric parameter measurement system for cracks on the building surface based on a drone, including:

[0046] An information acquisition module, which is used to obtain the surface image of the building to be detected and the distance information between the drone and the building to be detected by using the drone;

[0047] An image detection module, which is used to detect the surface image of the building to be detected by using a crack detection model based on multi-feature region attention and output a crack detection result map;

[0048] A crack geometric parameter calculation module, which is used to construct a crack geometric parameter measurement algorithm and use the crack geometric parameter measurement algorithm to calculate the crack size of the crack detection result map to obtain the pixel length information of the crack and the maximum pixel width information of the crack;

[0049] A crack actual parameter calculation module, which is used to obtain the actual length and the actual maximum width of the crack on the building surface according to the pixel length information of the crack, the maximum pixel width information of the crack, and the distance information between the drone and the building to be detected.

[0050] Compared with the prior art, the present invention has at least the following beneficial effects:

[0051] In view of the characteristics of the crack detection task, the present invention provides a method for measuring the geometric parameters of cracks on the surface of a building based on an unmanned aerial vehicle (UAV). First, aiming at the problems of low efficiency and certain safety hazards of the traditional manual detection method, the present invention uses the UAV as a carrier to break through the limitations of space and position, and obtains the images of the surface of the building to be detected and the distance information between the UAV and the building to be detected through the UAV. Then, aiming at the problem that it is difficult to focus on the semantic information at the crack region level, which leads to poor continuity of the detection results, a crack detection model based on multi-feature region attention is constructed and trained, and the captured images of the building surface are input into the model to obtain the crack detection results. Next, aiming at the problems of difficult determination of the crack measurement direction and difficult matching of edge pixel points, a geometric parameter measurement algorithm is constructed, and the crack detection results are input into the algorithm to obtain the crack pixel length information and the pixel maximum width information. Finally, the scale factor information is obtained according to the distance from the crack surface during shooting and multiplied by the crack pixel parameter information to output the crack length information and the maximum width information. This method can realize the automatic measurement of the widths of cracks with different orientations, has good measurement accuracy and high crack detection efficiency, can effectively alleviate the problems of the increasing number of buildings and the difficult maintenance of infrastructure, and provides a valuable reference solution for the sustainable development of construction projects.

[0052] Furthermore, in the crack geometric parameter measurement algorithm of the present invention, image geometric distortion correction is used to eliminate the measurement errors caused by image distortion; an information extraction module is used to extract the key features of the cracks, providing a basis for subsequent geometric measurement calculations; the crack pixel length is measured using the crack skeleton instead of the crack itself; based on the strategy of determining the crack orientation and matching edge pixel points, the maximum width measurement of cracks with complex shapes and severely curved edges can be realized, and the measurement results have small errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flowchart of a method for measuring the geometric parameters of the surface of a building based on an unmanned aerial vehicle in the present invention;

[0054] Figure 2 is the crack detection network based on multi-feature region attention in the present invention;

[0055] Figure 3 is the structural diagram of the region-level channel attention module in the present invention;

[0056] Figure 4 is the structural diagram of the crack geometric parameter measurement algorithm in the present invention;

[0057] Figure 5 is a partial experimental result diagram of the present invention on the CRACK500 dataset;

[0058] Figure 6 This is the experimental result diagram of crack measurement in the actual verification of the present invention. Specific implementation manner

[0059] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation manners.

[0060] A method for measuring geometric parameters of building surface cracks based on an unmanned aerial vehicle, and the specific process is as Figure 1 shown.

[0061] S1 Construct a crack image training data set and a verification data set, use an unmanned aerial vehicle to take crack images of the building to be detected, and obtain the distance information between the unmanned aerial vehicle and the building to be detected;

[0062] S2 Construct and train a crack detection network based on multi-feature region attention, as Figure 2 shown, to obtain a crack detection model based on multi-feature region attention, and then perform crack detection on the surface image of the building to be detected, and output a crack detection result diagram. The crack detection network based on multi-feature region attention includes a U-Net backbone network with an encoder-decoder structure and a regional channel attention module, and the regional channel attention module is embedded in the decoder.

[0063] S3 Construct a crack geometric parameter measurement algorithm, use the crack geometric parameter measurement algorithm to calculate the crack size of the crack detection result diagram to obtain the pixel length information and the maximum width information of the crack, and finally obtain a scale factor according to the distance from the crack surface during shooting and multiply it by the crack pixel information to output the actual length and the actual maximum width of the crack.

[0064] Further, in step S2, the encoder part of the U-Net backbone network follows the typical architecture of a convolutional network, and VGG-16 is used as the feature extraction network.

[0065] Specifically, the fully connected layer and part of the convolutional layers of VGG-16 are removed, so its encoder includes 5 stages of convolutional layers, and the number of convolutional layers in each stage is 2, 2, 2, 2, and 3 respectively. Assuming that the size of the input image is M×M, the convolutional layer parameters of each stage and the size of the feature map output after pooling are shown in Table 1.

[0066] Table 1 Encoder structure parameters

[0067]

[0068] Further, in step S2, the decoder part is symmetric to the encoder part, and the feature map is mapped to the original image resolution size through an upsampling operation with a stride of 2, and a feature map concatenation operation is used to connect with the encoder part.

[0069] Specifically, the decoder is divided into four convolutional stages and one output stage. Each convolutional stage contains two upsampling layers. The feature map output by one upsampling is connected to the regional channel attention module, and the other upsampling layer is connected to two 3×3 convolutions. The output stage consists of a 1×1 convolution connected to the softmax activation function.

[0070] Furthermore, the upsampling layer uses 2×2 bilinear interpolation to enlarge the feature map size. To reduce the problem of feature map information loss caused by upsampling, the feature maps of the encoding layer and the decoding layer at the same depth are also concatenated to improve the segmentation accuracy. Assuming that the input feature map size is M / 16×M / 16, the output feature map sizes of each stage are shown in Table 2.

[0071] Table 2 Decoder Structure Parameters

[0072]

[0073] Furthermore, in step S2, the regional channel attention module is as Figure 3 shown. To make the network pay more attention to the regional information of the image, the input feature map is first divided into partial regions, and the network focuses on the feature mapping in the regional channel dimension of the crack feature map to generate a feature descriptor, so as to assign attention weights to each feature map region.

[0074] Specifically, let the input feature map be The feature descriptor is obtained through Equation (1)

[0075] U = mean - pooling(V) (1)

[0076] In the formula, mean - pooling represents an average pooling operation with a stride of 4 and a convolution kernel of 4.

[0077] Furthermore, the obtained feature descriptor U is passed through Equation (2) to obtain the regional channel feature map Then the regional channel feature map is passed through Equation (3) to obtain the crack feature map that strengthens the regional - level information

[0078] T = F U {σ[W 2 δ(W 1 U)]} (2)

[0079]

[0080] In the formula, W 1 and W 2They respectively represent the 1×1 convolution for dimensionality reduction and the 1×1 convolution for dimensionality increase. The operation of dimensionality reduction and then dimensionality increase on the feature map helps to enhance the interaction between channels, extract effective features, and further enhance the sensitivity of crack features to regional information. Among them, where C is the number of input channels, is the number of output channels, and γ is the dimensionality reduction rate; where is the number of input channels, C is the number of output channels. σ is the sigmoid non-linear activation, δ is the ReLu non-linear activation, and F U is upsampling, is the multiplication of corresponding pixels of the feature map, is the addition of corresponding pixels of the feature map.

[0081] Furthermore, in step S2, the training dataset is used to train the crack detection network based on multi-feature region attention. Set the epoch to 60, use the random gradient descent method with restarts for training, the initial learning rate is 0.001258, and the minimum learning rate is set to 10-7.

[0082] Furthermore, in step S2, the constructed crack detection network based on multi-feature region attention is trained to obtain a crack detection model based on multi-feature region attention.

[0083] Specifically, first download the CRACK500 dataset, preprocess the images to obtain the dataset, and divide the dataset into a crack image training dataset and a validation dataset according to a ratio of 9:1. Then use the training dataset to train the crack detection network based on multi-feature region attention. Set the epoch to 60, use the random gradient descent method with restarts for training, the initial learning rate is 0.001258, and the minimum learning rate is set to 10-7. Through repeated parameter tuning, a crack detection model based on multi-feature region attention is obtained.

[0084] Furthermore, the crack images obtained by the drone are input into the crack detection model based on multi-feature region attention for detection, and a crack detection result map is output.

[0085] Furthermore, in step S3, the processing process of the crack geometric parameter measurement algorithm for the crack detection result map is as follows:

[0086] In step S3.1, the crack detection result map is subjected to image geometric distortion correction to eliminate the measurement error caused by image distortion, and a crack detection result map after distortion correction is obtained.

[0087] Specifically, the specific steps for image geometric distortion correction are as follows: First, make a calibration board and take several photos of the checkerboard from different angles; then detect feature points (Harris features) in the pictures; finally, solve the internal parameter matrix, external parameter matrix, and distortion parameters, and finally perform coordinate system transformation on the crack detection result image using the internal parameter matrix and distortion parameters to obtain the crack detection result image after distortion correction.

[0088] Preferably, Zhang Zhengyou calibration method is used to perform image geometric distortion correction on the crack detection result image;

[0089] In step S3.2, information extraction is performed on the crack detection result image after distortion correction to obtain the pixel length information and pixel maximum width information of the crack, specifically as follows:

[0090] 1) Perform connected component extraction on the crack detection result image after distortion correction to reduce the influence of multiple cracks on the crack geometric measurement result. Take the largest connected region in the crack detection result image as a crack for evaluation to obtain the key information image F of the main crack 1 ;

[0091] Specifically, first traverse each connected component, calculate the area of each connected component and put it into a list; then sort them in ascending order according to the area, fill the pixel values of the regions other than the area of the largest connected component with 0; finally, input the crack detection result image after distortion correction into the largest connected component extraction for calculation, and output the key information image F of the main crack 1 。

[0092] 2) Perform skeleton extraction on the key information image F of the main crack 1 to obtain the crack skeleton diagram. Specifically:

[0093] Skeleton extraction is to refine the crack into a thin curve with a single pixel width. The crack skeleton reflects the morphological characteristics such as the direction, structure, and length of the crack. Therefore, the crack skeleton can replace the crack itself as the measurement object;

[0094] Preferably, the K3M thinning algorithm is used for skeleton extraction. The K3M thinning algorithm is specifically divided into two steps:

[0095] The first step is to continuously refine the key information image F of the main crack 1 to extract the crack pseudo-skeleton. At this time, the morphology of the pseudo-skeleton is already close to the true skeleton of the crack, but there are still some skeleton regions with multiple pixel widths.

[0096] The second step is to refine the crack pseudo-skeleton into a real skeleton with a single pixel width and output the crack skeleton diagram.

[0097] 3) For the key information image F of the main crack 1Perform edge detection to obtain a crack edge map. Specifically:

[0098] Edge detection is to extract the crack contour, and the contour of the crack reflects the shape of the crack. According to the distribution characteristics of the crack, by calculating the pixel point spacing on both sides of the crack edge, the width of the crack can be measured.

[0099] Preferably, the Canny operator is used for edge detection. The detection process of the Canny operator is as follows:

[0100] The first step is to filter the key information image F of the main crack using a Gaussian smoothing rate filter 1 to filter out interference information, reduce the detection error rate, and output the key information image F of the main crack after noise reduction 1 ;

[0101] The second step is to perform convolution calculation on the key information image F of the main crack after noise reduction using the Sobel operator 1 to obtain the gradient and gradient direction of the key information image F of the main crack after noise reduction 1 calculate the maximum value of the gradient modulus and perform superposition. After traversing the entire image, output the crack edge map F 2 .

[0102] The third step is that since the global gradient obtained cannot determine the edge, it is necessary to retain the points with the largest local gradient to suppress non-maximum values. First, according to the gradient direction of the crack edge map F 2 compare the gradient intensities of the current point in the positive and negative gradient directions. If the gray value remains unchanged when the gradient intensity is the largest at the current point, on the contrary, the gray value is set to 0. After traversing the entire image, output the crack edge map F 3 .

[0103] The fourth step is that after using non-maximum suppression, there are still some edge pixels in the image due to noise or color changes, so it is necessary to filter the edge pixels. First, set a high threshold and a low threshold to segment the crack edge map F 3 Specifically: when the gradient value of the edge point is greater than the high threshold, it is regarded as a strong edge point; when the gradient value of the edge point is less than the high threshold and greater than the low threshold, it is regarded as a weak edge point. If there is no strong edge point in the neighborhood of the weak edge point, the point is suppressed and the gray value is set to 0; when the gradient value of the edge point is less than the low threshold, the point is suppressed and the gray value is set to 0. After traversing the entire image, output the crack edge map F 4 .

[0104] 4) Traverse each pixel point in the crack skeleton map, and then add up the distances between adjacent pixel points along the skeleton map to obtain the pixel length information of the crack. Specifically:

[0105] Since the positional relationships of adjacent pixel points in the skeleton map include horizontal, vertical connection, and diagonal connection. Based on this, the calculation rule of the crack skeleton length is defined as follows: In the horizontal and vertical directions, the length of adjacent pixel points is recorded as 1; if the previous pixel point is in the diagonal direction of the next pixel point, the length of adjacent pixel points is recorded as

[0106] The specific process of the crack length measurement algorithm is as follows:

[0107] 1) By traversing the crack skeleton map, the coordinates of n groups of pixel points from the starting point to the ending point are obtained, (x i , y i ), i = 1,..., n;

[0108] 2) Calculate the distance d i between two consecutive pixel points (xi, yi) and (xi+1, yi+1) according to formula (4);

[0109] 3) According to formula (5), sum up the distances d i between two consecutive pixel points to obtain the pixel length information l of the crack.

[0110]

[0111] l = ∑d i (5)

[0112] 5) Based on judging the crack direction, measure the distance between edge pixel points in the crack edge map and record its maximum value. The process is as follows: First, determine the crack direction according to the crack edge map; secondly, search for the crack edge line through the crack direction; then match the crack edge points based on the crack edge line, and perform edge pixel point matching according to the edge line ratio relationship matching strategy; finally, calculate the distance between corresponding points and record its maximum distance. Specifically:

[0113] 1) Determine the crack direction based on the crack edge map

[0114] Determining the crack direction is the prerequisite for measuring the crack width. The process of judging the crack direction is as follows:

[0115] 1.1) Obtain the coordinates of pixel points in the crack edge map and get the width W and height H of the crack area.

[0116] 1.2) Establish the coordinate axes of the crack edge map, with the upper left corner as the origin of the coordinate axes of the crack edge map, the horizontal axis as x, and the rightward direction as the positive direction, and the vertical axis as y, and the downward direction as the positive direction.

[0117] 1.3) If |W - H| < min(W, H) / 2, the crack is inclined. If the ordinate of the first pixel point of the crack edge image is greater than the ordinate of the last pixel point on the right side of the crack edge image, the crack is inclined upward from left to right; otherwise, the crack is inclined downward from left to right.

[0118] 1.4) If |W - H| ≥ min(W, H) / 2 and W > H, the crack is horizontal.

[0119] 1.5) If |W - H| ≥ min(W, H) / 2 and W < H, the crack is vertical.

[0120] 2) Search for the crack edge based on the crack direction

[0121] After determining the crack direction, search for the crack edge points to obtain the pixel position coordinates on the edge. The specific process is as follows:

[0122] 2.1) If the crack is horizontal, traverse and search for the edge from the upper and lower sides of the crack edge image towards the center; if the crack is vertical, traverse and search for the edge from the left and right sides of the crack edge image towards the center.

[0123] 2.2) If the crack is inclined upward from left to right, traverse and search for the edge from the upper left diagonal and the lower right diagonal towards the center; if the crack is inclined downward from left to right, traverse and search for the edge from the upper right diagonal and the lower left diagonal towards the center.

[0124] 2.3) Store the coordinate groups of the two edges of the obtained crack into two sets respectively, that is, the set of long edge points of the crack is L 1 = {l 1，1 , l 1，2 ,... l 1，n}, and the set of short edge points of the crack is L 2 = {l 2，1 , l 2，2 ... l 2，m}, where n is the number of pixel points on the long edge of the crack, m is the number of pixel points on the short edge of the crack, and n > m, for subsequent operations.

[0125] 3) Match the crack edge pixels based on the crack edge

[0126] The two side edges of the crack will have a certain inclination or bending at different parts, making the edge points not in one-to-one correspondence. In order to establish the correspondence relationship of the edge points, a strategy of matching the crack edge pixels according to the ratio relationship between the long and short sides is adopted with one side of the long edge of the crack as the reference. Specifically:

[0127] The pixel point l in the set of long edge points of the crack1，p Through formula (6) and pixel point l in the short side line of the crack 2，q Establish the mapping relationship f. f(L 2 (l 2，q )) → L 1 (l 1，p ) (6)

[0128] In the formula, f is the mapping relationship between the pixel points of the long side list and the short side list, p is the number of pixel points of the long side of the crack, q is the number of pixel points of the short side of the crack, l 1 is the long side, l 2 represents the short side, p ∈ [1, n], q ∈ [1, m], p = [q × rate + 0.5], and [p × rate + 0.5] is the result of rounding down q × rate + 0.5.

[0129] 4) Calculate the distance between the matching edge pixel points

[0130] It can be seen from the mapping relationship f that for a pixel point (x l , y l ) in the long side list of the crack, there is a corresponding pixel point (x s , y s ) in the short side list. Based on the above relationship, calculate the Euclidean distance between the pixel points through formula (7), and record the maximum value as the maximum pixel width information of the crack.

[0131]

[0132] Furthermore, in step S3, obtain the scale factor according to the distance from the crack surface during shooting and multiply it by the crack pixel information to output the actual length and actual maximum width of the crack. The scale factor is the physical size corresponding to the pixel size of the object in the picture. Through the experiment of setting the distance between the drone and the measured target, obtain the fitting relationship between the actual length corresponding to the pixel point and the distance between the drone and the building to be detected.

[0133] Specifically, first make a black rectangular block with a white background as a reference object, then use the drone to take pictures of the reference object and ensure that the reference object is always in the center of the drone's shooting screen. Then move the drone starting from the initial position (200 mm) and move it at a step size of 60 mm each time to the maximum position of 2000 mm. At the same time, use a laser rangefinder to measure the distance between the drone and the reference object during the movement and record it. Repeat the experiment three times and take the average value. Finally, obtain the fitting relationship between the actual length (mm) corresponding to each pixel point and the distance (mm) of the measured target as shown in formula (8).

[0134] In the formula, R is the scale factor (mm), representing the actual physical size corresponding to a single pixel point in the picture, p 1 = 0.0003434, p 2 = 0.003446, D is the distance (mm) between the drone and the building to be detected, D ∈ {200, 2000}.

[0135] Specifically, the actual length L of the crack is obtained from the pixel length information of the crack through Equation (9), and the actual maximum width W of the crack is obtained from the pixel maximum width information of the crack through Equation (10).

[0136] L = R × l (9)

[0137] W = R × w max (10)

[0138] The present invention also provides a measurement system for geometric parameters of building surface cracks based on a drone, including:

[0139] An information acquisition module for using the drone to obtain the surface picture of the building to be detected and the distance information between the drone and the building to be detected;

[0140] An image detection module for using a crack detection model based on multi-feature region attention to detect the surface picture of the building to be detected and output a crack detection result picture;

[0141] A crack geometric parameter calculation module for constructing a crack geometric parameter measurement algorithm and using the crack geometric parameter measurement algorithm to calculate the crack size of the crack detection result picture to obtain the pixel length information of the crack and the pixel maximum width information of the crack;

[0142] A crack actual parameter calculation module for obtaining the actual length and actual maximum width of the building surface crack according to the pixel length information of the crack, the pixel maximum width information of the crack, and the distance information between the drone and the building to be detected.

[0143] Furthermore, as Figure 4 shown, the crack geometric parameter measurement algorithm includes an image geometric distortion correction module, an information extraction module, a length measurement module, and a maximum width measurement module, where:

[0144] The image geometric distortion correction module is used to eliminate the measurement error caused by image distortion using the Zhang Zhengyou calibration method, and the crack detection result picture is corrected using the image geometric distortion correction module to obtain a crack detection result picture after distortion correction.

[0145] The information extraction module is used to extract the maximum connected domain of the crack detection result image after distortion correction to reduce the influence of multiple cracks on the crack geometry measurement results. The maximum connected area in the crack detection result image is selected as a crack for evaluation to obtain the main crack key information image F. 1 , and the main crack key information image F 1 Perform skeleton extraction and edge detection to obtain crack skeleton map and crack edge map F 4 .

[0146] The length measurement module is used to traverse each pixel point in the crack skeleton map, and then add the distances between adjacent pixels along the skeleton map to obtain the pixel length information of the crack.

[0147] The maximum width measurement module is used to measure the distance between the pixels at the edge of the crack based on the determination of the crack direction, and obtain the pixel maximum width information of the crack.

[0148] Figure 5 Part of the test results of the present invention on the CRACK500 dataset are shown. The first column is the original image, the second column is the true value image, and the third to eighth columns are the crack detection results of FCN, DeepLabV3, SegNet, PSPNet, U-Net and the method of the present invention respectively. Figure 5 (a) The contrast between the pothole and the crack background area on the right side of the first row of cracks is not obvious. Among the six models, only the method proposed in this paper can recognize the general outline of the pothole. Figure 5 (a) The second row of cracks are distributed in a block-like and grid-like manner, which makes detection more difficult. The proposed method focuses on the regional-level features of cracks, effectively strengthening the model's learning of the local strongly correlated features of cracks, making its detection results closest to the true value map. Figure 5 (a) The third and fourth rows are narrow and small cracks. The mainstream semantic segmentation models cannot extract continuous cracks, while the proposed method completely identifies the topological structure of the cracks, and there is no break in the detection results, which once again proves that the strategy of focusing on the crack region-level features proposed in this paper is feasible and effective. Figure 5 (a) The fifth row shows a crack with gravel interference. In the experiment, all models identified the interruption area between the gravel and the crack as a crack, but even so, the proposed method still extracts the morphology of the gravel better.

[0149] Figure 6 The test results of the present invention in photographing typical cracks in real life are shown, where the first row is the original crack image, the second row is the crack detection result, the third row is the maximum connected domain extraction result, the fourth row is the length measurement result, and the fifth row is the maximum width measurement result. Table 3 is Figure 6 Crack measurement results.

[0150] Table 3 Fracture Measurement Results

[0151]

[0152] As can be seen from the fracture length measurement results in Table 3, the algorithm proposed in the present invention has a fracture length measurement result close to the actual length, and the overall relative error is controlled within the range of ±5%, with an average relative error of 3.5%. On the one hand, the reason for the error is that the fracture skeleton diagram extracted by the skeleton algorithm is a thin curve, which appears to be more curved than during manual measurement, resulting in a measurement result greater than the actual length. On the other hand, there is also a certain deviation between the scale factor in the calibrated pixel ratio relationship and the actual fracture length.

[0153] In addition, for the measurement results of the maximum width of the fractures in Table 3, Figure 6 (a), 6(b), and 6(d) have relatively good measurement effects, and the relative error is controlled within the range of ±3%. However, Figure 6 (c) has wider fracture detection results than the actual due to the interference of the water stain area around the fracture, which also leads to a 7.6% higher measurement of the maximum width than the true value. And in Figure 6 (e), the measured value in the detection result of the thin fracture is 6.3% higher than the true value, which is caused by the detection result error. However, the overall relative error is controlled within the range of ±8%, and the average relative error is 4.4%.

Claims

1. A method for measuring geometric parameters of building surface cracks based on drones, characterized in that, the specific steps are as follows: S1 Use drones to obtain pictures of the surface of the building to be detected and the distance information between the drones and the building to be detected; S2 Use a crack detection model based on multi-feature region attention to detect the pictures of the surface of the building to be detected, and output a crack detection result map; S3 Construct a crack geometric parameter measurement algorithm, and use the crack geometric parameter measurement algorithm to calculate the crack size of the crack detection result map to obtain the crack pixel length information and the maximum crack pixel width information; S4 Obtain the actual length and actual maximum width of the building surface cracks according to the crack pixel length information, the maximum crack pixel width information, and the distance information between the drone and the building to be detected; In step S3, the crack geometric parameter measurement algorithm specifically includes the following steps: After performing image geometric distortion correction on the crack detection result image in step S3.1, the largest connected region in the distortion-corrected crack detection result image is extracted as a crack for evaluation, and the key information image F of the main crack is obtained. 1 ; Step S3.2 performs crack skeleton extraction on the key information image F of the main crack 1 to obtain a crack skeleton map, traverses each pixel point in the crack skeleton map, and then adds the distances between adjacent pixel points along the skeleton map to obtain the pixel length information of the crack; Step S3.3 performs edge detection on the key information image F of the main crack 1 to obtain the crack edge map F 4 , determines the crack trend based on the crack edge map, measures the distance between the pixel points on the crack edge, and obtains the maximum pixel width information of the crack; In step S3.3, the specific steps for obtaining the maximum crack pixel width information are as follows: 1) Determine the crack trend according to the crack edge map, and obtain the two side lines of the crack in the crack edge map based on the crack trend; 2) Based on the long side line among the two side lines of the crack, match the crack edge pixels according to the mapping relationship between the long and short sides; 3) Obtain the coordinates of two corresponding matching crack edge pixels on the long side line and the short side line, and calculate the maximum crack pixel width information. The specific formula is: where: x l , y l are the coordinates of the matching crack edge pixels on the long side line, x s , y s are the coordinates of the matching crack edge pixels on the short side line In step 2), the mapping relationship f between the pixel points l in the set of long crack edges 1,p and the pixel points l in the short crack edges 2,q is as follows: f(L 2 (l 2,q ))→L 1 (l 1,p ) (6) Where f is the mapping relationship between the pixel points of the long side line of the crack and the pixel points of the short side line of the crack, and l 1 represents the long side line, l 2 represents the short side line, p ∈ [1, n], q ∈ [1, m], p = [q × rate + 0.5], and [p × rate + 0.5] is the result of rounding down q × rate + 0.

5.

2. According to the method for measuring geometric parameters of building surface cracks based on drones described in claim 1, characterized in that, In step S3.2, the calculation process of the crack pixel length information is specifically as follows: 1) Traverse the crack skeleton diagram to obtain the coordinates of n groups of target points from the starting point to the ending point, (x i , y i ), where i = 1, …, n; 2) Calculate the distance d i between two consecutive pixel points (x i , y i+1 ) and (x i+1 , y i ). When the two consecutive pixel points are in the horizontal and vertical directions, d i takes 1; if the previous pixel point is in the diagonal direction of the next pixel point, d i takes 3) Accumulate the sum of the distances d between two consecutive pixel points i to obtain the pixel length information l of the crack.

3. According to the method for measuring geometric parameters of building surface cracks based on drones described in claim 1, characterized in that, In step S3.3, the specific steps of the edge detection are: 1) Perform convolution calculation on the key information image F of the main crack 1 to obtain the gradient and gradient direction of the key information image F of the main crack 1 and calculate the maximum value of the gradient modulus as the edge of the key information image F of the main crack 1 ; 2) Set a high threshold and a low threshold. When the gradient value of the edge points in the key information image F of the main crack 1 is greater than the high threshold, it is regarded as a strong edge point; when the gradient value of the edge point is less than the high threshold and greater than the low threshold, it is regarded as a weak edge point; when there is no strong edge point in the neighborhood of the weak edge point, it is not regarded as an edge point; when the gradient value of the edge point is less than the low threshold, it is not regarded as an edge point, and the crack edge map F 4 is obtained.

4. According to the method for measuring geometric parameters of building surface cracks based on drones described in claim 1, characterized in that, In step 1), the method for judging the crack trend is as follows: 1.1) Obtain the coordinates of the pixels in the crack edge map, and obtain the width W and height H of the crack area; 1.2) Establish a crack edge map coordinate axis, with the upper left corner as the origin of the crack edge map coordinate axis, the horizontal axis as x, the rightward direction as the positive direction, the vertical axis as y, and the downward direction as the positive direction; 1.3) If |W - H| < min(W, H) / 2, the crack is inclined. If the ordinate of the first pixel point of the crack edge map is greater than the ordinate of the last pixel point on the right side of the crack edge map, the crack is inclined upward from left to right; otherwise, the crack is inclined downward from left to right; 1.4) If |W - H| ≥ min(W, H) / 2 and W > H, the crack is horizontal; 1.5) If |W - H| ≥ min(W, H) / 2 and W < H, the crack is vertical.

5. According to the method for measuring geometric parameters of building surface cracks based on drones described in claim 4, characterized in that, In step 1), the method for obtaining the two side lines of the crack in the crack edge map is as follows: 1.6) If the crack is horizontal, search for the side lines from the upper and lower parts of the crack edge map towards the center respectively; 1.7) If the crack is vertically oriented, traverse and search for the edge lines from the left and right sides of the crack edge map towards the center respectively; 1.8) If the crack is diagonally upward from left to right, traverse and search for the edge lines from the upper left diagonal and the lower right diagonal towards the center respectively; 1.9) If the crack is diagonally downward from left to right, traverse and search for the edge lines from the upper right diagonal and the lower left diagonal towards the center respectively; Obtain the set of pixel coordinate points of the two side lines of the crack: The set of points on the long side line of the crack is L 1 ={l 1,1 ,l 1,2 ,…l 1,n}, and the set of points on the short side line of the crack is L 2 ={l 2,1 ,l 2,2 …l 2,m}, where n is the number of pixel points on the long side line of the crack, m is the number of pixel points on the short side line of the crack, and n > m.

6. A method for measuring geometric parameters of building surface cracks based on an unmanned aerial vehicle, as claimed in claim 1, characterized in that, in step S4, a scale factor is calculated according to the distance information between the unmanned aerial vehicle and the building to be detected, and the pixel length information of the crack and the maximum pixel width information of the crack are multiplied by the scale factor to obtain the actual length and the actual maximum width of the building surface crack. The calculation formula of the scale factor is: where p 1 = 0.0003434, p 2 = 0.003446, D is the distance between the UAV and the building to be detected, D ∈ {200,2000}。 7. A system for measuring geometric parameters of building surface cracks based on an unmanned aerial vehicle, characterized in that, comprising: an information acquisition module, configured to use an unmanned aerial vehicle to obtain images of the surface of the building to be detected and the distance information between the unmanned aerial vehicle and the building to be detected; an image detection module, configured to use a crack detection model based on multi-feature region attention to detect the image of the surface of the building to be detected and output a crack detection result map; a crack geometric parameter calculation module, configured to construct a crack geometric parameter measurement algorithm and use the crack geometric parameter measurement algorithm to calculate the crack size of the crack detection result map to obtain the pixel length information of the crack and the maximum pixel width information of the crack; a crack actual parameter calculation module, configured to obtain the actual length and the actual maximum width of the building surface crack according to the pixel length information of the crack, the maximum pixel width information of the crack, and the distance information between the unmanned aerial vehicle and the building to be detected; the crack geometric parameter measurement algorithm specifically includes the following steps: After performing image geometric distortion correction on the crack detection result image, extract the largest connected region in the distortion-corrected crack detection result image as a crack for evaluation to obtain the key information image F of the main crack 1 ; Step 2 performs crack skeleton extraction on the key information image F of the main crack 1 to obtain a crack skeleton diagram, traverses each pixel point in the crack skeleton diagram, and then adds up the distances between adjacent pixel points along the skeleton diagram to obtain the pixel length information of the crack; Step 3 performs edge detection on the key information image F of the main crack 1 to obtain the crack edge map F 4 , determines the crack direction based on the crack edge map, measures the distance between the pixel points on the crack edge, and obtains the maximum pixel width information of the crack; In step 3, the specific steps for obtaining the maximum pixel width information of the crack are as follows: 1) Determine the crack orientation according to the crack edge map, and obtain the two side edge lines of the crack in the crack edge map based on the crack orientation; 2) Based on the long edge line among the two side edge lines of the crack, match the crack edge pixels according to the long and short side mapping relationship; 3) Obtain the coordinates of two corresponding matched crack edge pixels on the long edge line and the short edge line, and calculate the maximum pixel width information of the crack. The specific formula is: Where: x l , y l are the coordinates of the matching crack edge pixels on the long side line, x s , y s are the coordinates of the matching crack edge pixels on the short side line In step 2), the mapping relationship f between the pixel points l in the set of long crack edges 1,p and the pixel points l in the short crack edges is as follows: 2,q Specifically: f(L 2 (l 2,q ))→L 1 (l 1,p ) (6) Wherein, f is the mapping relationship between the pixel points of the long side line of the crack and the pixel points of the short side line of the crack, and l 1 is the long side line, and l 2 represents the short side line, p ∈ [1, n], q ∈ [1, m], p = [q × rate + 0.5], and [p × rate + 0.5] is the result of rounding down q × rate + 0.5.

Images