An underwater concrete crack segmentation method and computing device

Abstract

The present application belongs to the technical field of concrete structure damage detection, and provides an underwater concrete crack segmentation method and a computing device. The present application obtains crack video of underwater concrete by using a wall-climbing robot, disassembles the crack video into a picture sequence, and then processes the picture sequence by using a CLN model to realize prediction of the video image and accurately grasp the overall damaged information of the structure surface. The present application can expand the structure damage detection range to underwater according to factors such as water flow action in the water environment and interference of aquatic organisms, and can complete the overall surface of the structure in combination with a three-dimensional reconstruction technology, thereby realizing accurate restoration and disease detection of the underwater concrete structure. The present application has high segmentation precision for cracks on the surface of underwater concrete, can measure the geometric parameters of the cracks in the image, and restores the three-dimensional structure of the underwater concrete, so that the algorithm can provide complete damage information of the structure and has strong practical application value and good engineering significance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of concrete structure damage detection, and particularly relates to an underwater concrete crack segmentation method and a computing device. BACKGROUND

[0002] Cracks are the most common and the most influential disease of concrete structures. The development of cracks will reduce the effective load area and eventually lead to the decline of the overall safety and durability of the structure, causing accidents. Due to the characteristics of concrete materials, cracks are inevitable, especially for underwater concrete structures exposed to river or seawater environments, the expansion speed of cracks will be much faster than that on land. Therefore, the surface crack detection of underwater concrete structures is an important task to ensure the safety of bridges.

[0003] In recent years, the method of detecting concrete cracks based on computer vision deep learning has been used in the field of health detection of civil structures. The commonly used algorithm is semantic segmentation algorithm. Semantic segmentation algorithm understands and interprets images from the pixel level, and can accurately segment cracks and backgrounds from the boundary. Most of the current semantic segmentation algorithms are for research on land cracks, and the establishment or optimization of the model is also carried out on the land crack dataset. Moreover, the output of the algorithm is the predicted segmentation result of a single image, and the researchers' field of vision is limited to a single image. Without effective three-dimensional structure reconstruction means, the overall damage information of the structure surface cannot be obtained.

[0004] In summary, the existing technology has the following problems: the algorithm lacks special optimization for the surface cracks of underwater concrete structures, and the land model is not applicable; the algorithm is limited to the prediction result of a single image, and cannot accurately grasp the overall damage information of the structure surface.

[0005] Therefore, it is necessary for those skilled in the art to improve the semantic segmentation algorithm according to factors such as the action of water flow in the water environment and the interference of aquatic organisms, expand the structure damage detection range to underwater, and combine with three-dimensional reconstruction technology to complete the overall surface of the structure, so as to realize the accurate restoration and disease detection of underwater concrete structures. SUMMARY

[0006] In order to achieve the above purpose, the present application provides the following technical solutions:

[0007] The present application provides an underwater concrete crack segmentation method, which comprises:

[0008] (1) collecting crack videos of the surface of underwater concrete structures by using a wall-climbing robot and disassembling the crack videos into image sequences;

[0009] (2) inputting the images into a CLN model to output a segmentation image Image-m;

[0010] (3) Quantize the segmented image Image-m in size using a skeleton extraction algorithm:

[0011] (4) Three-dimensional reconstruction of the underwater concrete structure using a NeRF model.

[0012] The specific steps for training the CLN model (convolutional neural network model) are as follows:

[0013] (1) Obtain crack data;

[0014] (2) Use the Labelme program to label the cracks;

[0015] (3) Input the data into the CLN model for training;

[0016] (4) Verify the training results and adjust the hyperparameter set of the model;

[0017] (5) The model with the best verification result in training is saved and set as the final CLN model.

[0018] Further, the processing process of the crack video in step (1) is as follows:

[0019] Use the wall-climbing robot to obtain the crack video on the surface of the underwater concrete structure, denoted as Video-a. The crack video Video-a is disassembled into an image sequence in frame order, and the disassembled images are set as three channels.

[0020] Further, in step (2), the specific process of the CLN model processing is as follows:

[0021] Specifically, after the image is input, it is processed in two networks, one for high-level feature extraction and the other for low-level feature extraction.

[0022] The image enters the high-level feature extraction network, and its initial size is (W, H, 3), denoted as Image-a. After passing through a convolutional neural network, an image Image-b with a size of (W / 16, H / 16, 2048), an image Image-c with a size of (W / 2, H / 2, 2048), and an image Image-d with a size of (W / 4, H / 4, 2048) are obtained.

[0023] After passing through a convolutional neural network, the image is down-sampled by 2 four times, and the size becomes (W / 16, H / 16, 2048), denoted as Image-b, the image of size (W / 2, H / 2, 2048) down-sampled by 2 once in the process is denoted as Image-c, and the image of size (W / 4, H / 4, 2048) down-sampled by 2 twice in the process is denoted as Image-d, wherein (W, H) represents the width and height of the image, and 3 and 2048 represent the number of channels of the image;

[0024] Image-b enters the feature extraction pyramid network, which includes a global pooling layer, three dilated convolution layers with dilated rates of 6, 12 and 18 respectively, and a 1x1 convolution layer. The outputs of the five layers are spliced in the channel dimension, and then output an image Image-e of size (W / 16, H / 16, 256) through a 1x1 convolution layer;

[0025] Image-c and Image-d both enter the low-level feature extraction network, and are outputted as an image Image-f of size (W / 2, H / 2, 96) and an image Image-g of size (W / 4, H / 4, 96) respectively through a 1x1 convolution layer;

[0026] Image-e is bilinearly up-sampled by 4 to output an image Image-h of size (W / 4, H / 4, 256);

[0027] After splicing Image-g and Image-h in the channel dimension, they are bilinearly up-sampled by 2 to output an image Image-i of size (W / 2, H / 2, 352);

[0028] After splicing Image-i and Image-f in the channel dimension, they are bilinearly up-sampled by 2, and then pass through a 3x3 convolution layer to obtain an image Image-j of size (W, H, 3) with segmentation results;

[0029] For the cracks in Image-j, the highest point (W P1 , H P1 ), the lowest point (W P2 , H P2 ), the leftmost point (W P3 , H P3 ), and the rightmost point (W P4 , H P4 ) of the crack contour are taken, wherein (W Pi , H Pi ) represents the image coordinates corresponding to Pi points (i=1, 2, 3, 4), and (W P3, H P1 ), (W P4 , H P1 ), (W P3 , H P2 ), (W P4 , H P2 ) four points form a rectangular frame, denoted as Rec-a, count the number of crack pixels in the rectangular frame Rec-a, and the image Image-j is denoted as the image Image-k with the most crack pixels, and the remaining images Image-j are collectively denoted as the image Image-n;

[0030] All contents of the rectangular frame Rec-a corresponding to the image Image-k are propagated to all images Image-n in the form of label propagation to obtain the segmentation image Image-m of the final segmentation result.

[0031] Further, the accuracy of image segmentation is evaluated by the mIoU index, and the calculation formula of the mIoU index is:

[0032]

[0033] Wherein, P represents the predicted value, G represents the true value, and k represents the number of categories.

[0034] Further, the convolution method used by the dilated convolution layers with a dilated rate of 6, 12 and 18 in the feature extraction pyramid network is deformable convolution, which specifically includes:

[0035] The deformable convolution samples the input feature Figure X Y using an irregular convolution kernel R, and then performs weighted summation on the sampled values using a weight W to obtain an output feature map Y, wherein the output feature map Y is:

[0036]

[0037] Wherein, p0 represents the position of the pixel grid on the feature map Y;

[0038] p n represents the sampling position of the convolution kernel;

[0039] Δp n represents the sampling position offset.

[0040] Further, the label propagation specifically includes:

[0041] Due to the influence of bubbles, floating objects and aquatic organisms in the underwater environment, each frame of crack may be occluded, so it is necessary to select a single image with the most complete crack as the source of label propagation for nearby frames;

[0042] frame M t containing the complete crack segmentation result is propagated to the frame M t+i in which the crack is incomplete:

[0043] M t+i (x+ui, y+vi, (R i , G i , B i )) = M t (x, y, (R, G, B))

[0044] (u, v) = C(M t : M t+i )

[0045] wherein i represents the time elapsed from frame M t to frame M t+i ;

[0046] (x, y) represents the coordinates of any pixel point within the rectangular frame Rec-a;

[0047] (R i , G i , B i ) and (R, G, B) represent the RGB values corresponding to the pixel point;

[0048] (u, v) represents the motion vector of the wall-climbing robot;

[0049] C(M t : M t+i ) represents a convolutional neural network used to calculate the motion vector based on the input frame M t to frame M t+i .

[0050] Further, the specific process of step (3) is: using a skeleton extraction algorithm on the segmented image Image-m, reducing the width of the crack, extracting the skeleton of the crack and measuring the width of the crack, and the extracted skeleton map is recorded as Image-n.

[0051] The number of skeleton points contained in each crack in the skeleton map Image-n is counted as the length of each crack; the number, width and length of each crack in all images Image-j are written into a table Table-a.

[0052] Further, the specific implementation steps of the skeleton extraction algorithm in step (3) are:

[0053] The distance from any point inside the crack to the set of crack contour points is calculated, and if there are two equal minimum distances, the any point inside the crack is considered as a skeleton point. The Euclidean distance from the skeleton point to the crack contour is calculated, and the wrong skeleton point is corrected by calculating the difference of Euclidean distance. The Euclidean distance g from the skeleton point to the crack contour c (p, L) is:

[0054] g c (p, L) = min{g(p, n) | n ∈ L} (1)

[0055] where p represents any point inside the crack, L represents the crack contour, n represents any point on the crack contour L, and g c (·) and g(·) represent Euclidean distance.

[0056] Whether the skeleton point needs to be corrected is determined by formula (2):

[0057] Suppose that the correction judgment criteria of two adjacent skeleton points a and b are p a and p b , whether to correct is determined by the following formula:

[0058]

[0059] If p a = 0, it means that no correction is needed.

[0060] p a = 1 means that correction is needed, and the correction is performed according to formula (3):

[0061]

[0062] where g a1 and g a2 are two Euclidean distances from the skeleton point a to the crack contour L;

[0063] g b1 and g b2 are two Euclidean distances from the skeleton point b to the crack contour L;

[0064] a' represents the corrected skeleton point, g b1 (x a , y a ) and g b2 (x a , y a ) represent two intersection points obtained by intersecting the Euclidean distance calculation direction of the skeleton point b at the skeleton point a with the crack contour L.

[0065] Further, the specific steps of step (4) are:

[0066] The segmented image Image-m is input into a NeRF model to perform three-dimensional reconstruction on the underwater concrete structure, and a video with clear photos at any viewing angle is rendered, recorded as Video-b;

[0067] The table Table-a and the video Video-b are spliced together to obtain the final underwater concrete crack segmentation and quantification result.

[0068] The NeRF model is full name Neural Radiance Fields, which trains the NeRF model by inputting sparse multi-angle images with attitude, and renders clear photos at any viewing angle to achieve three-dimensional reconstruction effect.

[0069] The application provides a kind of computing device, including receiver, memory, processor and the computer program of processing data in memory running on processor, the crack image collected by wall climbing robot is received in real time by the receiver, the image received and the image output after detection are stored in memory, the computer program is implemented when the processor executes described underwater concrete crack segmentation method.

[0070] The application has the following beneficial effects:

[0071] (1) the application obtains crack video on underwater concrete by using wall climbing robot, and the video is disassembled into picture sequence and processed by using CLN model, to realize the prediction of video image and accurately master the overall damage information of structure surface.

[0072] (2) the application can expand the structure damage detection range to underwater according to the factors such as water flow action in water environment and interference of aquatic organisms, and the combination of three-dimensional reconstruction technology can complete the overall surface of structure, so as to realize the accurate recovery and disease detection of underwater concrete structure.

[0073] (3) the application has high segmentation accuracy for cracks on the surface of underwater concrete, can measure the geometric parameters of cracks in the image, and restores the three-dimensional structure of underwater concrete, so that the algorithm can provide complete damage information of structure, has strong practical application value, and has good engineering significance. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 is the flow chart of the application.

[0075] Figure 2 is the schematic diagram of CLN model network structure.

[0076] Figure 3 is the underwater original image one.

[0077] Figure 4 is the underwater original image two

[0078] Figure 5 is the segmentation result map of the CLN model output Figure 3

[0079] Figure 6 is the segmentation result map of the CLN model output Figure 4

[0080] Figure 7 is the label propagation schematic diagram.

[0081] Figure 8 is the deformable convolution schematic diagram.

[0082] Figure 9 is the effect schematic diagram of measuring the crack width by the improved centerline transformation algorithm of the application.

[0083] Figure 10 is the comprehensive result schematic diagram of the crack segmentation, quantification and structure three-dimensional reconstruction finally obtained by the application. DETAILED DESCRIPTION

[0084] The specific embodiments of the application will be described in detail below with reference to the accompanying drawings. It should be pointed out that the embodiments are only specific descriptions of the application and should not be regarded as limitations of the application. The purpose of the embodiments is to enable those skilled in the art to better understand and reproduce the technical solutions of the application. The protection scope of the application should still be limited by the scope defined in the claims.

[0085] As shown in Figure 1 , the application provides an underwater concrete crack segmentation method, which comprises:

[0086] S1, collecting crack video of underwater concrete structure surface by using wall-climbing robot and disassembling it into image sequence;

[0087] The specific steps are: using a wall-climbing robot to obtain crack video on the surface of an underwater concrete structure, setting the video duration to 2 minutes and the frame rate to 60 frames, denoted as Video-a, disassembling the crack video Video-a into image sequence in frame order, a total of 7200 images, the resolution is 1920x1080, and the disassembled images are set as three channels.

[0088] S2, inputting the image into a CLN model to output a segmentation image Image-m;

[0089] The specific steps of training the CLN model are:

[0090] (1) obtaining crack data, including data obtained on land and underwater;

[0091] (2) using Labelme program to perform pixel-level labeling on the crack;​​

[0092] (3) The data is divided into training set, validation set and test set in the ratio of 8:1:1, and input into the CLN model for training;

[0093] (4) The training result is verified, and the hyperparameter group of the model is adjusted;

[0094] (5) The weight of the best iteration in the training verification result is saved and set as the final CLN model.

[0095] The specific steps of step S2 are as follows:

[0096] After the image is input, it is processed in two networks respectively, one is a high-level feature extraction network, and the other is a low-level feature extraction network. The network structure of the model is as shown in Figure 2 .

[0097] The image enters the high-level feature extraction network, and the initial size is (W, H, 3), denoted as Image-a. After passing through a convolutional neural network, an image Image-b with a size of (W / 16, H / 16, 2048), an image Image-c with a size of (W / 2, H / 2, 2048), and an image Image-d with a size of (W / 4, H / 4, 2048) are obtained.

[0098] After passing through a convolutional neural network, the image is down-sampled by 2 times for 4 times, and the size becomes (W / 16, H / 16, 2048), denoted as Image-b. The image with a size of (W / 2, H / 2, 2048) that is down-sampled by 2 times in the process is denoted as Image-c, and the image with a size of (W / 4, H / 4, 2048) that is down-sampled by 2 times in the process is denoted as Image-d, wherein (W, H) represents the width and height of the image, and 3 and 2048 represent the number of channels of the image.

[0099] The image Image-b enters the feature extraction pyramid network, which includes a global pooling layer, three dilated convolution layers with dilated rates of 6, 12 and 18 respectively, and a 1x1 convolution layer. The outputs of the five layers are spliced in the channel dimension, and then output through a 1x1 convolution layer to obtain an image Image-e with a size of (W / 16, H / 16, 256).

[0100] The image Image-c and the image Image-d both enter the low-level feature extraction network, and pass through a 1x1 convolution layer to output an image Image-f with a size of (W / 2, H / 2, 96) and an image Image-g with a size of (W / 4, H / 4, 96) respectively.

[0101] Image-e is 4 times bilinear up-sampling, and an image Image-h with a size of (W / 4, H / 4, 256) is output;

[0102] Image-g and Image-h are spliced from the channel dimension, and then 2 times bilinear up-sampling is performed, and an image Image-i with a size of (W / 2, H / 2, 352) is output;

[0103] Image-i and Image-f are spliced from the channel dimension, and then 2 times bilinear up-sampling is performed, and a 3x3 convolution layer is used to obtain an image Image-j with a size of (W, H, 3) and a segmentation result, as shown in Figures 3-6

[0104] In this embodiment, 7200 images are used, of which 5384 images contain cracks, and 1816 images do not contain cracks. The segmentation prediction is performed on the 5384 images containing cracks, and the segmentation accuracy is evaluated by the mIoU index. The mIoU is calculated according to the following formula:

[0105] The accuracy of image segmentation is evaluated by the mIoU index, and the calculation formula of the mIoU index is:

[0106]

[0107] Where P represents the predicted value, G represents the true value, and k represents the number of categories.

[0108] According to statistics, the mIoU value obtained in this embodiment is 0.927, which is sufficient to replace manual work.

[0109] For the cracks in the image Image-j, the highest point (W P1 , H P1 ), the lowest point (W P2 , H P2 ), the leftmost point (W P3 , H P3 ), and the rightmost point (W P4 , H P4 ) of the crack contour are taken, where (W Pi , H Pi ) represents the image coordinates corresponding to the Pi point (i=1, 2, 3, 4), and (W P3 , H P1 ), (W P4 , H P1 ), (W P3 , H P2 ), and (W P4 , H P2 ​) four points form a rectangular frame, denoted as Rec-a, the number of pixels in the rectangular frame Rec-a is counted, and the image Image-j with the largest number of crack pixels is denoted as image Image-k, and the remaining images Image-j are collectively denoted as image Image-n;

[0110] The label propagation specific steps include:

[0111] Due to the influence of bubbles, floating objects and aquatic organisms in the underwater environment, each frame may be occluded, so it is necessary to select a single image with the most complete crack as the source of label propagation for nearby frames;

[0112] The frame M t containing the complete crack segmentation result is propagated to the frame M t+i with incomplete cracks:

[0113] M t+i (x+ui,y+vi,(R i ,G i ,B i ))=M r (x,y,(R,G,B))

[0114] (u,v)=C(M t :M r+i )

[0115] Where (x, y) represents the coordinates of any pixel point in the rectangular frame Rec-a;

[0116] (R i , G i , B i ) and (R, G, B) represent the RGB values corresponding to the pixel point;

[0117] (u, v) represents the motion vector of the wall-climbing robot;

[0118] C(M t : M t+i ) represents a convolutional neural network for calculating the motion vector based on the input frame M t to frame M t+i .

[0119] All contents of the rectangular frame Rec-a corresponding to the image Image-k are propagated to all images Image-n in the form of label propagation, to obtain the segmentation image Image-m of the final segmentation result, and the label propagation schematic diagram is shown in Figure 7 , wherein figure (a) represents no occlusion and no interruption of the crack; figure (b) represents occlusion and interruption of the crack, and figure (c) represents the propagation result.

[0120] The deformable convolution is used in the convolution layers with the hole rates of 6, 12 and 18 in the feature extraction pyramid network, and the deformable convolution specifically comprises the following steps:

[0121] The deformable convolution is used in the convolution layers with the hole rates of 6, 12 and 18 in the feature extraction pyramid network, and the deformable convolution specifically comprises the following steps: Figure X The deformable convolution is used in the convolution layers with the hole rates of 6, 12 and 18 in the feature extraction pyramid network, and the deformable convolution specifically comprises the following steps:

[0122]

[0123] Wherein, p0 represents the position of a pixel grid on the feature map Y; p n represents the sampling position of the convolution kernel; Δp n represents the sampling position offset; and a schematic diagram of the deformable convolution is shown in Figure 8 .

[0124] S3, using a skeleton extraction algorithm to quantize the size of the segmented image Image-m:

[0125] The specific process is as follows: using a skeleton extraction algorithm on the segmented image Image-m, reducing the width of the crack, extracting the skeleton of the crack and measuring the width of the crack, and the extracted skeleton map is recorded as Image-n.

[0126] The number of skeleton points contained in each crack in the skeleton map Image-n is counted as the length of each crack; and the number, width and length of each crack in all images Image-j are written into a table Table-a.

[0127] The specific implementation steps of the skeleton extraction algorithm are as follows:

[0128] The distance between any point inside the crack and the set of crack contour points is calculated, and if there are two equal minimum distances, it is considered that the any point inside the crack is a skeleton point. The Euclidean distance between the skeleton point and the crack contour is calculated, and the wrong skeleton point is corrected. The Euclidean distance g c (p, L) is:

[0129] g c (p, L) = min{g(p, n) | n e L} Equation (1)

[0130] Wherein, p represents any point inside the crack, L represents the crack contour, n represents any point on the crack contour L, g c (·) and g(·) represent the Euclidean distance;

[0131] Whether the skeleton point needs to be corrected is determined by equation (2):

[0132] Let two adjacent skeleton points a and b be p a and p b , whether the correction is needed according to the following formula:

[0133]

[0134] p a =0 indicates no correction is needed; p a =1 indicates that the correction is needed, and the correction is made according to formula (3):

[0135]

[0136] wherein g a1 and g a2 are two distances from the skeleton point a to the crack profile L;

[0137] g b1 and g b2 are two distances from the skeleton point b to the crack profile L;

[0138] a' indicates the corrected skeleton point, g b1 (x a ,y a ) and g b2 (x a ,y a ) indicate two intersection points obtained by intersecting the direction of the Euclidean distance calculation from the skeleton point a to the skeleton point b with the crack profile L.

[0139] The original centerline transformation algorithm (skeleton extraction algorithm) has the problem of extra branches when extracting the end crack skeleton, and the improved centerline transformation algorithm (skeleton extraction algorithm improved by formula (2)) solves this problem, and the specific effect is shown in Figure 9 .

[0140] In order to evaluate the measurement effect of the improved centerline transformation algorithm (skeleton extraction algorithm improved by formula (2)), 10 images containing cracks were randomly selected for measurement and error calculation, and the results are shown in Table 1. Compared with the true situation, the error of the measured value is about 0.6 pixels, and most of the error is about 0.3 pixels, and the result is relatively accurate.

[0141] Table 1

[0142]

[0143] S4, three-dimensional reconstruction of the underwater concrete structure by using a NeRF model; the NeRF model is full name of Neural Radiance Fields, a clear photo under any viewing angle is rendered by inputting sparse multi-angle images with poses to train the NeRF model, so as to achieve the effect of three-dimensional reconstruction.

[0144] The segmented image Image-m is input into the NeRF model, and the underwater concrete structure is three-dimensionally reconstructed, and a video with clear photos under any viewing angle is rendered, which is recorded as Video-b;

[0145] The table Table-a and the video Video-b are cut together to obtain the final underwater concrete crack segmentation and quantification result, and the finally obtained crack segmentation, quantification and structure three-dimensional reconstruction result schematic diagram is as shown in Figure 10 The six cracks are respectively marked as a, b, c, d, e and f, and compared with a reference (pixel-mm conversion standard) g, and the data after comparison is as shown in Table 2.

[0146] Table 2

[0147]

[0148] As shown in Table 2, the application can provide image-level information of all cracks on the structure surface, the actual maximum width, average width and length characteristics of the cracks can be calculated according to the pixel-mm conversion standard, and more practical calculation results are given; by Figure 10 Observing the distribution of the cracks on the structure surface can be used to infer the load type mainly borne by the structure, and is convenient for technicians to manage.

[0149] The application also provides a computing device, which comprises a receiver, a memory, a processor and a computer program running on the processor for processing data in the memory, the receiver receives crack images collected by the wall-climbing robot in real time, the memory stores the received images and output images after detection, and the processor implements the underwater concrete crack segmentation method described above when executing the computer program.

[0150] It should be noted that technical features not described in detail in the application can be realized by any existing technology.

[0151] Although the preferred embodiments of the application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all changes and modifications falling within the scope of the application.

Claims

1. A method for underwater concrete crack segmentation, characterized in that, The segmentation method includes: (1) Use a wall-climbing robot to collect video of cracks on the surface of underwater concrete structures and decompose them into image sequences; (2) Input the image into the CLN model to output the segmented image Image-m; (3) Use a skeleton extraction algorithm to quantize the Image-m size of the segmented image: (4) Three-dimensional reconstruction of underwater concrete structures using the NeRF model; The specific process of step (2) is as follows: The image enters the advanced feature extraction network with an initial size of (W, H, 3), denoted as Image-a. After passing through a convolutional neural network, it yields images Image-b (W / 16, H / 16, 2048), Image-c (W / 2, H / 2, 2048), and Image-d (W / 4, H / 4, 2048). Image Image-b enters the feature extraction pyramid network and outputs Image-e with a size of (W / 16, H / 16, 256); Images Image-c and Image-d are both fed into a low-level feature extraction network, which outputs Image-f with a size of (W / 2, H / 2, 96) and Image-g with a size of (W / 4, H / 4, 96), respectively. Perform a 4x bilinear upsampling on the image Image-e to output an image Image-h of size (W / 4, H / 4, 256); After concatenating the images Image-g and Image-h along the channel dimension, perform a 2x bilinear upsampling to output an image Image-i with a size of (W / 2, H / 2, 352); After concatenating images Image-i and Image-f along the channel dimension, perform double bilinear upsampling and pass through a 3×3 convolutional layer to obtain an image Image-j with segmentation results (W, H, 3); For the crack in image Image-j, take the highest point (W) of the crack contour. P1 H P1 ), lowest point (W) P2 H P2 ), the leftmost point (W) P3 H P3 ), the rightmost point (W) P4 H P4 ), of which (W Pi H Pi Let (i, 1, 2, 3, 4) represent the in-image coordinates of point Pi, where i = 1, 2, 3, 4. P3 H P1 ), (W P4 H P1 ), (W P3 H P2 ), (W P4 H P2 Four points form a rectangle, denoted as Rec-a. Count the number of crack pixels within the rectangle Rec-a. The image with the most crack pixels in Image-j is denoted as Image-k. The remaining images Image-j are collectively referred to as Image-n. The contents of the bounding box Rec-a corresponding to image Image-k are propagated to all images Image-n in the form of label propagation to obtain the final segmentation image Image-m; The specific steps for the label propagation include: Propagate the labels of frames containing complete crack segmentation results to frames with incomplete cracks: ， Where i represents frame M t To frame M t+i Time elapsed; (x,y) represents the coordinates of any pixel within the rectangle Rec-a; (R i G i B i (R,G,B) represent the RGB values ​​corresponding to the pixel. (u,v) represents the motion vector of the wall-climbing robot; C(M t :M t+i ) represents a computation based on input frame M t To frame M t+i A convolutional neural network for motion vectors.

2. The underwater concrete crack segmentation method according to claim 1, characterized in that, The processing procedure for step (1) is as follows: A wall-climbing robot was used to acquire video of cracks on the surface of an underwater concrete structure, denoted as Video-a. The crack video Video-a was decomposed into an image sequence in frame order, and the decomposed images were set to three channels.

3. The underwater concrete crack segmentation method according to claim 1, characterized in that, The accuracy of image segmentation is evaluated using the mIoU metric, which is calculated as follows: ， in, P Indicates the predicted value. G Represents the actual value. k Indicates the number of categories.

4. The underwater concrete crack segmentation method according to claim 1, characterized in that, The dilated convolutional layers in the feature extraction pyramid network with dilation rates of 6, 12, and 18 use deformable convolution, specifically including: Deformable convolution in input feature map X Above, use irregular convolution kernels. R Sampling, then using weights W The sampled values ​​are weighted and summed to obtain the output feature map. Y Output feature map Y for: ， in, Indicates the position of a pixel grid on the feature map Y; Indicates the sampling position of the convolution kernel; This indicates the sampling position offset.

5. The underwater concrete crack segmentation method according to claim 1, characterized in that, The specific process of step (3) is as follows: The skeleton extraction algorithm is used on the segmented image Image-m to reduce the width of the cracks, extract the skeleton of the cracks, and measure the width of the cracks. The extracted skeleton image is denoted as Image-n. Count the number of skeleton points contained in each crack in the skeleton image Image-n, and use this as the length of each crack; write the number, width, and length of each crack in all images Image-j into Table-a.

6. The underwater concrete crack segmentation method according to claim 5, characterized in that, The specific implementation steps of the skeleton extraction algorithm in step (3) are as follows: To find the distance from any point inside the crack to the set of points on the crack outline, if two equal minimum distances exist, then that point inside the crack is considered a skeleton point. For all skeleton points, calculate the Euclidean distance from each skeleton point to the crack outline, perform Euclidean distance difference calculations, and correct any erroneous skeleton points. for: Equation (1) Where p represents any point inside the crack, L represents the crack profile, and n represents any point on the crack profile L. and Indicates Euclidean distance; Whether the skeleton points need to be corrected is determined by equation (2): Suppose two adjacent skeleton points a and b The correction criterion is and Should the judgment criteria be revised as follows: Equation (2) if, This indicates that no correction is needed; This indicates that a correction is needed, and the correction should be made according to equation (3): Equation (3) in, and It is a skeleton point a Two Euclidean distances to the crack profile L; and It is a skeleton point b Two Euclidean distances to the crack profile L; This represents the corrected skeleton points; and Indicates at skeleton point a The two intersection points are obtained by intersecting the crack profile along the Euclidean distance calculation direction of the skeleton point.

7. The underwater concrete crack segmentation method according to claim 1, characterized in that, The specific steps of step (4) are as follows: The segmented image Image-m is input into the NeRF model to perform three-dimensional reconstruction of the underwater concrete structure and render a video with clear photos from any viewpoint, denoted as Video-b. By editing Table-a and Video-b together, the final underwater concrete crack segmentation and quantification results are obtained.

8. A computing device, characterized in that, The device includes a receiver, a memory, a processor, and a computer program running on the processor to process data in the memory. The receiver receives crack images collected by the wall-climbing robot in real time. The memory stores the received images and the images output after detection. When the processor executes the computer program, it implements the underwater concrete crack segmentation method according to any one of claims 1-7.

Images