Engineering structure crack detection and measurement method based on machine vision

Abstract

The invention belongs to the technical field of machine vision and deep learning, and discloses an engineering structure crack detection and measurement method based on machine vision, and the method comprises the steps: collecting original image data of an engineering structure, discretizing the original image data into image blocks, carrying out the preprocessing, building a crack classification model through a Resnet-50 neural network, and carrying out the training, the method comprises the following steps of: firstly, constructing an image block, identifying a crack image in the image block, then constructing a SegFormer segmentation model based on a Transformer framework to segment the crack image to obtain a crack mask graph, evaluating the performance of the two models, then analyzing the two models by adopting an orthogonal skeleton line method, obtaining a contour line and a skeleton line of a crack, and calculating the length of the crack. According to the method, the ResNet-50 image classification model and the SegFormer segmentation model are combined to realize high-precision crack identification and segmentation of the building structure crack, the contour line and the skeleton line of the crack are obtained and the length of the crack is measured by means of the orthogonal skeleton line method, and the crack of an engineering structure is well detected and subjected to risk assessment.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision and deep learning, and specifically relates to a method for detecting and measuring cracks in an engineering structure based on machine vision. Background Art

[0002] In the actual inspection process, the crack information collected by the inspectors varies according to different needs. For example, to determine whether the road surface or bridge deck needs to be repaired, it is necessary to measure the length of the crack. For another example, in tunnel inspection, measuring the width of the crack helps to determine the stability of the tunnel lining. In addition, the crack angle needs to be measured when analyzing the structural damage mechanism; the spatial distribution of the crack needs to be recorded when evaluating the overall health of the structure, and so on. Therefore, it is necessary to design a specific crack detection method according to specific inspection needs.

[0003] According to different detection tasks, researchers have proposed a variety of crack detection methods, such as crack classification. In essence, this classification task performs classification work at different granularity levels. Taking the image block as the basic unit, crack classification divides the image block into two types of image sets: images containing cracks or images without cracks. Taking a 100×100 pixel image block as an example, the segmentation model needs to perform 10,000 binary classification tasks. From the perspective of computational efficiency, in actual engineering crack detection tasks, classification tasks based on image scale levels are less efficient in the crack detection process. In actual engineering applications at this stage, crack detection methods based on machine vision are generally used, but there are still some technical difficulties in this method. Among them, the outstanding problems can be summarized into three aspects: 1) The crack detection accuracy is not high; 2) The crack detection efficiency is insufficient; 3) It is difficult to balance the detection accuracy and detection efficiency. Summary of the invention

[0004] In view of the shortcomings of the background technology and taking into account the characteristics of remote sensing ship images, the present invention proposes a method for detecting and measuring cracks in engineering structures based on machine vision, and the steps are as follows: Step S1, collecting original image data of the engineering structure, and discretizing the original image data of the engineering structure into a plurality of small-sized image blocks; Step S2, integrating the image blocks into a data set and preprocessing the data set; Step S3, using Resnet-50 as the basic neural network to build a crack classification model to extract image feature values ​​for identifying crack conditions of image blocks; Step S4, training the crack classification model; Step S5, identifying the crack image through the crack classification model, building a SegFormer segmentation model based on the Transformer architecture to segment the crack image and obtain a crack mask map; Step S6, evaluating the crack classification model and the SegFormer segmentation model; Step S7, using the orthogonal skeleton line method to analyze the crack mask image, obtain the contour line and skeleton line of the crack, and calculate the crack length.

[0005] Preferably, the discretization process in step S1 is as follows: a sliding window model is used to discretize the high-resolution original image of the engineering structure into a plurality of small-sized image blocks of 64×64 pixels.

[0006] Preferably, the process of data set preprocessing in step S2 is as follows: First, the image block is subjected to denoising. A Gaussian filter is used to smooth the image block to reduce high-frequency noise. The parameters of the Gaussian filter are Set to , indicating that the standard deviation of the filter is small, which can effectively remove the fine noise in the image. The calculation formula of the Gaussian filter is as follows: ; In addition, a median filter with a window size of 3x3 is used to process the image. The mutation noise is eliminated by replacing the value of the central pixel with the median value of the pixels in the window. The calculation formula of the median filter is as follows: ; Finally, data enhancement methods are used to enhance various features of cracks.

[0007] Preferably, the process of constructing the crack classification model in step S3 is as follows: 1) The preprocessed small-size image blocks of 64×64 pixels are input into the input layer of the Resnet-50 neural network that receives raw data. Each image block contains three color channels with a dimension of 64×64×3, that is, each image block is a three-channel RGB image of 64×64 pixels. The use of small-size image blocks can better meet the needs of crack detection; 2) The convolution layer in the ResNet-50 neural network is used to extract local features in the image block, which is the initial stage of feature extraction. The convolution layer uses a convolution kernel of 3x3 size and depth 64, and then uses a nonlinear ReLU activation function to extract nonlinear features. The ReLU activation function can retain positive eigenvalues ​​while removing redundant information, so that the neural network can better process complex nonlinear data; 3) Three residual blocks in the ResNet-50 neural network are used to extract feature information from image blocks. Each residual block contains two 3x3 convolutional layers and an identity map, and residual connections are used to solve the gradient vanishing problem. That is, each residual block uses a skip connection to add the input directly to the output for residual connection, so that the neural network can be trained at a deeper level. 4) Through a fully connected layer containing 128 neurons, the features are nonlinearly transformed and combined using the ReLU activation function, further integrating the features extracted from the residual block to prepare for the classification task; 5) The output layer of the ResNet-50 neural network contains two neurons, corresponding to the classification of cracks and non-cracks respectively. The probability of each category is output through the Softmax activation function to build a crack classification model; 6) Use the cross entropy loss function for error fitting. The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the probability distribution of the true label to optimize the classification performance of the model. The calculation formula of the cross entropy loss function is as follows: ; Where N is the number of samples, is the sample label, is the probability predicted by the model.

[0008] Preferably, the process of training the crack classification model in step S4 is as follows: Adam is used as the optimizer to train the crack classification model, and some hyperparameters are set during the training process, namely: the learning rate is set to 0.001, the batch size is set to 32, and the training cycle is set to 200 rounds; wherein, the training data set is divided into a training data set and a validation data set, and the division is carried out in proportion, and the ratio of the divided training data set to the validation data set is 8:2, so as to ensure the generalization ability of the model.

[0009] Preferably, the process of constructing the SegFormer segmentation model based on the Transformer architecture in step S5 is as follows: 1) Use the crack classification model to identify whether the small-size image block contains crack information. When the original image block of the identified engineering structure does not contain crack information, the image block is the background image; when the original image block of the identified engineering structure contains crack information, the image block is regarded as part of the pixel points in the image area, also called crack pixels or crack images; when the size of the image block is small, it can be considered that the image block contains crack information; 2) The input crack image is divided into multiple small blocks with the help of the encoder. Each small block is processed by multi-head self-attention and feedforward neural network in the local range, which can effectively extract the features in the image block; then the local features are gradually integrated through the hierarchical structure to obtain the global information of the whole image; the local features are then passed to the decoder stage, and the decoder classifies each pixel and divides it into crack pixels and background pixels; crack recognition is performed on the segmented image blocks to generate a crack mask map of crack morphological information; 3) The combined loss function of the cross entropy loss function and the Dice loss function is used to fit the error of the crack segmentation model. The calculation formula of the cross entropy loss function is as follows: ; in, is the number of pixels, is the number of categories, It is pixels in the category The real label on For the pixels are predicted to be of class The probability of The calculation formula of Dice loss function is as follows: ; in, It is The true label of each pixel, It is The probability that a pixel is predicted to be a positive class; The combined loss function is the weighted sum of the cross entropy loss function and the Dice loss function. The calculation formula of the combined loss function is as follows: ; in, is the weight coefficient used to balance the contribution of cross entropy loss and Dice loss.

[0010] Preferably, the evaluation process in step S6 is as follows: 1) The crack classification model is evaluated by classification accuracy: In the crack image binary classification task, the accuracy can measure the model's ability to identify the category to which the image belongs. The specific calculation formula for the accuracy is as follows: ; Among them, TP is true positive, which indicates the number of positive samples correctly predicted by the model, TN is true negative, which indicates the number of negative samples correctly predicted by the model, FP is false positive, which indicates the number of positive samples incorrectly predicted by the model, and FN is false negative, which indicates the number of negative samples incorrectly predicted by the model; 2) The crack segmentation model is evaluated using precision, intersection-over-union ratio, recall rate, and F1-score: Pixel accuracy is the ratio of the number of correctly classified crack pixels to the total number of pixels in the image domain. The calculation formula for pixel accuracy is as follows: ; The intersection-and-union ratio quantifies the relationship between the intersection and union of the predicted crack area and the actual crack area. The calculation formula of the intersection-and-union ratio is: ; The recall rate is the proportion of real crack pixels that are correctly predicted as cracks. The calculation formula of the recall rate is: ; F1-score is the harmonic mean of precision and recall. The calculation formula of F1-score is: .

[0011] Preferably, the process of calculating the orthogonal skeleton method in step S7 is as follows: 1) Contour extraction Using Canny edge detection or Sobel operator to extract the contour of the crack can accurately describe the shape of the crack; 2) Skeleton extraction Thinning is performed on the binary image to gradually shrink the crack area and retain its center line. Commonly used thinning algorithms include Zhang-Suen thinning algorithm and Guo-Hall thinning algorithm. The thinning result is then used to generate the skeleton line of the crack. 3) Length calculation For pixels in the horizontal and vertical directions, the distance is , for pixels in the diagonal direction, the distance is Then, the distances between all pixels are accumulated to obtain the total length of the crack, which is the skeleton line. Let the sequence of pixels on the skeleton line be , skeleton length The calculation formula is: ; in, .

[0012] Compared with the prior art, the present invention proposes a method for detecting and measuring cracks in engineering structures based on machine vision. The present invention has the following advantages: 1) This paper proposes a method for detecting and measuring cracks in engineering structures based on machine vision. Combining the ResNet-50 image classification model and the SegFormer segmentation model, this method can achieve high-precision crack recognition and segmentation of building structure cracks, aiming to improve the accuracy and real-time performance of crack detection and achieve quantitative description of crack images. 2) The small-scale image block classification method proposed in this paper effectively distinguishes crack image blocks from non-crack image blocks through the ResNet-50 model, achieving high accuracy and recall rate; the segmentation model based on SegFormer realizes accurate segmentation of cracks at different scales; 3) The present invention demonstrates high efficiency and accuracy in quantitative crack detection, significantly improves the automation level of crack detection, and reduces the time and cost of manual detection; the present invention is not only suitable for crack detection in buildings and bridges, but can also be extended to other structures such as tunnels, pipelines, spacecraft and other fields, providing technical support for structural health management in these fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a specific flow chart of the engineering structure crack detection and measurement method based on machine vision of the present invention; Figure 2 It is a specific production flow chart of the data set of the present invention; Figure 3 This is a training curve diagram of the classification model based on the ResNet-50 neural network for the training image set of the present invention; Figure 4 A prediction result diagram of the classification model test for the original structural image that has not participated in the training of the present invention; Figure 5 This is a training curve diagram of the crack segmentation model based on the SegFormer architecture for the training image of the present invention; Figure 6 A prediction result diagram of the crack segmentation model test performed on the original structural image that has not participated in the training of the present invention; Figure 7 The skeleton line graph of the cracks extracted by the present invention based on the Canny edge detection and contour tracing algorithm; Figure 8 It is a crack morphology detail image based on Canny edge detection and contour tracking algorithm of the present invention; Fig. 9 This is a diagram of crack measurement results based on the orthogonal skeleton line method of the present invention. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present application to further clearly and completely describe the technical solutions in the embodiments of the present application. It should be noted that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0015] In order to make the invention objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application are further described in detail in conjunction with the drawings in the specification: In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the advantages of the present invention will be further illustrated by comparing the embodiments in conjunction with the drawings and specific implementation methods.

[0016] The present invention proposes a method for detecting and measuring cracks in engineering structures based on machine vision. The flowchart of the method is as follows: Figure 1 As shown, the steps of this method are described in detail: Step S1, collecting original image data of the engineering structure, and discretizing the original image data of the engineering structure into a plurality of small-sized image blocks; Furthermore, the discretization process in step S1 is as follows: the sliding window model discretizes the high-resolution original image of the engineering structure into a plurality of small-sized image blocks of 64×64 pixels.

[0017] Step S2, integrating the image blocks into a data set and preprocessing the data set; Furthermore, the process of data set preprocessing in step S2 is as follows: Data enhancement technology is introduced in the data preprocessing stage to improve robustness and generalization ability. Data enhancement generates diversified training samples and expands the data set size by performing a series of transformation operations on image blocks, thereby preventing overfitting and improving its adaptability in different scenarios. First, the image block is subjected to denoising. A Gaussian filter is used to smooth the image block to reduce high-frequency noise. The parameters of the Gaussian filter are Set to , indicating that the standard deviation of the filter is small, which can effectively remove the fine noise in the image. The calculation formula of the Gaussian filter is as follows: ; Secondly, the image is processed using a median filter with a window size of 3x3. The mutation noise is eliminated by replacing the value of the central pixel with the median value of the pixels in the window. The calculation formula of the median filter is as follows: ; Finally, the data enhancement methods used include rotation, flipping, and random cropping. The image blocks are randomly rotated with a rotation angle between ±15 degrees. The rotation operation can increase the diversity of the image and simulate the crack morphology at different angles. The calculation formula for random rotation is as follows: ; The image block is flipped horizontally and vertically to generate a mirror image. The flipping operation can simulate the distribution of cracks in different directions. The calculation formulas for horizontal and vertical flipping are as follows: ; The image blocks are randomly cropped, and the cropping size is between 90% and 110% of the original image size. The cropping operation can simulate the crack characteristics at different scales. The calculation formula for random cropping is as follows: ; Through the above data enhancement method, the size of the data set was ultimately increased by 3 times, significantly improving the diversity and richness of training samples. The enhanced data can better learn the various characteristics of cracks and improve the accuracy and robustness in practical applications.

[0018] Step S3, using Resnet-50 as the basic neural network to build a crack classification model to extract image feature values ​​for identifying crack conditions of image blocks; Furthermore, the process of constructing the crack classification model in step S2 is as follows: 1) The preprocessed small-size image blocks of 64×64 pixels are input into the input layer of the Resnet-50 neural network that receives raw data. Each image block contains three color channels (RGB) and has a dimension of 64×64×3, that is, each image block is a three-channel RGB image of 64×64 pixels. The use of small-size image blocks can better meet the needs of crack detection; 2) The convolution layer in the ResNet-50 neural network is used to extract local features in the image block, which is the initial stage of feature extraction. The convolution layer uses a convolution kernel of 3x3 size and depth 64. The convolution operation is linear in nature, and the ReLU activation function is required to extract nonlinear features. The ReLU activation function can retain positive eigenvalues ​​while removing redundant information, so that the neural network can better process complex nonlinear data; 3) Three residual blocks in the ResNet-50 neural network are used to extract feature information from image blocks. Each residual block contains two 3x3 convolutional layers and an identity map, and residual connections are used to solve the gradient vanishing problem. That is, each residual block uses a skip connection to add the input directly to the output for residual connection, so that the neural network can be trained at a deeper level, thereby maintaining the depth and complexity of the model; 4) Through a fully connected layer containing 128 neurons, the features are nonlinearly transformed and combined using the ReLU activation function, further integrating the features extracted from the residual block to prepare for the classification task; 5) The output layer of the ResNet-50 neural network contains two neurons, corresponding to the classification of cracks and non-cracks respectively. The probability of each category is output through the Softmax activation function to build a crack classification model; 6) Use the cross entropy loss function for error fitting. The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the probability distribution of the true label to optimize the classification performance of the model. The calculation formula of the cross entropy loss function is as follows: ; Where N is the number of samples, is the sample label, is the probability predicted by the model.

[0019] Step S4, training the crack classification model; Furthermore, the process of training the crack classification model in step S3 is as follows: Adam is used as the optimizer to train the crack classification model, and some hyperparameters are set during the training process, namely: the learning rate is set to 0.001, the batch size is set to 32, and the training cycle is set to 200 rounds; wherein, the training data set is divided into a training data set and a validation data set, and the division is carried out in proportion, and the ratio of the divided training data set to the validation data set is 8:2, so as to ensure the generalization ability of the model.

[0020] S5, identify the crack image through the crack classification model, build the SegFormer segmentation model based on the Transformer architecture to segment the crack image and obtain the crack mask map; Furthermore, the process of constructing the SegFormer segmentation model based on the Transformer architecture in step S4 is as follows: 1) Use the crack classification model to identify whether the small-size image block contains crack information. When the original image block of the identified engineering structure does not contain crack information, the image block is the background image; when the original image block of the identified engineering structure contains crack information, the image block is regarded as part of the pixel points in the image area, also called crack pixels or crack images; when the size of the image block is small, it can be considered that the image block contains crack information; 2) SegFormer is based on the Transformer architecture, and its core is the encoder-decoder structure. The encoder is used to segment the input crack image into multiple small blocks. The encoder consists of a multi-head self-attention and feedforward neural network. Each small block is processed by multi-head self-attention and feedforward neural network in a local range, which can effectively extract the features in the image block. Then, the local features are gradually integrated through a hierarchical structure to obtain the global information of the entire image. The local features are then passed to the decoder stage, which classifies each pixel and divides it into crack pixels and background pixels. Cracks are identified on the segmented image blocks to generate crack mask images of crack morphological information. 3) The combined loss function of the cross entropy loss function and the Dice loss function is used to fit the error of the crack segmentation model. The calculation formula of the cross entropy loss function is as follows: ; in, is the number of pixels, is the number of categories, It is Pixels in category The real label on For the pixels are predicted to be of class The probability of The calculation formula of Dice loss function is as follows: ; in, It is The true label of each pixel, It is The probability that a pixel is predicted to be a positive class; The combined loss function is the weighted sum of the cross entropy loss function and the Dice loss function. The calculation formula of the combined loss function is as follows: ; in, is the weight coefficient used to balance the contribution of cross entropy loss and Dice loss.

[0021] S6, evaluation of the crack classification model and the SegFormer segmentation model; Furthermore, the evaluation process in step S5 is as follows: 1) The classification accuracy is used to evaluate the crack classification model: the classification accuracy is an intuitive and effective reference indicator; in the crack image binary classification task (dividing the crack image into two categories: crack area and non-crack area), the accuracy can measure the model's ability to identify the category to which the image belongs; the specific calculation formula for the accuracy is as follows: ; Among them, TP is true positive, which indicates the number of positive samples correctly predicted by the model, TN is true negative, which indicates the number of negative samples correctly predicted by the model, FP is false positive, which indicates the number of positive samples incorrectly predicted by the model, and FN is false negative, which indicates the number of negative samples incorrectly predicted by the model; 2) For crack segmentation models, accuracy, intersection over union (IOU), recall rate and F1-score are often used for evaluation: Pixel accuracy is the ratio of the number of correctly classified crack pixels to the total number of pixels in the image domain. The calculation formula for pixel accuracy is as follows: ; The intersection-over-union (IOU) quantifies the relationship between the intersection and union of the predicted crack area and the actual crack area. The calculation formula of the intersection-over-union is: ; Recall is the ratio of real crack pixels that are correctly predicted as cracks. The calculation formula of recall is: ; F1-score is the harmonic mean of precision and recall. The calculation formula of F1-score is: ; S7, using the orthogonal skeleton line method to analyze the crack mask image, obtain the contour line and skeleton line of the crack, and calculate the crack length; Furthermore, the process of calculating the orthogonal skeleton line method in step S6 is as follows: On the basis of crack detection and segmentation, the cracks are quantitatively measured. The method can use the orthogonal skeleton method to analyze the crack mask image, obtain the contour line and skeleton line of the crack, and calculate the crack length. The orthogonal skeleton method is an image processing technology used to extract the skeleton of an object in a binary image. The skeleton refers to the central axis of the object after gradual refinement. It can retain the topological structure of the object and remove redundant information, thereby simplifying complex shapes. By extracting the skeleton line of the crack, the length and morphological characteristics of the crack can be accurately measured. The specific implementation steps of the orthogonal skeleton method are as follows: 1) Contour extraction Use Canny edge detection or Sobel operator to extract the contour line of the crack. The contour line refers to the boundary between the crack area and the background area, which can accurately describe the shape of the crack. 2) Skeleton extraction Thinning is performed on the binary image to gradually shrink the crack area and retain its center line. Commonly used thinning algorithms include Zhang-Suen thinning algorithm and Guo-Hall thinning algorithm. The thinning result is then used to generate the skeleton line of the crack. 3) Length calculation The skeleton line is composed of a series of connected pixels. The length calculation can be achieved by accumulating the distances of these pixels. For pixels in the horizontal and vertical directions, the distance is , for pixels in the diagonal direction, the distance is Then, the distances between all pixels are accumulated to get the total length of the crack; let the pixel sequence on the skeleton line be , skeleton length The calculation formula is: ; in, .

[0022] Specific embodiments of the practical experiment of the present invention: The computational model was built using Python language and PyTorch deep learning platform, and the effectiveness and reliability of the above detection method were verified by combining images of concrete spalling and exposed steel bars in actual engineering structures. To ensure accuracy and consistency, the training, testing and prediction of the detection model were all carried out in the same hardware environment. The computing device used was an Intel(R) Xeon(R) W-2123 processor, RAM 64.0 GB, CPU @3.60GHz, and GPU NVIDIA GeForce RTX 2080 Ti. 1) Collecting Datasets Crack images of different structural styles and different material types are collected through three methods: public datasets, online retrieval, and on-site photography. These include images of reinforced concrete pavement cracks, images of concrete structure surface cracks, and images of steel structure surface cracks. Regardless of whether images from public datasets or selfies taken by researchers are used, these RGB images need to be manually annotated. Mask annotation tools are used to produce crack morphologies of engineering structure images for training classification and segmentation models. Due to the computing bottleneck of hardware equipment, the image resolution of engineering structures varies depending on the shooting equipment. In semantic segmentation, the photos input to the crack segmentation model need to be cut into small-sized image blocks. For classification tasks, small-sized image blocks are artificially divided into background images and crack images based on the presence or absence of crack information. The specific production process of the dataset is as follows: Figure 2 As shown; A total of 200 high-resolution crack images were collected from a construction site and bridge structure, with an image resolution of 4032x3024. Two public crack detection datasets, Concrete Crack Images for Classification (CCIC) and SDNET2018, were used to expand the number of samples. According to the production process of the above data samples, the dataset shown in Table 1 was generated. Table 1 Datasets for training models

[0023] 2) Image block classification results After preprocessing the data set, a classification model based on the ResNet-50 neural network is used to distinguish the background and cracks in the surface image of the engineering structure. The training history curve of the classifier used is as follows: Figure 5 As shown; during the training process, the optimal model can be saved by monitoring the loss value on the validation set. Figure 3 From the training curve of the classification model, it can be seen that the concrete image classifier began to converge from the 10th cycle, and the loss values ​​of the training set and the test set alternately led, and generally stabilized at around 0.05. Taking the classification accuracy as the evaluation indicator of the quality of the training model, when the model was iterated 7 times, the classification accuracy of the training set and the validation set had reached above 0.96. With the increase in the number of iterations, the classification accuracy of concrete images has approached 0.99; for the steel structure crack classification model, the loss function curve of the model tended to converge after 15 iterations, and the classification accuracy curve reached above 0.95 after 10 iterations; compared with concrete cracks, the morphology of steel structure cracks is more subtle, which makes it more difficult for the ResNet-50 network to extract crack features; From the training results of the model, it can be seen that the classifier can maintain a high classification accuracy for the binary classification task at the image level (dividing the crack image into two categories: crack area and non-crack area), which provides guarantee and support for the subsequent crack angle measurement and accurate segmentation of crack morphology; The original structure images that were not involved in the training were selected to test the performance of the classification model. The test results are as follows: Figure 4 As shown in the figure, from the prediction results, it can be seen that the original image is divided into multiple 64×64 image blocks, the image blocks that are judged to contain cracks are retained in the corresponding positions of the original image, and the image blocks that are considered not to contain cracks are filled with solid colors. The classified original image is pixel-binarized to obtain a crack-sensitive area map, which reflects the approximate location of the crack, and then the morphological algorithm is used to fit the approximate extension direction of the crack; 3) Crack segmentation results During the model training and testing process, the performance of the segmentation model is evaluated using indicators such as Precision, IoU, Recall, F1-score, and loss function convergence value. The training curve of the crack segmentation model is shown in Figure 2. Figure 5 As shown in the figure, from the trend of the loss function curve, the concrete crack model gradually began to converge from the 46th training cycle. Finally, the convergence value of the loss function on the training set finally approached 0.038, which was lower than 0.073 on the validation set. The intersection-over-union ratio reflects the segmentation performance of the model to a certain extent. When the model converges, the intersection rates of the training set and the validation set of the concrete crack segmentation model reach 0.943 and 0.896 respectively. The original structure image that was not involved in the training was selected to test the performance of the segmentation model. The test results are as follows: Figure 6 As shown in the figure, from the prediction results, it can be seen that the crack morphology of the original image is completely segmented, and the predicted crack binary image can basically reflect the true appearance of the crack, including details such as spatial position, morphology, length and width; 4) Crack segmentation results The segmented crack binary image will be further processed. Through the Canny edge detection and contour tracing algorithm, the extracted crack contour line has high clarity. The edge detection accurately captures the boundary of the crack, and the contour tracing algorithm can fully depict the crack morphology, such as Figure 7 and Figure 8 As shown in the figure, under different image resolutions and noise levels, the Canny edge detection algorithm shows high robustness and stability; The orthogonal skeleton method is used to accurately measure the crack length by identifying the contour lines and skeleton lines in the crack binary image. The refined skeleton line can accurately reflect the central path of the crack, eliminate unnecessary branches and noise points, and ensure the smoothness and continuity of the skeleton line. After processing multiple crack images, the extracted crack contour line can clearly depict the shape and direction of the crack, such as Fig. 9 As shown in Figure 2, the measured crack width and crack skeleton length are statistically analyzed in Fig. 9 On the right side, due to the slender shape of the crack, the present invention records the top three width values ​​of the crack width; the experimental results show that the model still maintains a high detection accuracy under complex background and illumination changes, which is significantly better than the traditional method.

[0024] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0025] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting and measuring cracks in engineering structures based on machine vision, characterized in that: include: Step S1, collecting original image data of the engineering structure, and discretizing the original image data of the engineering structure into a plurality of small-sized image blocks; Step S2, integrating the image blocks into a data set and preprocessing the data set; Step S3, using Resnet-50 as the basic neural network to build a crack classification model to extract image feature values ​​for identifying crack conditions of image blocks; Step S4, training the crack classification model; Step S5, identifying the crack image through the crack classification model, building a SegFormer segmentation model based on the Transformer architecture to segment the crack image and obtain a crack mask map; Step S6, evaluating the crack classification model and the SegFormer segmentation model; Step S7, using the orthogonal skeleton line method to analyze the crack mask image, obtain the contour line and skeleton line of the crack, and calculate the crack length.

2. The method for detecting and measuring cracks in an engineering structure based on machine vision according to claim 1, characterized in that: The discretization process in step S1 is as follows: a sliding window model is used to discretize the high-resolution original image of the engineering structure into a plurality of small-size image blocks of 64×64 pixels.

3. The method for detecting and measuring cracks in engineering structures based on machine vision according to claim 1, characterized in that: The process of data set preprocessing in step S2 is as follows: First, the image block is subjected to denoising. A Gaussian filter is used to smooth the image block to reduce high-frequency noise. The parameters of the Gaussian filter are Set to , indicating that the standard deviation of the filter is small, which can effectively remove the fine noise in the image. The calculation formula of the Gaussian filter is as follows: ； Secondly, the image is processed using a median filter with a window size of 3x3. The mutation noise is eliminated by replacing the value of the central pixel with the median value of the pixels in the window. The calculation formula of the median filter is as follows: ； Finally, data enhancement methods are used to enhance various features of cracks.

4. The method for detecting and measuring cracks in engineering structures based on machine vision according to claim 1, characterized in that: The process of constructing the crack classification model in step S3 is as follows: 1) The preprocessed small-size image blocks of 64×64 pixels are input into the input layer of the Resnet-50 neural network that receives raw data. Each image block contains three color channels with a dimension of 64×64×3, that is, each image block is a three-channel RGB image of 64×64 pixels. The use of small-size image blocks can better meet the needs of crack detection; 2) The convolution layer in the ResNet-50 neural network is used to extract local features in the image block, which is the initial stage of feature extraction. The convolution layer uses a convolution kernel of 3x3 size and depth 64, and then uses a nonlinear ReLU activation function to extract nonlinear features. The ReLU activation function can retain positive eigenvalues ​​while removing redundant information, so that the neural network can better process complex nonlinear data; 3) Three residual blocks in the ResNet-50 neural network are used to extract feature information from image blocks. Each residual block contains two 3x3 convolutional layers and an identity map, and residual connections are used to solve the gradient vanishing problem. That is, each residual block uses a skip connection to add the input directly to the output for residual connection, so that the neural network can be trained at a deeper level. 4) Through a fully connected layer containing 128 neurons, the features are nonlinearly transformed and combined using the ReLU activation function, further integrating the features extracted from the residual block to prepare for the classification task; 5) The output layer of the ResNet-50 neural network contains two neurons, corresponding to the classification of cracks and non-cracks respectively. The probability of each category is output through the Softmax activation function to build a crack classification model; 6) Use the cross entropy loss function for error fitting. The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the probability distribution of the true label to optimize the classification performance of the model. The calculation formula of the cross entropy loss function is as follows: ； Where N is the number of samples, is the sample label, is the probability predicted by the model.

5. The method for detecting and measuring cracks in engineering structures based on machine vision according to claim 1, characterized in that: The process of training the crack classification model in step S4 is as follows: Adam is used as the optimizer to train the crack classification model, and some hyperparameters are set during the training process, namely: the learning rate is set to 0.001, the batch size is set to 32, and the training cycle is set to 200 rounds; wherein, the training data set is divided into a training data set and a validation data set, and the division is carried out in proportion, and the ratio of the divided training data set to the validation data set is 8:2, so as to ensure the generalization ability of the model.

6. The method for detecting and measuring cracks in engineering structures based on machine vision according to claim 1, characterized in that: The process of constructing the SegFormer segmentation model based on the Transformer architecture in step S5 is as follows: 1) Use the crack classification model to identify whether the small-size image block contains crack information. When the original image block of the identified engineering structure does not contain crack information, the image block is the background image; when the original image block of the identified engineering structure contains crack information, the image block is regarded as part of the pixel points in the image area, also called crack pixels or crack images; when the size of the image block is small, it can be considered that the image block contains crack information; 2) The input crack image is divided into multiple small blocks with the help of the encoder. Each small block is processed by multi-head self-attention and feedforward neural network in a local range, which can effectively extract the features in the image block; Then, local features are gradually integrated through a hierarchical structure to obtain global information of the entire image. The local features are then passed to the decoder stage, which classifies each pixel into crack pixels and background pixels. Cracks are identified on the segmented image blocks to generate crack mask images with crack morphology information. 3) The combined loss function of the cross entropy loss function and the Dice loss function is used to fit the error of the crack segmentation model. The calculation formula of the cross entropy loss function is as follows: ； in, is the number of pixels, is the number of categories, It is Pixels in category The real label on For the pixels are predicted to be of class probability; The calculation formula of Dice loss function is as follows: ； in, It is The true label of each pixel, It is The probability that a pixel is predicted to be a positive class; The combined loss function is the weighted sum of the cross entropy loss function and the Dice loss function. The calculation formula of the combined loss function is as follows: ； in, is the weight coefficient used to balance the contribution of cross entropy loss and Dice loss.

7. The method for detecting and measuring cracks in engineering structures based on machine vision according to claim 1, characterized in that: The evaluation process in step S6 is as follows: 1) The crack classification model is evaluated by classification accuracy: In the crack image binary classification task, the accuracy can measure the model's ability to identify the category to which the image belongs. The specific calculation formula for the accuracy is as follows: ； Among them, TP is true positive, which indicates the number of positive samples correctly predicted by the model, TN is true negative, which indicates the number of negative samples correctly predicted by the model, FP is false positive, which indicates the number of positive samples incorrectly predicted by the model, and FN is false negative, which indicates the number of negative samples incorrectly predicted by the model; 2) The crack segmentation model is evaluated using precision, intersection-over-union ratio, recall rate, and F1-score: Pixel accuracy is the ratio of the number of correctly classified crack pixels to the total number of pixels in the image domain. The calculation formula for pixel accuracy is as follows: ； The intersection-and-union ratio quantifies the relationship between the intersection and union of the predicted crack area and the actual crack area. The calculation formula of the intersection-and-union ratio is: ； The recall rate is the proportion of real crack pixels that are correctly predicted as cracks. The calculation formula of the recall rate is: ； F1-score is the harmonic mean of precision and recall. The calculation formula of F1-score is: 。 8. The method for detecting and measuring cracks in engineering structures based on machine vision according to claim 1, characterized in that: The process of calculating the orthogonal skeleton method in step S7 is as follows: 1) Contour extraction Using Canny edge detection or Sobel operator to extract the contour of the crack can accurately describe the shape of the crack; 2) Skeleton extraction Perform thinning operation on the binary image to gradually shrink the crack area and retain its center line. Commonly used thinning algorithms include Zhang-Suen thinning algorithm and Guo-Hall thinning algorithm. The thinning result is then used to generate the skeleton line of the crack. 3) Length calculation For pixels in the horizontal and vertical directions, the distance is , for pixels in the diagonal direction, the distance is Then, the distances between all pixels are accumulated to obtain the total length of the crack, which is the skeleton line. Let the sequence of pixels on the skeleton line be , skeleton length The calculation formula is: ； in, .

Images