Concrete dam crack detection method and system based on long short-term memory network
Patent Information
- Application Number
- CN202311537782.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-17
- Publication Date
- 2026-10-30
- Estimated Expiration
- 2043-11-17
AI Technical Summary
[0006]本发明旨在解决目前存在的大坝裂缝检测方法裂缝定位不准确、漏检、精度不高等问题，针对此，提供一种基于长短期记忆网络(Long Short-Term Memory，LSTM)的混凝土大坝裂缝检测方法和系统
[0027]本发明的优点是：考虑到已有的目标检测方法定位不准确的问题，则使用了切片分类的方法。同时利用裂缝的方向性和连续性，组成切片序列，使用LSTM来进行序列检测，提升裂缝识别的准确度。
Smart Images
Figure CN117475140B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of target detection, specifically relating to a method and system for detecting cracks in concrete dams based on long short-term memory networks. Background Technology
[0002] Currently, my country's technologies and methods for inspecting hydropower dams are becoming increasingly sophisticated. Drones, video surveillance, underwater robots, robotic dogs, and wall-climbing robots are now widely used in dam inspection and monitoring. With the diversification of dam inspection methods, engineering technicians face the challenge of identifying defects from a large number of images, estimating their dimensions, marking them, and providing a panoramic view. This task, if performed entirely manually, is not only inefficient but also prone to omissions. Therefore, an efficient and accurate method for detecting dam cracks is urgently needed.
[0003] Existing methods for detecting cracks in dams can be broadly categorized into three types: sliding window-based image classification, object detection, and image segmentation. Sliding window-based image classification methods classify cracks by sliding a window across the image and using a deep neural network to classify the window; commonly used deep neural networks include ResNet and GoogLeNet. Object detection methods primarily aim to detect the location and size of cracks in an image. Currently, commonly used object detection methods are mainly divided into two types: single-stage object detection methods (such as YOLO and SSD) and multi-stage object detection methods (such as Faster R-CNN). These methods extract features and contextual information from the object in the image for object detection and crack localization. Image segmentation methods divide the image into several non-overlapping regions, segment each region, and then stitch the segmentation results together to detect cracks. Deep learning-based image segmentation methods have shown good performance; commonly used deep learning networks include U-Net and FCN.
[0004] Furthermore, a closely related technique in this invention is LSTM (Long Short-Term Memory). When dealing with sequence-related problems, LSTM networks are commonly used as an efficient deep learning method for feature extraction and sequence modeling. LSTM effectively avoids the vanishing and exploding gradient problems that occur in traditional RNN models when passing long sequences, and is therefore widely used in sequence data mining, text generation, and speech recognition. In addition, LSTM models have also achieved good results in tasks such as image processing and classification.
[0005] Existing methods for detecting cracks in dams have shown some effectiveness, but each still has its own problems: sliding window-based image classification methods can accurately locate crack areas, but due to the small receptive field of each sliding window, they suffer from low accuracy and false positives; crack detection methods based on general object detection algorithms have a large receptive field, but the detection results are too broad in terms of crack location, not accurate enough, and the cracks occupy too small a proportion within the detection box; crack detection methods based on image segmentation are very accurate in detecting crack locations, but suffer from low accuracy, easy to miss detections, and require a large amount of training data. This invention combines convolutional neural networks (CNNs) and LSTM networks. CNNs extract image features, and LSTM leverages its advantages in sequence prediction to achieve crack recognition in slice sequences, effectively solving the problems of inaccurate crack location, missed detections, and low accuracy. Summary of the Invention
[0006] This invention aims to address the problems of inaccurate crack location, missed detections, and low precision in existing dam crack detection methods. To this end, it provides a concrete dam crack detection method and system based on Long Short-Term Memory (LSTM) networks. This method utilizes the continuity and directionality between consecutive slices in the crack's neighborhood, combined with the advantages of LSTM in sequence prediction, to improve detection accuracy, reduce missed detections, and thus enhance overall detection performance.
[0007] A method for detecting cracks in concrete dams based on long short-term memory networks includes the following steps:
[0008] 1. Image preprocessing and concrete dam crack slice mesh generation: The size of the concrete dam crack image to be detected is enlarged to match the pre-set slice size;
[0009] First, select an image of a concrete dam crack to be inspected, with dimensions (h0, w0). For more precise analysis, divide the image into a p×p grid of slices, where p is chosen based on the size of the crack in the image. Then, enlarge the image size to (h1, w1), where... This ensures that the grid slices cover the entire image, while any excess areas are filled with black pixels.
[0010] 2. Traverse the concrete dam crack slices in sequence to find the new crack head slice and determine its crack direction;
[0011] Each concrete dam crack slice is traversed sequentially from top to bottom and left to right. For each slice, the number of traversals is recorded. If the number of traversals reaches or exceeds a set limit, the traversal is stopped. cIf the default value is 2, then skip the traversal of that slice. When traversing to slice A, set A as the target slice and the prefix sequence as an empty sequence S.
[0012] Using a convolutional neural network (such as ResNet), crack features are extracted from the target slice image A of a concrete dam crack. This feature vector is used as input to an LSTM to predict the direction of the cracks in the slice. The output is dirs, representing the probability of each direction of the crack in the slice, specifically the probabilities of right, left, down, and up. If the probability value in one or more directions in the recognition result dirs is greater than or equal to the confidence level conf (default is 0.5), then the target slice A is identified as having a crack. The adjacent slices in the direction corresponding to the probability value in dirs that is greater than or equal to the confidence level conf are regarded as the next target slice A′, and A is used as the prefix sequence S, proceeding to step 3. If the probability value corresponding to each direction in the recognition result dirs is less than the confidence level conf, then the target slice A is determined to have no cracks.
[0013] 3. Arrange multiple concrete dam crack slices into a sequence according to spatial proximity and crack direction;
[0014] If multiple orientations of cracks are predicted in target slice A (such as bifurcated cracks), that is, in the prediction results dirs -1 If multiple values are greater than or equal to the confidence level conf, then the corresponding slices are taken as target slices A′. Then, the target slices A′ are concatenated to the end of the prefix sequence S to form a new crack slice sequence S′, and then proceed to step 4.
[0015] 4. Extract the sequence features of cracks in concrete dams and make further predictions on the crack orientation in the end slice images of the sequence;
[0016] Using the convolutional neural network mentioned in step 2, crack features are extracted from the slice images in the crack slice sequence S′. The extracted crack feature vector sequence is used as the input of LSTM to obtain the output dirs′, which is used to predict the direction of cracks in the target slice at the end of the sequence, and then proceed to step 5.
[0017] 5. Based on the predicted crack orientation in the end slice of the sequence, determine the next target slice and add it to the end of the crack slice sequence;
[0018] Based on the output result dirs′, if a crack exists in the target slice A′, then the adjacent slices in the direction corresponding to the probability value greater than or equal to the confidence level conf in the prediction result are regarded as the next target slice A″, and S′ is used as the prefix sequence, and the process proceeds to step 3 again. If there is no crack in the target slice, the process returns to step 2.
[0019] A second aspect of the present invention relates to a concrete dam crack detection system based on long short-term memory networks, comprising:
[0020] The image preprocessing and concrete dam crack slice mesh generation module enlarges the size of the concrete dam crack image to be detected to match the pre-set slice size.
[0021] The concrete dam crack slice traversal module is used to traverse the concrete dam crack slices in sequence, find the new crack head slice and determine its crack direction.
[0022] A module for sorting multiple concrete dam crack slices is used to sequence multiple concrete dam crack slices according to their spatial proximity and crack orientation.
[0023] The crack orientation prediction module is used to extract the characteristics of the crack sequence in concrete dams and further predict the crack orientation in the end slice image of the sequence.
[0024] The next target slice sorting module is used to determine the next target slice based on the predicted crack orientation in the last slice of the sequence and add it to the end of the crack slice sequence.
[0025] A third aspect of the present invention relates to a concrete dam crack detection device based on a long short-term memory network, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the concrete dam crack detection method based on a long short-term memory network of the present invention.
[0026] A fourth aspect of the present invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the concrete dam crack detection method based on a long short-term memory network of the present invention.
[0027] The advantages of this invention are: considering the inaccurate localization problem of existing target detection methods, a slice classification method is used. Simultaneously, by utilizing the directionality and continuity of cracks, a slice sequence is formed, and LSTM is used for sequence detection, thereby improving the accuracy of crack identification. Attached Figure Description
[0028] Figure 1 This is a schematic diagram of the Long Short-Term Memory (LSTM) network in this invention.
[0029] Figure 2 This is a schematic diagram of the process of the concrete dam crack detection method based on long short-term memory network of the present invention. Detailed Implementation
[0030] The technical solution of the present invention will be clearly and completely explained and described below with reference to the accompanying drawings.
[0031] Example 1
[0032] Reference Figure 1 and Figure 2 This embodiment relates to a method for detecting cracks in concrete dams based on long short-term memory networks, including the following steps:
[0033] 1. Image preprocessing and concrete dam crack slice mesh generation: The size of the concrete dam crack image to be detected is enlarged to match the pre-set slice size;
[0034] First, select an image of a concrete dam crack to be inspected, with dimensions (h0, w0). For more precise analysis, divide the image into a p×p grid of slices, where p is chosen based on the size of the crack in the image. Then, enlarge the image size to (h1, w1), where... This ensures that the grid slices cover the entire image, while any excess areas are filled with black pixels.
[0035] 2. Traverse the concrete dam crack slices in sequence to find the new crack head slice and determine its crack direction;
[0036] Each concrete dam crack slice is traversed sequentially from top to bottom and left to right. For each slice, the number of traversals is recorded. If the number of traversals reaches or exceeds a set limit, the traversal is stopped. c If the default value is 2, then skip the traversal of that slice. When traversing to slice A, set A as the target slice and the prefix sequence as an empty sequence S.
[0037] Using a convolutional neural network (such as ResNet), crack features are extracted from the target slice image A of a concrete dam crack. This feature vector is used as input to an LSTM to predict the direction of the cracks in the slice. The output is dirs, representing the probability of each direction of the crack in the slice, specifically the probabilities of right, left, down, and up. If the probability value in one or more directions in the recognition result dirs is greater than or equal to the confidence level conf (default is 0.5), then the target slice A is identified as having a crack. The adjacent slices in the direction corresponding to the probability value in dirs that is greater than or equal to the confidence level conf are regarded as the next target slice A′, and A is used as the prefix sequence S, proceeding to step 3. If the probability value corresponding to each direction in the recognition result dirs is less than the confidence level conf, then the target slice A is determined to have no cracks.
[0038] 3. Arrange multiple concrete dam crack slices into a sequence according to spatial proximity and crack direction;
[0039] If multiple orientations of cracks are predicted in target slice A (such as bifurcated cracks), that is, in the prediction results dirs -1 If multiple values are greater than or equal to the confidence level conf, then the corresponding slices are taken as target slices A′. Then, the target slices A′ are concatenated to the end of the prefix sequence S to form a new crack slice sequence S′, and then proceed to step 4.
[0040] 4. Extract the sequence features of cracks in concrete dams and make further predictions on the crack orientation in the end slice images of the sequence;
[0041] Using the convolutional neural network mentioned in step 2, crack features are extracted from the slice images in the crack slice sequence S′. The extracted crack feature vector sequence is used as the input to an LSTM to obtain the output dirs′. The LSTM principle diagram is shown below. Figure 1 As shown, to predict the direction of the crack in the target slice at the end of the sequence, proceed to step 5.
[0042] 5. Based on the predicted crack orientation in the end slice of the sequence, determine the next target slice and add it to the end of the crack slice sequence;
[0043] As in the algorithm flow Figure 2 As shown, based on the output result dirs′, if there is a crack in the target slice A′, the adjacent slices in the direction corresponding to the probability value greater than or equal to the confidence level conf in the prediction result are regarded as the next target slice A″, and S′ is used as the prefix sequence, and the process proceeds to step 3 again. If there is no crack in the target slice, the process returns to step 2.
[0044] The method of this invention overcomes the problems of inaccurate positioning and small sensing field of traditional methods, and improves the accuracy and stability of crack detection in concrete dams by utilizing the continuity characteristics of cracks.
[0045] Example 2
[0046] This embodiment relates to a concrete dam crack detection system based on long short-term memory networks, used to implement the concrete dam crack detection method based on long short-term memory networks in Embodiment 1, including:
[0047] The image preprocessing and concrete dam crack slice mesh generation module enlarges the size of the concrete dam crack image to be detected to match the pre-set slice size.
[0048] The concrete dam crack slice traversal module is used to traverse the concrete dam crack slices in sequence, find the new crack head slice and determine its crack direction.
[0049] A module for sorting multiple concrete dam crack slices is used to sequence multiple concrete dam crack slices according to their spatial proximity and crack orientation.
[0050] The crack orientation prediction module is used to extract the characteristics of the crack sequence in concrete dams and further predict the crack orientation in the end slice image of the sequence.
[0051] The next target slice sorting module is used to determine the next target slice based on the predicted crack orientation in the last slice of the sequence and add it to the end of the crack slice sequence.
[0052] Example 3
[0053] This embodiment relates to a concrete dam crack detection device based on long short-term memory networks, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the concrete dam crack detection method based on long short-term memory networks of Embodiment 1.
[0054] Example 4
[0055] A fourth aspect of the present invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the concrete dam crack detection method based on a long short-term memory network of Embodiment 1.
[0056] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms described in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.
Claims
1. A method for detecting cracks in concrete dams based on long short-term memory networks, comprising the following steps: Step 1. Image preprocessing and concrete dam crack slice mesh generation: The size of the concrete dam crack image to be detected is enlarged to match the pre-set slice size; Step 2. Traverse the concrete dam crack sections sequentially to find new crack head sections and determine their crack direction; including: Each concrete dam crack slice is traversed sequentially from top to bottom and left to right. For each slice, the number of traversals is recorded. If the number reaches or exceeds a set threshold, the traversal is considered complete. If the slice is not found, skip the traversal of that slice; when the slice is reached... At that time, set For the target slice, the prefix sequence is an empty sequence. ; Using a convolutional neural network, extract target slice images of cracks in concrete dams. The crack features in the slice are used as input to an LSTM to predict the direction of the cracks in the slice, and the output is... This represents the probability of each direction of the crack in the slice, specifically the probability of right, left, downward, and upward; if the identification result... There exists a probability value greater than or equal to the confidence level in one or more directions. When that happens, the target slice is identified. There are cracks in it, and Confidence level greater than or equal to The adjacent slices in the direction corresponding to the probability value are considered the next target slice. At the same time As a prefix sequence Proceed to step 3; if the recognition result is... The probability value for each direction is less than the confidence level. When this happens, the target slice is determined. No cracks; Step 3. Arrange multiple concrete dam crack slices into a sequence according to spatial proximity and crack direction; Step 4. Extract features from the crack sequence of the concrete dam and further predict the crack orientation in the final slice image of the sequence; this includes: using the convolutional neural network mentioned in Step 2 to analyze the crack slice sequence. The sliced images are processed, and crack features are extracted from each slice. The extracted crack feature vector sequence is used as input to an LSTM to obtain the output. To predict the direction of the crack in the target slice at the end of the sequence, proceed to step 5; Step 5. Based on the predicted crack orientation in the end slice of the sequence, determine the next target slice and add it to the end of the crack slice sequence.
2. The method for detecting cracks in concrete dams based on long short-term memory networks as described in claim 1, characterized in that: Step 1 includes: First, select an image of the concrete dam cracks to be inspected, with a size of [size missing]. To perform more precise analysis, the image is divided into segments of size [size missing]. The sliced mesh, in which The selection is determined based on the size of the crack in the image; by enlarging the image size to... ,in This ensures that the grid slices cover the entire image, while any excess areas are filled with black pixels.
3. The method for detecting cracks in concrete dams based on long short-term memory networks as described in claim 1, characterized in that: Step 3 includes: If the target slice The prediction indicated that the cracks had multiple orientations, meaning that the prediction results... There are multiple values greater than or equal to the confidence level. Then the corresponding slices will be used as target slices. Then slice the target spliced in the prefix sequence Following this, a new sequence of crack slices is formed. Proceed to step 4.
4. The method for detecting cracks in concrete dams based on long short-term memory networks as described in claim 1, characterized in that: Step 5 includes: Based on the output results Determine if the target slice If a crack exists, then the prediction result with a confidence level greater than or equal to the confidence level will be included. The adjacent slices in the direction corresponding to the probability value are considered the next target slice. At the same time As a prefix sequence, proceed to step 3 again; if there are no cracks in the target slice, return to step 2.
5. A concrete dam crack detection system based on long short-term memory networks, characterized in that, include: The image preprocessing and concrete dam crack slice mesh generation module enlarges the size of the concrete dam crack image to be detected to match the pre-set slice size. The concrete dam crack slice traversal module is used to sequentially traverse concrete dam crack slices, find new crack header slices, and determine their crack direction; including: Each concrete dam crack slice is traversed sequentially from top to bottom and left to right. For each slice, the number of traversals is recorded. If the number reaches or exceeds a set threshold, the traversal is considered complete. If the slice is not found, skip the traversal of that slice; when the slice is reached... At that time, set For the target slice, the prefix sequence is an empty sequence. ; Using a convolutional neural network, extract target slice images of cracks in concrete dams. The crack features in the slice are used as input to an LSTM to predict the direction of the cracks in the slice, and the output is... This represents the probability of each direction of the crack in the slice, specifically the probability of right, left, downward, and upward; if the identification result... There exists a probability value greater than or equal to the confidence level in one or more directions. When that happens, the target slice is identified. There are cracks in it, and Confidence level greater than or equal to The adjacent slices in the direction corresponding to the probability value are considered the next target slice. At the same time As a prefix sequence If the recognition result The probability value for each direction is less than the confidence level. When this happens, the target slice is determined. No cracks; A module for sorting multiple concrete dam crack slices is used to sequence multiple concrete dam crack slices according to their spatial proximity and crack orientation. The crack orientation prediction module is used to extract features from the crack sequence of concrete dams and further predict the crack orientation in the final slice images of the sequence; this includes: using a convolutional neural network to analyze the crack slice sequence. The sliced images are processed, and crack features are extracted from each slice. The extracted crack feature vector sequence is used as input to an LSTM to obtain the output. To predict the direction of cracks in the target slice at the end of the sequence; The next target slice sorting module is used to determine the next target slice based on the predicted crack orientation in the last slice of the sequence and add it to the end of the crack slice sequence.
6. A concrete dam crack detection device based on long short-term memory networks, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the concrete dam crack detection method based on long short-term memory network as described in any one of claims 1-4.
7. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the concrete dam crack detection method based on a long short-term memory network as described in any one of claims 1-4.