SAM-based crack image mapping method for curved members

Abstract

The SAM-based crack image drawing method of a curved component comprises the following steps: S1, obtaining a target curved component image; S2, segmenting the target curved component; S3, extracting the contour of the target curved component; S4, identifying the crack of the target curved component; and S5, generating a crack shape map of the target curved component. The SAM segmentation method is used to obtain a component edge image, the sliding non-overlapping window method is used to obtain a crack image, and finally, the crack image of the experimental beam is drawn by combining them, all the image size parameters are the same as those of the original image, the position in the final crack image is the original position in the original image, the size change and repositioning are avoided, the manual operation in the repeated experiments of a large number of experimental components is reduced, the complex preparation work is not needed, the one-key generation of the crack image is realized, the problem of low efficiency of the traditional manual crack image drawing and crack width measurement is avoided, and the method has the characteristics of strong generalization and is convenient to popularize and use.

Description

Technical Field

[0001] The present invention relates to the technical field of crack detection, in particular to a method for drawing a crack map of a curved component based on SAM. Background Art

[0002] Cracks are a common building defect and a key indicator of potential structural safety hazards. They play a vital role in assessing structural condition. Regular inspections help detect and repair cracks early, thereby ensuring structural integrity and safety.

[0003] Determining the relative position of cracks and the overall component, that is, drawing a crack morphology map, can help inspectors more clearly and directly observe the distribution and quantity of cracks, estimate the overall damage level of the component, and improve the effectiveness of building damage assessment. On the other hand, the acquisition of crack morphology maps provides material for the measurement of crack parameters. The crack morphology and position image results obtained can be used for image analysis to locate and measure building cracks. In actual surveys, it is not difficult to find that the overall shape of most cracked components is not regular. There are two main reasons for this: one is that the original shape of the component is a curved component, and the other is that the component is deformed due to stress, causing the regular component to deform into a curved component.

[0004] To study structural performance, researchers typically need to analyze crack images of concrete test beams in the laboratory. Current methods for rendering crack images in concrete test beams primarily focus on rendering local crack images, which presents the following difficulties in engineering practice:

[0005] 1. The traditional component outline image automatic drawing method has high requirements for the laboratory components obtained and requires a lot of preparatory work. For example, if a single-color background with a large contrast to the color of the experimental component is required, or if the experimental beam White paint When the background color grayscale has low contrast or the white paint is unevenly applied, the extraction effect will be poor and the final image will be inaccurate. This preliminary work is necessary because the selected Canny segmentation algorithm requires the manual operation of setting a threshold. The purpose of setting the threshold is to set the grayscale requirement. To ensure that repeated measurements do not change the threshold, it is necessary to ensure that the component surface, cracks, and background in the image have a certain amount of grayscale difference. When the experimental components are large, the repeated white paint application, maintaining the original position of the components to arrange the background, and taking images, these repeated manual arrangements will become extremely tedious.

[0006] 2. Traditional crack images have low accuracy and are often only useful for observing crack location and morphology. Traditional crack image training datasets often use common crack datasets for training, resulting in poor training results. While they can be applied to a wide range of scenarios, they are less targeted for components in specialized scenarios, such as experimental beams. However, crack images in the laboratory require higher accuracy than in other scenarios, often requiring the highest possible precision for the extracted crack images. Conventional training methods sometimes cannot meet the high-precision measurement requirements of the laboratory.

[0007] 3. The previous method of manually selecting component corner points to draw the outline of a damaged component can only be applied to concrete components that have not undergone significant deformation and retain a regular shape upon failure. For concrete components that exhibit a curved appearance after failure, manual selection has no solution and can only meet the experimental needs of extremely special cases.

[0008] Therefore, it is necessary to study and develop a SAM-based method for rendering crack images of curved components for structural safety monitoring. Summary of the Invention

[0009] The present invention proposes a method for drawing crack images of curved components based on SAM, which uses the SAM segmentation method to obtain component edge images, and the sliding non-overlapping window method to obtain crack images, and finally merges them to draw the crack image of the experimental beam. All image parameters such as image size are the same as the original image obtained, and the position in the final crack image is the original position in the original image, avoiding size changes and repositioning, thereby avoiding a large amount of manual operations in repeated experiments on a large number of experimental components, thereby solving the problems raised in the background technology.

[0010] The technical solution adopted in the present invention is:

[0011] The method for rendering crack images of curved components based on SAM includes the following steps:

[0012] S1. Obtaining an image of a target curved component: Obtaining an image of the concrete component by a camera, shooting the image of the target curved component including the curved component and cracks in a forward direction, uploading the image of the target curved component to a processor and storing it;

[0013] S2. Segmenting the target curved component: using an interactive interface connected to the processor, using points or boxes in the OpenCV-based interactive interface to prompt the SAM of the concrete curved component target to be drawn, and selecting and extracting the segmentation result of the target curved component from the acquired image;

[0014] S3. Extracting the contour of the target curved component: identifying the contour edge boundary line of the target curved component from the segmentation result, and performing post-processing correction on it to obtain the contour image of the target curved component;

[0015] S4. Identify cracks in the target curved component: The process is as follows:

[0016] S4.1. Use a sliding non-overlapping window method to identify cracks on the target curved component and extract a distribution map of cracks on the target curved component.

[0017] S4.2. Capture at least one obvious reference point on the contour boundary of the target curved component, and mark at least one reference point on the identified and extracted crack set map;

[0018] S4.3. Measure the distance from the reference point to the benchmark point, establish a plane coordinate system based on the benchmark point and the reference point and their relative positional relationship, and obtain the distribution relationship of the cracks on the target curved component;

[0019] S5. Generate a crack morphology diagram of the target curved component: The process is as follows:

[0020] S5.1. Adjust the scale of the contour image and the crack set map so that both the contour image and the crack set map are scaled in the same proportion as the original image of the target curved component;

[0021] S5.2. Locate the reference point on the contour image and the reference point on the crack morphology map, and calculate the adaptive distance between the reference point and the reference point based on the scaling ratio of the contour image and the crack set map;

[0022] S5.3. Based on the plane coordinate system relationship, superimpose the crack morphology map of the same scale on the contour image. After combining, the crack morphology map of the target curved component is obtained, which is used to describe the crack distribution on the concrete curved component.

[0023] Preferably, in S2, when extracting the outline of the target curved component, use the point or box prompt SAM on the interactive interface to extract the target curved component, define the box prompt by clicking the upper left corner and the lower right corner, and divide the target curved component pattern within the selected range of the definition box area by dragging the definition box to move the position and adjust the area of ​​the definition box area.

[0024] Preferably, in S3, the cv2.findContours function is used to extract the contour image of the concrete curved component after SAM segmentation, and the contour in the image is repaired in combination with the post-processing step. The process is as follows:

[0025] S3.1. Identify and analyze the contour line trajectory in the contour image to determine whether the extracted contour image obstructs the target curved component;

[0026] S3.2. Based on the judgment result, fill the outline of the outline image and repair the edge lines.

[0027] Preferably, if there is an occlusion to the target curved member in the contour image, the occluded part is erased, the missing part is filled with content recognition after the missing part selection area is established, and the corrected contour image of the target curved member is outputted subsequently.

[0028] If there is no occlusion to the target curved member in the contour image, the contour image of the target curved member is directly outputted.

[0029] Preferably, in S4.1, the step of crack identification of the target curved member using the sliding non-overlapping window method is as follows:

[0030] S4.1.1, a lightweight semantic segmentation network SegFormer-B0 is used, the data sets adopt CFD, CrackTree and DeepCrack, each data set is randomly divided into a training set, a verification set and a test set according to a ratio of 8:1:1, and the training set, the verification set and the test set are respectively cross-merged and used for model training;

[0031] S4.1.2, the image is filled to be consistent with the size of the image after perspective correction, and the corrected image is divided into image blocks in the same training configuration;

[0032] S4.1.3, the trained model is used to segment the cracks in the image blocks;

[0033] S4.1.4, after the cracks in the image blocks are segmented, the results are spliced together according to their numbers to reconstruct the whole image;

[0034] S4.1.5, the non-image part is cropped, the binary crack identification result is obtained, and the binary crack identification result is saved in a lossless compression picture format;

[0035] S4.1.6, a dynamic interactive method is adopted, different sizes and background color consistent circle cover noise are used based on OpenCV, and the final image is obtained as the crack segmentation result.

[0036] Preferably, in S4.1.1, the batch size used in the model training process is 8.

[0037] Preferably, in S4.1.2, the unit module of the image block is set to a rectangle.

[0038] Preferably, in S4.1.5, the binary crack identification result is stored in PNG format.

[0039] The present application has the following beneficial effects:

[0040] Compared with the prior art, the present application has the following characteristics and advantages:

[0041] 1. By using the SAM segmentation method to obtain component edge images and the sliding non-overlapping window method to obtain crack images, the two images are finally merged to draw the crack image of the experimental beam. All image dimensions and other parameters are the same as the original image. The position of the crack in the final image is the same as that in the original image, avoiding size changes and repositioning, thus avoiding a lot of manual operations in repeated experiments on a large number of experimental components. Extensive preparatory work is not required, and crack images can be generated with one click. This avoids the inefficiency of traditional manual crack image drawing and crack width measurement methods. The method has strong generalization characteristics and is easy to promote and use.

[0042] 2. By selecting the cracked component image directly from the experimental environment in the SAM interactive interface or clicking on the experimental component to separate it from the laboratory environment, the preparatory work for drawing can be greatly avoided, saving tedious tasks such as painting and background arrangement, and breaking the limitation of the traditional manual point-selection method that can only draw components with regular shapes;

[0043] 3. The OOD crack segmentation method based on prior knowledge is adopted to select high-quality labeled data as candidate data in a targeted manner, which can fully improve the training effect and measurement accuracy, and improve the crack extraction effect. It can not only improve the model performance, but also significantly shorten the training time. By manually filtering data with high-quality annotations and similar to the target scene, the model training efficiency is optimized and the performance in OOD data is improved. The generated crack image accuracy reaches the pixel level, which solves the problem that the traditional drawing method has low accuracy and can only be used for roughly observing the crack location. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flow chart of the SAM-based curved component crack image drawing process provided in an embodiment of the present invention;

[0045] Figure 2 This is a flowchart of the sliding non-overlapping window crack recognition based on deep learning in the present invention. DETAILED DESCRIPTION

[0046] The method for rendering crack images of curved components based on SAM includes the following steps:

[0047] S1. Obtaining an image of a target curved component: Obtaining an image of the concrete component by a camera, shooting the image of the target curved component including the curved component and cracks in a forward direction, uploading the image of the target curved component to a processor and storing it;

[0048] S2. Segmenting the target curved component: using an interactive interface connected to the processor, using points or boxes in the interactive interface based on OpenCV to prompt the SAM of the concrete curved component target to be drawn, reading the target curved component image, selecting the target curved component, and extracting the segmentation result of the target curved component;

[0049] When extracting the contour of the target curved component, use the point or box prompt SAM on the interactive interface to extract the target curved component. Click the upper left corner and lower right corner to define the box prompt. Drag the definition box to move the position and adjust the area of ​​the definition box to split the target curved component pattern within the selected range of the definition box.

[0050] S3. Extracting the contour of the target curved component: identifying the contour edge boundary line of the target curved component from the segmentation result, and performing post-processing correction on it to obtain the contour image of the target curved component;

[0051] The cv2.findContours function is used to extract the contour image of the concrete curved component after SAM segmentation, and the contour in the image is repaired in combination with the post-processing process. The process is as follows:

[0052] S3.1. Identify and analyze the contour line trajectory in the contour image to determine whether the extracted contour image obstructs the target curved component. If obstruction occurs, the accurate reflection of the target curved component's shape will be affected.

[0053] S3.2. Based on the judgment result, fill the outline of the outline image and repair the edge lines.

[0054] Preferably, if there is occlusion of the target curved component in the contour image, the occluded portion is erased, and after establishing a selection of the missing portion, the missing portion is repaired by content-aware filling, and then the corrected contour image of the target curved component is output;

[0055] If there is no occlusion of the target curved component in the contour image, directly output the contour image of the target curved component;

[0056] S4. Identify cracks in the target curved component: The process is as follows:

[0057] S4.1. Use a sliding non-overlapping window method to identify cracks on the target curved component and extract a distribution map of cracks on the target curved component.

[0058] For the crack recognition model, an OOD crack segmentation method based on prior knowledge can be used. First, training data is established, the quality of existing data is manually evaluated, data with low annotation quality is removed, and the remaining high-quality annotated data is selected as candidate data; the characteristics of the target scene are compared, and the candidate data is matched with the target scene based on similarity (such as the same scene); based on the above matching results, high-quality annotated data with similar characteristics to the expected application scene is organized for model training; the deep learning model is initialized using pre-trained weights from a large data set, and the model is fine-tuned to obtain fine-tuning weights; the fine-tuned existing model is applied to the target OOD crack image to obtain the crack segmentation image result. The steps are as follows:

[0059] S4.1.1. Using the lightweight semantic segmentation network SegFormer-B0, the datasets were trained using CFD, CrackTree, and DeepCrack. Each dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio. The training, validation, and test sets were then cross-merged and used for model training. The batch size used during model training was 8.

[0060] S4.1.2. Pad the image to match the size of the perspective-corrected image and divide the corrected image into image blocks with the same training configuration. Set the unit module of the image block to be rectangular to reduce performance fluctuations caused by the mismatch between the input image size and the training phase when applying the model.

[0061] S4.1.3. Use the trained model to segment cracks in the image blocks.

[0062] S4.1.4, after segmenting the cracks in the image blocks, stitch the results together according to their numbers to reconstruct the entire image;

[0063] S4.1.5. Crop the non-image portion to obtain a binary crack identification result, and save it in a lossless compressed image format (e.g., PNG) to maintain pixel value accuracy without introducing color changes;

[0064] S4.1.6. Using the dynamic interaction method based on OpenCV, use circles of different sizes and the same color as the background to cover the noise and obtain the final image as the crack segmentation result;

[0065] S4.2. Capture at least one obvious reference point on the contour boundary of the target curved component, and mark at least one reference point on the identified and extracted crack set map;

[0066] S4.3. Measure the distance from the reference point to the benchmark point, establish a plane coordinate system based on the benchmark point and the reference point and their relative positional relationship, and obtain the distribution relationship of the cracks on the target curved component;

[0067] S5. Generate a crack morphology diagram of the target curved component: The process is as follows:

[0068] S5.1. Adjust the scale of the contour image and the crack set map so that both the contour image and the crack set map are scaled in the same proportion as the original image of the target curved component;

[0069] S5.2. Locate the reference point on the contour image and the reference point on the crack morphology map, and calculate the adaptive distance between the reference point and the reference point based on the scaling ratio of the contour image and the crack set map;

[0070] S5.3. Based on the plane coordinate system relationship, superimpose the crack morphology map of the same scale on the contour image. After combining, the crack morphology map of the target curved component is obtained, which is used to describe the crack distribution on the concrete curved component.

[0071] As an embodiment of the present invention:

[0072] like Figure 1 As shown in the figure, a T-shaped concrete beam bending test photo was selected, with an image resolution of 3648 × 2736 pixels. The specimen is 2500 mm long and 350 mm high. The SAM-based curved component contour extraction method is as follows:

[0073] Step 1: Acquire an image of a curved component with cracks: Acquire an image containing the outline of the curved component;

[0074] Step 2: Extract curved components using SAM: Use the interactive interface to prompt SAM to extract curved components by clicking or selecting the box. Select the target curved component image to be segmented in the image by clicking the upper left corner and the lower right corner and dragging the prompt box to adjust the position.

[0075] Step 3. Extract its contour through cv2.findContours function: After obtaining the segmentation result of the target curved component, use cv2.findContours function to extract its contour shape and save it as the contour extraction result of the target curved component;

[0076] Step 4: Post-process the extraction results: In some cases, there are some occlusions of the target curved component in the image, and the extracted beam contour may not accurately reflect the shape of the beam. Therefore, post-processing is required for the target curved component. The specific post-processing method depends on the situation.

[0077] like Figure 2 , a method for crack identification using sliding non-overlapping windows based on deep learning, comprising the following steps:

[0078] Step 1. Model training: Use the lightweight semantic segmentation network SegFormer-B0. The datasets are trained using CFD, CrackTree, and DeepCrack. Each dataset is randomly divided into training, validation, and test sets in an 8:1:1 ratio. These sets are then merged for model training. The batch size used during training is 8.

[0079] Step 2: Image preprocessing: Pad the image to match the size of the perspective-corrected image and divide the corrected image into image blocks with the same size as the training configuration, specifically 512×512. This is to reduce performance fluctuations caused by the input image size being different from the training phase when the model is applied.

[0080] Step 3, crack segmentation: Use the trained model to segment the cracks in the image block, and the crack segmentation image block;

[0081] Step 4: Stitching image blocks: After segmenting the cracks in the image blocks, stitch the results together according to their numbers to reconstruct the entire image;

[0082] Step 5. Crop the image: Crop the non-image part to obtain the binary crack recognition result, and save it in a lossless compression format such as PNG to maintain the pixel value accuracy without introducing color changes;

[0083] Step 6: Eliminate noise: Using the dynamic interaction method, based on OpenCV, circles of different sizes and the same color as the background are used to cover the noise, and the final image is obtained as the crack segmentation result.

[0084] The OOD crack segmentation method based on prior knowledge includes the following steps:

[0085] Step 1: Screen candidate data: When establishing training data, manually evaluate the quality of existing data, remove data with low annotation quality, and select the remaining high-quality annotated data as candidate data;

[0086] Step 2: Screen target scenario data: Compare the characteristics of the target scenario and match the candidate data with the target scenario based on the same scenario. Based on the results of these two steps, organize high-quality annotated data with similar characteristics to the expected application scenario for model training;

[0087] Step 3: Model training: Initialize the deep learning model using pre-trained weights from a large dataset and fine-tune the model to obtain fine-tuned weights.

[0088] Step 4: Model application: Apply the fine-tuned existing model to the target OOD crack image to obtain the crack segmentation result image.

[0089] An OOD crack segmentation method based on prior knowledge is used to improve crack extraction. Experimenters will have expectations for the crack results that appear in the experiment. Based on the expected results, they can select high-quality labeled data as candidate data in a targeted manner to fully improve the training effect and ultimately improve measurement accuracy. The impact of different training data and domain knowledge on the model's crack detection ability has been experimentally analyzed, and the model performance has been evaluated on the test set and external test set. It is proved that the crack segmentation method based on prior knowledge can not only improve model performance, but also significantly shorten training time. In the experimental results, the average value of the Dice coefficient obtained on the test set corresponding to the training data is closest to that obtained as the pre-training weight for in-domain transfer learning. According to the experimental results, by manually filtering data with high-quality annotations and similar to the target scene, the model training efficiency can be optimized and the performance in OOD data can be improved.

[0090] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for rendering crack images of curved components based on SAM, characterized by: The following steps are involved: S1. Acquire a target curved component image: photograph the target curved component image including the curved component and cracks in a forward direction, upload the target curved component image to a processor, and store it; S2. Segmenting the target curved component: selecting and extracting the segmentation result of the target curved component from the acquired image using an interactive interface connected to the processor; S3. Extracting the contour of the target curved component: identifying the contour edge boundary line of the target curved component from the segmentation result, and performing post-processing correction on it to obtain the contour image of the target curved component; S4. Identify cracks in the target curved component: The process is as follows: S4.

1. Use a sliding non-overlapping window method to identify cracks on the target curved component and extract a distribution map of cracks on the target curved component. S4.

2. Capture at least one obvious reference point on the contour boundary of the target curved component, and mark at least one reference point on the identified and extracted crack set map; S4.

3. Measure the distance from the reference point to the benchmark point, establish a plane coordinate system based on the benchmark point and the reference point and their relative positional relationship, and obtain the distribution relationship of the cracks on the target curved component; S5. Generate a crack morphology diagram of the target curved component: The process is as follows: S5.

1. Adjust the scale of the contour image and the crack set map so that both the contour image and the crack set map are scaled in the same proportion as the original image of the target curved component; S5.

2. Locate the reference point on the contour image and the reference point on the crack morphology map, and calculate the adaptive distance between the reference point and the reference point based on the scaling ratio of the contour image and the crack set map; S5.

3. Based on the plane coordinate system relationship, the crack morphology diagram of the same scale is superimposed on the contour image, and the crack morphology diagram of the target curved component is obtained after combination.

2. The method for rendering crack images of curved components based on SAM according to claim 1, characterized in that: In S2, when extracting the contour of the target curved component, use the point or box prompt SAM on the interactive interface to extract the target curved component. Click the upper left corner and the lower right corner to define the box prompt. By dragging the definition box to move the position and adjust the area of ​​the definition box, the target curved component pattern within the selected range of the definition box area is divided.

3. The method for rendering crack images of curved components based on SAM according to claim 1, characterized in that: In S3, the cv2.findContours function is used to extract the contour shape of the target curved component, and the contour in the image is repaired in combination with the post-processing process. The process is as follows: S3.

1. Identify and analyze the contour line trajectory in the contour image to determine whether there is any occlusion of the target curved component in the extracted contour image; S3.

2. Based on the judgment result, fill the outline of the outline image and repair the edge lines.

4. The method for rendering crack images of curved components based on SAM according to claim 3, characterized in that: If there is any occlusion of the target curved component in the contour image, the occluded part is erased, and after establishing a selection of the missing part, the missing part is repaired using content-aware filling, and then the contour image of the corrected target curved component is output; If there is no occlusion of the target curved component in the contour image, the contour image of the target curved component is directly output.

5. The method for rendering crack images of curved components based on SAM according to claim 1, characterized in that: In S4.1, the steps for crack identification of target curved components using the sliding non-overlapping window method are as follows: S4.1.

1. Using the lightweight semantic segmentation network SegFormer-B0, the datasets were trained using CFD, CrackTree, and DeepCrack. Each dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio. The training, validation, and test sets were then cross-merged for model training. S4.1.

2. Pad the image to make it the same size as the perspective-corrected image and divide the corrected image into image blocks with the same configuration as the training image. S4.1.

3. Use the trained model to segment cracks in the image blocks. S4.1.4, after segmenting the cracks in the image blocks, stitch the results together according to their numbers to reconstruct the entire image; S4.1.

5. Crop the non-image portion to obtain a binary crack identification result, and save it in a lossless compressed image format; S4.1.

6. Using the dynamic interaction method, based on OpenCV, circles of different sizes and the same color as the background are used to cover the noise and obtain the final image as the crack segmentation result.

6. The method for rendering crack images of curved components based on SAM according to claim 5, characterized in that: In S4.1.1, the batch size used during model training is 8.

7. The method for rendering crack images of curved components based on SAM according to claim 5, characterized in that: In S4.1.2, the unit module of the image block is set to be rectangular.

8. The method for rendering crack images of curved components based on SAM according to claim 5, characterized in that: In S4.1.5, the binary crack identification results are stored in PNG format.

Images