SAM-based curved component crack image drawing method

Abstract

The SAM-based curved component crack image drawing method comprises the steps of S1, acquiring a target curved component image; s2, segmenting the target curved component; s3, extracting the contour of the target curved component; s4, identifying a crack of the target curved component; and S5, generating a crack form graph of the target curved component. An SAM segmentation method is utilized to obtain a component edge image, a sliding non-overlapping window method is utilized to obtain a crack image, finally, the component edge image and the crack image are combined to draw an experimental beam crack image, all parameters such as image sizes are the same as those of an obtained original image, and the position in the final crack image is the original position in the original image. The method has the advantages that size changing and repositioning are avoided, manual operation in repeated experiments of a large number of experimental components is reduced, complex preparation work is not needed, one-key generation of the crack image is achieved, the problem that the efficiency of a traditional mode of manually drawing the crack image and measuring the crack width is low is solved, and the method has the advantage of being high in generalization and convenient to use and popularize.

Description

Technical Field

[0001] The invention relates to the technical field of crack detection, in particular to a method for drawing a crack diagram of a curved component based on SAM. Background Art

[0002] Cracks are common building defects and an important sign of potential structural safety hazards. They play a vital role in assessing structural conditions. Regular inspections can help detect and repair cracks early, thereby ensuring the integrity and safety of the structure.

[0003] Determining the relative position of cracks and the entire component, that is, drawing a crack morphology map, can help inspectors observe the distribution and quantity of cracks more clearly and directly, estimate the overall damage degree of the component, and improve the effect of building damage assessment. On the other hand, the acquisition of crack morphology maps provides material for the parameter measurement of cracks. The crack morphology and position image results obtained can be used for image analysis to locate and measure building cracks. In the actual survey, it is not difficult to find that most of the components with cracks do not present a regular shape. The reasons are mainly divided into two types: one is that the original shape of the component is a curved component, and the other is that the regular component is deformed due to the force on the component, which makes the regular component deform into a curved component.

[0004] In order to conduct structural performance research, researchers generally need to analyze the crack images of concrete test beams in the laboratory. However, the current methods for drawing crack images of concrete test beams mainly focus on drawing local crack images, which faces the following difficulties in engineering practice:

[0005] 1. The traditional component contour image automatic drawing method has high requirements for the laboratory components obtained and requires a lot of preparatory work. For example, a monochrome background with a large contrast with the color of the experimental component is required, or the experimental beam White paint When the grayscale difference of the background color is low or the white paint is unevenly applied, the extraction effect will be poor and the final image will be inaccurate. The reason for this preparatory work is that the selected canny segmentation algorithm requires the manual operation of setting the threshold. The purpose of setting the threshold is to set the grayscale requirement. In order to ensure that the repeated measurement of the threshold does not change, it is necessary to ensure that the component surface, cracks, and background in the image have a certain amount of grayscale difference. When there are many experimental components, repeated white paint painting, maintaining the original position of the component to arrange the background and take pictures, these repeated manual arrangements will become extremely tedious work.

[0006] 2. Traditional crack images have low precision and can only be used to observe crack location and morphology. Traditional crack image training data sets often use common crack data sets for training, which has poor training effects. They can be used in a variety of scenarios, but are less targeted for components in special scenarios such as experimental beams. However, crack images in laboratories require higher precision than other scenarios, and often require the accuracy of extracted crack images to be increased as much as possible. Ordinary training methods sometimes cannot meet the high-precision measurement needs in laboratories.

[0007] 3. The previous method of manually selecting component corner points to draw the outline of the destroyed component can only be applied to concrete components that have not undergone significant deformation and still maintain a regular shape when destroyed. For concrete components that have a curved appearance after destruction, the manual selection method has no solution and can only meet the experimental needs of extremely special cases.

[0008] Therefore, it is necessary to study and develop a SAM-based method for drawing crack images of curved components for structural safety monitoring. Summary of the invention

[0009] The present invention proposes a method for drawing crack images of curved components based on SAM, which uses the SAM segmentation method to obtain component edge images, uses a sliding non-overlapping window method to obtain crack images, and finally merges them to draw an experimental beam crack image. All image size and other parameters are the same as the obtained original image, and the position in the final crack image is the original position in the original image, avoiding size changes and repositioning, thereby avoiding a large number of manual operations in repeated experiments on a large number of experimental components, thereby solving the problems raised in the background technology.

[0010] The technical solution adopted by the present invention is:

[0011] The method for drawing crack images of curved components based on SAM includes the following steps:

[0012] S1. Obtaining a target curved component image: Obtaining an image of the concrete component through a camera, shooting a target curved component image including the curved component and cracks in a forward direction, and uploading the target curved component image to a processor for storage;

[0013] S2. Segmentation of target curved components: using an interactive interface connected to the processor, using points or boxes in the interactive interface based on Opencv to prompt SAM of the concrete curved component target to be drawn, and selecting and extracting the segmentation result of the target curved component from the acquired image;

[0014] S3, extracting the contour of the target curved component: identifying the contour edge boundary line of the target curved component from the segmentation result, and performing post-processing correction on it to obtain the contour image of the target curved component;

[0015] S4. Identify cracks in the target curved component: The process is as follows:

[0016] S4.1. Use a sliding non-overlapping window method to identify cracks on a target curved component and extract a set of cracks distributed on the target curved component;

[0017] S4.2, capturing at least one obvious reference point on the contour boundary of the target curved component, and marking at least one reference point on the identified and extracted crack set map;

[0018] S4.3, measure the distance from the reference point to the benchmark point, establish a plane coordinate system based on the benchmark point and the reference point and their relative position relationship, and obtain the distribution relationship of the cracks on the target curved component;

[0019] S5. Generate a crack morphology diagram of the target curved component: The process is as follows:

[0020] S5.1. Adjust the image ratio of the contour image and the crack set map so that the contour image and the crack set map are scaled in the same proportion as the original image of the target curved component obtained;

[0021] S5.2, locate the reference point on the contour image and the reference point on the crack morphology map, and calculate the adaptation distance between the reference point and the reference point according to the scaling ratio of the contour image and the crack set map;

[0022] S5.3. According to the plane coordinate system relationship, the crack morphology diagram of the same scale is superimposed on the contour image, and the crack morphology diagram of the target curved component is obtained after combination, which is used to describe the crack morphology image of the crack distribution on the concrete curved component.

[0023] Preferably, in S2, when extracting the contour of the target curved component, use the point or box prompt SAM on the interactive interface to extract the target curved component, define the box prompt by clicking the upper left corner and the lower right corner, and divide the target curved component pattern within the selected range of the definition box area by dragging the definition box to move the position and adjust the area of ​​the definition box.

[0024] Preferably, in S3, the contour image of the curved concrete component after SAM segmentation is extracted using the cv2.findContours function, and the contour in the image is repaired in combination with a post-processing step, and the process is as follows:

[0025] S3.1, identifying and analyzing the contour line trajectory in the contour image, and determining whether there is any occlusion of the target curved component in the extracted contour image;

[0026] S3.2. Based on the judgment result, fill the outline of the outline image and repair the edge lines.

[0027] Preferably, if there is occlusion of the target curved component in the contour image, the occluded part is erased, and after the missing part selection is established, the missing part is repaired by content recognition filling, and then the corrected contour image of the target curved component is output;

[0028] If there is no occlusion of the target curved component in the contour image, the contour image of the target curved component is directly output.

[0029] Preferably, in S4.1, the steps of using the sliding non-overlapping window method to identify cracks in the target curved component are as follows:

[0030] S4.1.1. Use the lightweight semantic segmentation network SegFormer-B0. The datasets are CFD, CrackTree and DeepCrack. Each dataset is randomly divided into training set, validation set and test set in a ratio of 8:1:1. The training set, validation set and test set are cross-merged and used for model training.

[0031] S4.1.2, pad the image to make it consistent with the size of the perspective-corrected image, and divide the corrected image into image blocks with the same training configuration;

[0032] S4.1.3, use the trained model to segment the cracks in the image blocks;

[0033] S4.1.4, after segmenting the cracks in the image blocks, stitching the results together according to their numbers to reconstruct the entire image;

[0034] S4.1.5, trimming off the non-image part, obtaining the binary crack recognition result, and saving it in a lossless compressed image format;

[0035] S4.1.6. Using the dynamic interaction method, based on OpenCV, circles of different sizes and the same color as the background are used to cover the noise and obtain the final image as the crack segmentation result.

[0036] Preferably, in S4.1.1, the batch size used in the model training process is 8.

[0037] Preferably, in S4.1.2, the unit module of the image block is set to be a rectangle.

[0038] Preferably, in S4.1.5, the binary crack identification result is stored in PNG format.

[0039] The present invention has the following beneficial effects:

[0040] Compared with the prior art, the present invention has the following characteristics and advantages:

[0041] 1. The component edge image is obtained by using the SAM segmentation method, the crack image is obtained by using the sliding non-overlapping window method, and finally the experimental beam crack image is drawn by merging them. All image size and other parameters are the same as the original image. The position in the final crack image is the original position in the original image, avoiding size changes and repositioning, thereby avoiding a large number of manual operations in repeated experiments of a large number of experimental components. A large amount of preparatory work is not required, and the crack image is generated in one click, avoiding the low efficiency of traditional manual drawing of crack images and measuring crack width. It has the characteristics of strong generalization and is easy to promote and use;

[0042] 2. By selecting the crack component image directly obtained in the experimental environment in the interactive interface of SAM or clicking on the experimental component to separate it from the laboratory environment, the preparatory work for drawing can be avoided to a great extent, saving tedious work such as painting and arranging the background, and breaking the limitation that the traditional manual point selection method can only draw components of regular shapes;

[0043] 3. The OOD crack segmentation method based on prior knowledge is adopted to select high-quality labeled data as candidate data in a targeted manner, which can fully improve the training effect and measurement accuracy, and improve the crack extraction effect. It can not only improve the model performance, but also significantly shorten the training time. By manually filtering data with high-quality annotations and similar to the target scene, the model training efficiency is optimized and the performance in OOD data is improved. The generated crack image accuracy reaches the pixel level, which solves the problem that the traditional drawing method has low accuracy and can only be used to roughly observe the crack location. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flow chart of drawing crack images of curved components based on SAM provided in an embodiment of the present invention;

[0045] Figure 2 This is a flow chart of sliding non-overlapping window crack recognition based on deep learning in the present invention. DETAILED DESCRIPTION

[0046] The method for drawing crack images of curved components based on SAM includes the following steps:

[0047] S1. Obtaining a target curved component image: Obtaining an image of the concrete component through a camera, shooting a target curved component image including the curved component and cracks in a forward direction, and uploading the target curved component image to a processor for storage;

[0048] S2. Segmenting the target curved component: using an interactive interface connected to the processor, using points or boxes in the interactive interface based on Opencv to prompt SAM of the concrete curved component target to be drawn, reading the target curved component image, selecting the target curved component, and extracting the segmentation result of the target curved component;

[0049] When extracting the contour of the target curved component, use the point or box prompt SAM on the interactive interface to extract the target curved component, click the upper left corner and the lower right corner to define the box prompt, drag the definition box to move the position and adjust the area of ​​the definition box, and split the target curved component pattern within the selected range of the definition box area;

[0050] S3, extracting the contour of the target curved component: identifying the contour edge boundary line of the target curved component from the segmentation result, and performing post-processing correction on it to obtain the contour image of the target curved component;

[0051] The cv2.findContours function is used to extract the contour image of the concrete curved component after SAM segmentation, and the contour in the image is repaired in combination with the post-processing process. The process is as follows:

[0052] S3.1. Identify and analyze the contour line trajectory in the contour image to determine whether there is any occlusion of the target curved component in the extracted contour image. If there is any occlusion of the target curved component, it will affect the accurate reflection of the shape of the target curved component;

[0053] S3.2. Based on the judgment result, fill the outline of the outline image and repair the edge lines.

[0054] Preferably, if there is occlusion of the target curved component in the contour image, the occluded part is erased, and after the missing part selection is established, the missing part is repaired by content recognition filling, and then the corrected contour image of the target curved component is output;

[0055] If there is no occlusion of the target curved component in the contour image, directly output the contour image of the target curved component;

[0056] S4. Identify cracks in the target curved component: The process is as follows:

[0057] S4.1. Use a sliding non-overlapping window method to identify cracks on a target curved component and extract a set of cracks distributed on the target curved component;

[0058] For the crack recognition model, the OOD crack segmentation method based on prior knowledge can be used. First, establish training data, manually evaluate the quality of existing data, remove data with low annotation quality, and select the remaining high-quality annotated data as candidate data; compare the characteristics of the target scene, and match the candidate data with the target scene based on similarity (such as the same scene); based on the above matching results, organize high-quality annotated data with similar characteristics to the expected application scenario for model training; use the pre-trained weights from a large data set to initialize the deep learning model, and fine-tune the model to obtain the fine-tuning weights; use the fine-tuned existing model to apply to the target OOD crack image to obtain the crack segmentation image result. The steps are as follows:

[0059] S4.1.1. Use the lightweight semantic segmentation network SegFormer-B0. The data sets are CFD, CrackTree and DeepCrack. Each data set is randomly divided into training set, validation set and test set in a ratio of 8:1:1. The training set, validation set and test set are cross-merged and used for model training. The batch size used in the model training process is 8.

[0060] S4.1.2. Pad the image to make it consistent with the size of the perspective-corrected image, and divide the corrected image into image blocks with the same training configuration. The unit module of the image block is set to a rectangle to reduce the performance fluctuation caused by the inconsistency between the input image size and the training stage when the model is applied;

[0061] S4.1.3, use the trained model to segment the cracks in the image blocks;

[0062] S4.1.4, after segmenting the cracks in the image blocks, stitching the results together according to their numbers to reconstruct the entire image;

[0063] S4.1.5, cropping the non-image part to obtain the binary crack identification result, and saving it in a lossless compressed image format (such as PNG) to maintain the pixel value accuracy without introducing color changes;

[0064] S4.1.6. Using the dynamic interaction method, based on OpenCV, circles of different sizes and the same color as the background are used to cover the noise, and the final image is obtained as the crack segmentation result;

[0065] S4.2, capturing at least one obvious reference point on the contour boundary of the target curved component, and marking at least one reference point on the identified and extracted crack set map;

[0066] S4.3, measuring the distance from the reference point to the benchmark point, establishing a plane coordinate system based on the benchmark point and the reference point and their relative position relationship, and obtaining the distribution relationship of the cracks on the target curved component;

[0067] S5. Generate a crack morphology diagram of the target curved component: The process is as follows:

[0068] S5.1. Adjust the image ratio of the contour image and the crack set map so that the contour image and the crack set map are scaled in the same proportion as the original image of the target curved component obtained;

[0069] S5.2, locate the reference point on the contour image and the reference point on the crack morphology map, and calculate the adaptation distance between the reference point and the reference point according to the scaling ratio of the contour image and the crack set map;

[0070] S5.3. According to the plane coordinate system relationship, the crack morphology diagram of the same scale is superimposed on the contour image, and the crack morphology diagram of the target curved component is obtained after combination, which is used to describe the crack morphology image of the crack distribution on the concrete curved component.

[0071] As an embodiment of the present invention:

[0072] like Figure 1 As shown in the figure, a T-shaped concrete beam bending test photo is selected, and the image resolution is 3648×2736 pixels. The specimen is 2500 mm long and 350 mm high. The SAM-based curved component contour extraction method has the following steps:

[0073] Step 1, obtaining an image of a curved component with cracks: obtaining an image containing the outline of the curved component;

[0074] Step 2, using SAM to extract curved components: using the interactive interface, use the point selection or box selection method to prompt SAM to extract curved components, and select the target curved component image to be segmented in the image by clicking the upper left corner and the lower right corner and dragging to adjust the prompt box position;

[0075] Step 3, extract its contour through cv2.findContours function: After obtaining the segmentation result of the target curved component, use cv2.findContours function to extract its contour shape, and save it as the contour extraction result of the target curved component;

[0076] Step 4: Post-process the extraction results: In some cases, there are some occlusions of the target curved component in the image, and the extracted beam contour may not accurately reflect the shape of the beam. Therefore, post-processing is required for the target curved component. The specific post-processing method depends on the situation.

[0077] like Figure 2 , a method for crack identification using sliding non-overlapping windows based on deep learning, comprising the following steps:

[0078] Step 1, model training: Use the lightweight semantic segmentation network SegFormer-B0, and the data sets use CFD, CrackTree and DeepCrack. Each data set is randomly divided into training set, validation set and test set in a ratio of 8:1:1, and they are merged for model training respectively. The batch size used in the training process is 8;

[0079] Step 2: Image preprocessing: pad the image to make it consistent with the perspective-corrected image size, and divide the corrected image into image blocks with the same training configuration, specifically 512×512, in order to reduce the performance fluctuation caused by the inconsistency between the input image size and the training stage when the model is applied;

[0080] Step 3, crack segmentation: Use the trained model to segment the cracks in the image block, and the crack segmentation image block;

[0081] Step 4, stitching image blocks: After segmenting the cracks in the image blocks, stitch the results together according to their numbers to reconstruct the entire image;

[0082] Step 5, crop the image: crop the non-image part to obtain the binary crack recognition result, and save it in a lossless compression format such as PNG to maintain the pixel value accuracy without introducing color changes;

[0083] Step 6: Eliminate noise: Use the dynamic interaction method based on OpenCV to cover the noise with circles of different sizes and the same color as the background to obtain the final image as the crack segmentation result.

[0084] The OOD crack segmentation method based on prior knowledge includes the following steps:

[0085] Step 1: Screen candidate data: When establishing training data, first manually evaluate the quality of existing data, remove data with low annotation quality, and select the remaining high-quality annotated data as candidate data;

[0086] Step 2: Screen target scenario data: Compare the characteristics of the target scenario and match the candidate data with the target scenario based on the same scenario. Based on the results of these two steps, organize high-quality annotated data with similar characteristics to the expected application scenario for model training;

[0087] Step 3: Model training: Initialize the deep learning model using pre-trained weights from a large dataset and fine-tune the model to obtain fine-tuned weights.

[0088] Step 4: Model application: Apply the fine-tuned existing model to the target OOD crack image to obtain the crack segmentation result image.

[0089] The OOD crack segmentation method based on prior knowledge is used to improve the crack extraction effect. The experimenters will have expectations for the crack results that appear in the experiment. According to the expected results, they can select high-quality labeled data as candidate data in a targeted manner to fully improve the training effect, thereby ultimately improving the measurement accuracy. The impact of different training data and domain knowledge on the model's crack detection ability has been experimentally analyzed, and the performance of the model has been evaluated on the test set and the external test set, proving that the crack segmentation method based on prior knowledge can not only improve the model performance, but also significantly shorten the training time. In the experimental results, the closest average value of the Dice coefficient obtained on the test set corresponding to the training data is selected as the pre-training weight for in-domain transfer learning. According to the experimental results, by manually filtering data with high-quality annotations and similar to the target scene, the model training efficiency can be optimized and the performance in OOD data can be improved.

[0090] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for drawing crack images of curved components based on SAM, characterized in that: The following steps are involved: S1. Acquire a target curved component image: positively photograph the target curved component image including the curved component and cracks, upload the target curved component image to a processor and store it; S2. Segmenting the target curved component: using an interactive interface connected to the processor, selecting and extracting the segmentation result of the target curved component from the acquired image; S3, extracting the contour of the target curved component: identifying the contour edge boundary line of the target curved component from the segmentation result, and performing post-processing correction on it to obtain the contour image of the target curved component; S4. Identify cracks in the target curved component: The process is as follows: S4.

1. Use a sliding non-overlapping window method to identify cracks on a target curved component and extract a set of cracks distributed on the target curved component; S4.2, capturing at least one obvious reference point on the contour boundary of the target curved component, and marking at least one reference point on the identified and extracted crack set map; S4.3, measuring the distance from the reference point to the benchmark point, establishing a plane coordinate system based on the benchmark point and the reference point and their relative position relationship, and obtaining the distribution relationship of the cracks on the target curved component; S5. Generate a crack morphology diagram of the target curved component: The process is as follows: S5.

1. Adjust the image ratio of the contour image and the crack set map so that the contour image and the crack set map are scaled in the same proportion as the original image of the target curved component obtained; S5.2, locate the reference point on the contour image and the reference point on the crack morphology map, and calculate the adaptation distance between the reference point and the reference point according to the scaling ratio of the contour image and the crack set map; S5.

3. Based on the plane coordinate system relationship, the crack morphology diagram of the same scale is superimposed on the contour image, and the crack morphology diagram of the target curved component is obtained after combination.

2. The method for drawing crack images of curved components based on SAM according to claim 1, characterized in that: In S2, when extracting the contour of the target curved component, use the point or box prompt SAM on the interactive interface to extract the target curved component, click the upper left corner and the lower right corner to define the box prompt, drag the definition box to move the position and adjust the area of ​​the definition box to split the target curved component pattern within the selected range of the definition box area.

3. The method for drawing crack images of curved components based on SAM according to claim 1, characterized in that: In S3, the cv2.findContours function is used to extract the contour shape of the target curved component, and the contour in the image is repaired in combination with the post-processing process. The process is as follows: S3.1, identifying and analyzing the contour line trajectory in the contour image, and determining whether there is any occlusion of the target curved component in the extracted contour image; S3.

2. Based on the judgment result, fill the outline of the outline image and repair the edge lines.

4. The method for drawing crack images of curved components based on SAM according to claim 3, characterized in that: If there is occlusion of the target curved component in the contour image, the occluded part is erased, and after the missing part selection is established, the missing part is repaired by content-aware filling, and then the contour image of the corrected target curved component is output; If there is no occlusion of the target curved component in the contour image, the contour image of the target curved component is directly output.

5. The method for drawing crack images of curved components based on SAM according to claim 1, characterized in that: In S4.1, the steps for crack identification of the target curved component using the sliding non-overlapping window method are as follows: S4.1.

1. Use the lightweight semantic segmentation network SegFormer-B0. The datasets are CFD, CrackTree and DeepCrack. Each dataset is randomly divided into training set, validation set and test set in a ratio of 8:1:

1. The training set, validation set and test set are cross-merged and used for model training. S4.1.2, pad the image to make it consistent with the size of the perspective-corrected image, and divide the corrected image into image blocks with the same training configuration; S4.1.3, use the trained model to segment the cracks in the image blocks; S4.1.4, after segmenting the cracks in the image blocks, stitching the results together according to their numbers to reconstruct the entire image; S4.1.5, trimming off the non-image part, obtaining the binary crack recognition result, and saving it in a lossless compressed image format; S4.1.

6. Using the dynamic interaction method, based on OpenCV, circles of different sizes and the same color as the background are used to cover the noise and obtain the final image as the crack segmentation result.

6. The method for drawing crack images of curved components based on SAM according to claim 1, characterized in that: In S4.1.1, the batch size used during model training is 8.

7. The method for drawing crack images of curved components based on SAM according to claim 1, characterized in that: In S4.1.2, the unit module of the image block is set to be a rectangle.

8. The method for drawing crack images of curved components based on SAM according to claim 1, characterized in that: In S4.1.5, the binary crack identification results are stored in PNG format.

Images