Multi-scale feature fusion concrete defect detection method based on improved SAM

Abstract

The invention is suitable for the technical field of computer vision and deep learning, and provides a multi-scale feature fusion concrete defect detection method based on improved SAM, and the method comprises the steps: obtaining a concrete defect image data set; preprocessing the image data set and dividing the image data set into a training set, a verification set and a test set; the YOLOv9 is trained to automatically detect concrete defects; constructing an improved SAM model, replacing a prompt encoder of the SAM with YOLOv9 and edge detection, and modifying a mask encoder of the SAM; the training effect of the model is evaluated through the four indexes of the accuracy rate, the recall rate, the F1 value and the intersection-to-union ratio, and the improved SAM model is optimized according to the training effect. By improving the structure of the SAM, the detection capability of the model on small defects can be enhanced, and the detection accuracy and efficiency are improved. In addition, according to the method, the complexity of the model can be reduced, and the training efficiency and generalization ability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technologies, and in particular to a multi-scale feature fusion concrete defect detection method based on an improved SAM. Background Art

[0002] Existing CNN-based defect detection models such as ResNet, VGGNet, and DenseNet, as well as object detection algorithms such as YOLO and SSD, all enable defect detection in concrete images and can determine whether an image contains cracks. While these methods achieve automated detection, they still suffer from issues such as insufficient multi-scale defect detection and interference from complex backgrounds. When processing multi-scale information, traditional feature pyramids struggle to account for both large-scale spalling and microcracks, resulting in a high rate of missed detection of small objects. When the concrete surface texture in an image is complex and the contrast between the defect and the background is low, existing attention mechanisms (such as channel attention) cannot effectively distinguish defect areas from non-target noise.

[0003] SAM is a general image segmentation model that can generate masks for arbitrary cues and exhibits strong generalization and adaptability. However, SAM also has some limitations in concrete defect detection. For example, upsampling during the decoding process fails to restore image details, resulting in inaccurate detection of small defects. Furthermore, SAM has shortcomings in multi-scale feature fusion, making it difficult to effectively handle multi-scale concrete defects. Summary of the Invention

[0004] The present invention provides a multi-scale feature fusion concrete defect detection method based on improved SAM to solve the technical problems mentioned in the background technology.

[0005] A multi-scale feature fusion concrete defect detection method based on improved SAM, the method comprising:

[0006] Step S1, obtaining a concrete defect image dataset;

[0007] Step S2: preprocess the image dataset and divide it into training set, validation set and test set;

[0008] Step S3: training YOLOv9 to automatically detect concrete defects;

[0009] Step S4: construct an improved SAM model, use YOLOv9 and edge detection to replace the SAM hint encoder, and modify the SAM mask encoder;

[0010] Step S5: Evaluate the effect of model training through the four indicators of precision, recall, F1 value and intersection-over-union ratio, and optimize the improved SAM model according to the training effect to obtain high-precision automatic concrete defect segmentation.

[0011] As a further technical solution of the present invention, in step S2, the steps of preprocessing the image dataset and dividing it into a training set, a validation set, and a test set include:

[0012] By adding Gaussian noise to the image to simulate the noise interference in actual shooting, a Gaussian noise matrix with a mean of 0 and a standard deviation of 50 is selected and added to the original image. The color channels of the image are randomly jittered to change the color saturation, hue and brightness. The image is first converted from RGB space to HSV space, and then the hue channel is adjusted using random numbers;

[0013] Convert the image from HSV space back to RGB space to obtain the modified image, generate a random deformation field to simulate deformation, use Gaussian function to generate a smooth deformation field, and then use bilinear interpolation or nearest neighbor interpolation to move the image pixels to the new position to obtain the deformed image; after completing the above operations, for images with a size greater than 800×800, directly use them as training samples. For images with a size less than 800×800, use the Mosaic method to stitch four images into one to generate new training samples. Each sample is composed of four images. Randomly select four original images, and each image is modified to 400×400 pixels by compression or stretching. Then, place these four images at the four corners of the new 800×800 image to form a 2×2 grid to ensure that they are seamlessly spliced ​​in the new image. Finally, adjust the corresponding defect label of each image to correspond to the position in the new image to ensure the accuracy and consistency of the label.

[0014] The ratio of training set, validation set and test set is 7:2:1.

[0015] As a further technical solution of the present invention, in step S3, the steps of training YOLOv9 to automatically detect concrete defects are as follows:

[0016] The preprocessed defect image dataset was input into the improved network for 400 iteration cycles of training; starting from the first training cycle of the model, the Adam optimizer was used in the early stage for fast convergence, and the learning rate was set to 0.01; when the validation set loss did not decrease for a long time, the subsequent training adopted stochastic gradient descent optimization. Before switching to stochastic gradient descent optimization, the learning rate of Adam was gradually reduced through the learning rate scheduler; when switching to stochastic gradient descent optimization, the learning rate was set to 0.001, the momentum was set to 0.937, the weight decay coefficient was 0.0005, the training batch size was 16, and training continued until the model training rounds reached 400. At the same time, in order to obtain the best generalization ability and prevent overfitting, the early stopping method was introduced in the training process.

[0017] As a further technical solution of the present invention, in step S4, the steps of constructing an improved SAM model, using YOLOv9 and edge detection to replace the SAM hint encoder, and modifying the SAM mask encoder include:

[0018] The original SAM network structure mainly consists of three parts: image encoder, hint encoder and mask decoder; SAM uses an image encoder based on Vision Transformer; the hint encoder is used to embed the sparse hints and dense hints provided by the user into the image through position encoding or convolution operations so as to combine them with image features; the mask decoder is responsible for combining the image embedding and the hint embedding to generate the final segmentation mask; when outputting the segmentation result, the image embedding is upsampled using transposed convolution and the final mask output is generated through a multi-layer perceptron; at the same time, the confidence of the mask is also predicted to evaluate the segmentation quality;

[0019] In order to realize automatic defect recognition, the trained Yolov9 object detection results and Canny edge detection results of the image are used as prompt information and input into the Prompt Encoder;

[0020] Among them, Canny edge detection converts the input image into a grayscale image and suppresses noise interference through adaptive Gaussian filtering;

[0021] ;

[0022] σ: standard deviation of Gaussian filtering, which determines the filtering strength, I i : grayscale value of local pixel, μ: mean value of local pixel, n: total number of pixels in the local window, k: noise sensitivity coefficient;

[0023] Dynamically adjust the smoothness of Gaussian filtering to suppress noise while retaining edge details;

[0024] Use the Sobel operator to calculate the x and y gradients, combine the gradient direction to perform non-maximum suppression, retain the main edge contour, set the high threshold Thigh = 100 and the low threshold Tlow = 50, and generate a binary edge map through connectivity analysis;

[0025] For the mask decoder, the multi-level feature fusion decoder architecture is improved, and the single upsampling process can be expressed as:

[0026] ,in is the transposed convolution kernel parameter, σ is the activation function;

[0027] Multi-stage refinement expansion, when splitting a single upsampling into two stages, the operation of the i-th stage is:

[0028] ;

[0029] ;

[0030] ;

[0031] Among them E i is the encoder layer i feature, F skip Contains skip connection processing of 3×3 convolution, Convolution: adjusts the channel dimension to match the decoder. BN layer: stabilizes gradient propagation and accelerates training convergence. ReLU activation: enhances nonlinear expression capabilities.

[0032] Set up the feature pyramid fusion mechanism, cross-layer feature alignment, encoder shallow feature E i and the decoder deep feature D h The integration must meet the following requirements:

[0033] Resolution(D h )=Resolution(E l );

[0034] Scale alignment is achieved through bilinear interpolation:

[0035] Align(D h ,E l )=I(D h ,scale=2 ∣h-l∣ );

[0036] Where I is the interpolation function;

[0037] Use learnable projection convolution to adjust the channels so that the dimensions match:

[0038] Guarantee C l =Cd Implement element-by-element addition;

[0039] Through a complete multi-level refinement process, the segmentation accuracy of fine textures is further improved;

[0040] E4 layer, 1 / 4 resolution: preserves concrete surface texture details;

[0041] E8 layer, 1 / 8 resolution: captures high-frequency information of crack edges.

[0042] As a further technical solution of the present invention, in step S5, the step of evaluating the effect of model training by using the four indicators of precision, recall, F1 value and intersection-over-union ratio includes:

[0043] Accuracy:

[0044] Definition: Precision is a measure of the proportion of samples that are actually positive among the samples predicted by the model to be positive. It focuses on the accuracy of the model's predictions.

[0045] formula: ;

[0046] TP: True positive examples, that is, the number of samples correctly predicted by the model as positive;

[0047] FP: False positive, that is, the number of samples that the model incorrectly predicts as positive;

[0048] Recall:

[0049] Definition: Recall is a measure of the proportion of samples that are correctly predicted as positive among samples that are actually positive. It focuses on the recall ability of the model.

[0050] formula: ;

[0051] TP: True positive examples, that is, the number of samples correctly predicted by the model as positive;

[0052] FN: False negative examples, that is, the number of samples that the model incorrectly predicts as negative;

[0053] F1 value:

[0054] Definition: F1 value is the harmonic mean of precision and recall, which is used to comprehensively measure the performance of the model;

[0055] formula: ;

[0056] Intersection-over-Union Ratio:

[0057] Definition: Intersection over Union (IoU) is a measure of the degree of overlap between the segmented regions predicted by the model and the true segmented regions. It is commonly used in image segmentation tasks.

[0058] formula: ;

[0059] TP: the number of pixels correctly predicted by the model as positive;

[0060] FP: the number of pixels that the model incorrectly predicts as positive;

[0061] FN: The number of pixels that the model incorrectly predicted as negative.

[0062] Beneficial effects achieved by the present invention:

[0063] This paper proposes a multi-scale feature fusion concrete defect detection method based on an improved SAM. By improving the SAM structure, the model's ability to detect small defects is enhanced, improving detection accuracy and efficiency. Furthermore, the method reduces model complexity, improving training efficiency and generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is the overall flow chart of the present invention.

[0065] Figure 2 This is the concrete defect segmentation training flow chart of the improved SAM of the present invention.

[0066] Figure 3 This is a structural diagram of the mask decoder part of the improved SAM of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0068] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0069] See also Figures 1 to 3, an embodiment of the present invention provides a multi-scale feature fusion concrete defect detection method based on an improved SAM, the method comprising:

[0070] Step S1, obtaining a concrete defect image dataset;

[0071] Step S2: preprocess the image dataset and divide it into training set, validation set and test set;

[0072] Step S3: training YOLOv9 to automatically detect concrete defects;

[0073] Step S4: construct an improved SAM model, use YOLOv9 and edge detection to replace the SAM hint encoder, and modify the SAM mask encoder;

[0074] Step S5: Evaluate the effect of model training through the four indicators of precision, recall, F1 value and intersection-over-union ratio, and optimize the improved SAM model according to the training effect to obtain high-precision automatic concrete defect segmentation.

[0075] In this embodiment, in step S1, the method of the present invention uses three public defect image datasets, which are the LCW dataset, the Crackseg9k dataset, and the Facade390 dataset.

[0076] The LCW dataset is a comprehensive dataset that includes a variety of concrete structure defects in complex backgrounds. The dataset contains 3,774 images, ranging in size from 266 x 355 to 4,416 x 3,312 pixels. This dataset includes images of defects such as concrete cracking, surface peeling, and steel corrosion, as well as images of non-defective concrete structures for comparison and to test recognition performance in these conditions.

[0077] The Crackseg9k dataset contains 9,255 images that are a combination of different smaller open-source datasets. The images in the dataset have been preprocessed and resized to 400x400 pixels. It combines Crack500, Deepcrack, Sdnet, Cracktree, Gaps, Volker, Rissbilder, Noncrack, Masonry, and Ceramic datasets.

[0078] The Facade390 dataset contains 390 images of cracked concrete walls, all with a pixel size of 400x400.

[0079] The above datasets all include concrete defects that were manually identified and annotated, and the outlines of the defects in the images were segmented from black and white binary images. The images in the datasets have different lighting conditions such as shadows and occlusions.

[0080] In this embodiment, in step S2, the steps of preprocessing the image dataset and dividing it into a training set, a validation set, and a test set include:

[0081] By adding Gaussian noise to the image to simulate the noise interference in actual shooting, a Gaussian noise matrix with a mean of 0 and a standard deviation of 50 is selected and added to the original image. The color channels of the image are randomly jittered to change the color saturation, hue and brightness. The image is first converted from RGB space to HSV space, and then the hue channel is adjusted using random numbers;

[0082] Convert the image from HSV space back to RGB space to obtain the modified image, generate a random deformation field to simulate deformation, use Gaussian function to generate a smooth deformation field, and then use bilinear interpolation or nearest neighbor interpolation to move the image pixels to the new position to obtain the deformed image; after completing the above operations, for images with a size greater than 800×800, directly use them as training samples. For images with a size less than 800×800, use the Mosaic method to stitch four images into one to generate new training samples. Each sample is composed of four images. Randomly select four original images, and each image is modified to 400×400 pixels by compression or stretching. Then, place these four images at the four corners of the new 800×800 image to form a 2×2 grid to ensure that they are seamlessly spliced ​​in the new image. Finally, adjust the corresponding defect label of each image to correspond to the position in the new image to ensure the accuracy and consistency of the label.

[0083] The ratio of training set, validation set and test set is 7:2:1.

[0084] As a further technical solution of the present invention, in step S3, the steps of training YOLOv9 to automatically detect concrete defects are as follows:

[0085] The preprocessed defect image dataset is input into the improved network for 400 iteration cycles of training; starting from the first training cycle of the model, the Adam optimizer is used in the early stage for fast convergence, and the learning rate is set to 0.01. Since the Adam optimizer performs well in processing high-dimensional sparse gradients, it can help the model quickly learn effective feature representations; when the validation set loss does not decrease for a long time, the subsequent training adopts stochastic gradient descent optimization. Before switching to stochastic gradient descent optimization, the learning rate of Adam is gradually reduced through the learning rate scheduler to ensure that Adam does not produce too large updates in the later stage of training, thereby making the switching smoother; when switching to stochastic gradient descent optimization, the learning rate is set to 0.001, the momentum is set to 0.937, the weight decay coefficient is 0.0005, the training batch size is 16, and training continues until the model training round reaches 400. At the same time, in order to obtain the best generalization ability and prevent overfitting, the early stopping method is introduced in the training process.

[0086] In this embodiment, in step S4, the steps of constructing an improved SAM model, using YOLOv9 and edge detection to replace the SAM hint encoder, and modifying the SAM mask encoder include:

[0087] The original SAM network structure mainly consists of three parts: image encoder, hint encoder and mask decoder; SAM uses an image encoder based on Vision Transformer, which has been pre-trained on large-scale data and can effectively extract high-level features of the image; the hint encoder is responsible for embedding the sparse hints (such as points and boxes) and dense hints (such as masks) provided by the user into the image through position encoding or convolution operations so as to combine them with the image features; the mask decoder is responsible for combining the image embedding and the hint embedding to generate the final segmentation mask; when outputting the segmentation result, the image embedding is upsampled using transposed convolution and the final mask output is generated through a multi-layer perceptron; at the same time, the confidence of the mask (IOU) is also predicted to evaluate the segmentation quality;

[0088] The original prompt encoder requires the user to provide prompt information, otherwise it cannot output the segmentation required by the user. In order to achieve automatic defect recognition, the trained Yolov9 object detection results and Canny edge detection results of the image are used as prompt information and input into the Prompt Encoder.

[0089] Among them, Canny edge detection converts the input image into a grayscale image and suppresses noise interference through adaptive Gaussian filtering (kernel size 3×3, standard deviation σ is dynamically adjusted according to local noise);

[0090] ;

[0091] σ: standard deviation of Gaussian filtering, which determines the filtering strength, I i : grayscale value of local pixel, μ: mean value of local pixel, n: total number of pixels in the local window, k: noise sensitivity coefficient (take 0.5);

[0092] Dynamically adjust the smoothness of Gaussian filtering to suppress noise while retaining edge details;

[0093] Use the Sobel operator to calculate the x and y gradients, combine the gradient direction to perform non-maximum suppression, retain the main edge contour, set the high threshold Thigh = 100 and the low threshold Tlow = 50, and generate a binary edge map through connectivity analysis;

[0094] For the mask decoder, the SAM encoder adopts an image block embedding strategy, which inevitably loses detail information; on the other hand, the upsampling strategy in the SAM decoder cannot restore key details. Therefore, salient objects with complex details and edges cannot be fully captured. Figure 3 As shown, the following modifications are made:

[0095] For the mask decoder, the multi-level feature fusion decoder architecture is improved, and the single upsampling process can be expressed as:

[0096] ,in is the transposed convolution kernel parameter, σ is the activation function;

[0097] Multi-stage refinement expansion, when splitting a single upsampling into two stages, the operation of the i-th stage is:

[0098] ;

[0099] ;

[0100] ;

[0101] Among them E i is the encoder layer i feature, F skip Contains skip connection processing of 3×3 convolution, Convolution: adjusts the channel dimension to match the decoder. BN layer: stabilizes gradient propagation and accelerates training convergence. ReLU activation: enhances nonlinear expression capabilities.

[0102] Set up the feature pyramid fusion mechanism, cross-layer feature alignment, encoder shallow feature E i and the decoder deep feature D h The integration must meet the following requirements:

[0103] Resolution(D h)=Resolution(E l );

[0104] Scale alignment is achieved through bilinear interpolation:

[0105] Align(D h ,E l )=I(D h ,scale=2 ∣h-l∣ );

[0106] Where I is the interpolation function;

[0107] Use learnable projection convolution to adjust the channels so that the dimensions match:

[0108] Guarantee C l =C d Implement element-by-element addition;

[0109] Through a complete multi-level refinement process, the segmentation accuracy of fine textures is further improved;

[0110] E4 layer, 1 / 4 resolution: preserves concrete surface texture details;

[0111] E8 layer, 1 / 8 resolution: captures high-frequency information of crack edges.

[0112] In this embodiment, in step S5, the step of evaluating the effect of model training using the four indicators of precision, recall, F1 value, and intersection-over-union ratio includes:

[0113] Precision (Pr):

[0114] Definition: Precision is a measure of the proportion of samples that are actually positive among the samples predicted by the model to be positive. It focuses on the accuracy of the model's predictions.

[0115] formula: ;

[0116] TP (True Positive): True positive, that is, the number of samples correctly predicted by the model as positive;

[0117] FP (False Positive): False positive examples, that is, the number of samples that the model incorrectly predicts as positive;

[0118] Significance: A high precision means that most of the samples predicted as positive by the model are actually positive, which is very important for reducing false positives.

[0119] Recall (Re):

[0120] Definition: Recall is a measure of the proportion of samples that are correctly predicted as positive among samples that are actually positive. It focuses on the recall ability of the model.

[0121] formula: ;

[0122] TP (True Positive): True positive, that is, the number of samples correctly predicted by the model as positive;

[0123] FN (False Negative): False negative examples, that is, the number of samples that the model incorrectly predicts as negative;

[0124] Significance: A high recall rate means that the model can correctly identify most of the samples that are actually positive. This is very important for reducing false negatives.

[0125] F1 Score (F1):

[0126] Definition: F1 value is the harmonic mean of precision and recall, which is used to comprehensively measure the performance of the model;

[0127] formula: ;

[0128] Significance: The F1 score combines precision and recall and is suitable for situations where a balance between the two is important. If one of these is high and the other is low, the F1 score will be low, reflecting an imbalance in the model.

[0129] Intersection over Union (IoU):

[0130] Definition: Intersection over Union (IoU) is a measure of the degree of overlap between the segmented regions predicted by the model and the true segmented regions. It is commonly used in image segmentation tasks.

[0131] formula: ;

[0132] TP (True Positive): The number of pixels correctly predicted by the model as positive;

[0133] FP (False Positive): The number of pixels that the model incorrectly predicts as positive;

[0134] FN (False Negative): The number of pixels that the model incorrectly predicts as negative;

[0135] Meaning: The closer the IoU value is to 1, the closer the segmentation area predicted by the model is to the true segmentation area, and the better the segmentation effect. An IoU value of 0 indicates no overlap, and an IoU value of 1 indicates complete overlap.

[0136] It should be noted that, in this document, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0137] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-scale feature fusion concrete defect detection method based on improved SAM, characterized in that: The method comprises: Step S1, obtaining a concrete defect image dataset; Step S2: preprocess the image dataset and divide it into training set, validation set and test set; Step S3: training YOLOv9 to automatically detect concrete defects; Step S4: construct an improved SAM model, use YOLOv9 and edge detection to replace the SAM hint encoder, and modify the SAM mask encoder; Step S5: Evaluate the effect of model training through the four indicators of precision, recall, F1 value and intersection-over-union ratio, and optimize the improved SAM model according to the training effect to obtain high-precision automatic concrete defect segmentation.

2. The method for detecting concrete defects by multi-scale feature fusion based on improved SAM according to claim 1 is characterized in that: In step S2, the steps of preprocessing the image dataset and dividing it into a training set, a validation set, and a test set include: By adding Gaussian noise to the image to simulate the noise interference in actual shooting, a Gaussian noise matrix with a mean of 0 and a standard deviation of 50 is selected and added to the original image. The color channels of the image are randomly jittered to change the color saturation, hue and brightness. The image is first converted from RGB space to HSV space, and then the hue channel is adjusted using random numbers; Convert the image from HSV space back to RGB space to obtain the modified image, generate a random deformation field to simulate deformation, use Gaussian function to generate a smooth deformation field, and then use bilinear interpolation or nearest neighbor interpolation to move the image pixels to the new position to obtain the deformed image; after completing the above operations, for images with a size greater than 800×800, directly use them as training samples. For images with a size less than 800×800, use the Mosaic method to stitch four images into one to generate new training samples. Each sample is composed of four images. Randomly select four original images, and each image is compressed or stretched to 400×400 pixels. Then, place these four images at the four corners of the new 800×800 image to form a 2×2 grid. Finally, adjust the corresponding defect label of each image to correspond to the position in the new image. The ratio of training set, validation set and test set is 7:2:

1.

3. The multi-scale feature fusion concrete defect detection method based on improved SAM according to claim 1 is characterized in that: In step S3, the steps for training YOLOv9 to automatically detect concrete defects are as follows: The preprocessed defect image dataset is input into the improved network for 400 iteration cycles of training; starting from the first training cycle of the model, the Adam optimizer is used in the early stage for fast convergence, and the learning rate is set to 0.01; when the validation set loss does not decrease for a long time, the subsequent training adopts stochastic gradient descent optimization. Before switching to stochastic gradient descent optimization, the learning rate of Adam is gradually reduced through the learning rate scheduler; when switching to stochastic gradient descent optimization, the learning rate is set to 0.001, the momentum is set to 0.937, the weight decay coefficient is 0.0005, the training batch size is 16, and training is continued until the model training rounds reach 400. At the same time, in order to obtain the best generalization ability and prevent overfitting, the early stopping method is introduced in the training process.

4. The method for concrete defect detection based on multi-scale feature fusion of improved SAM according to claim 1 is characterized in that: In step S4, an improved SAM model is constructed, and YOLOv9 and edge detection are used to replace the SAM hint encoder. The steps of modifying the SAM mask encoder include: The original SAM network structure mainly consists of three parts: image encoder, hint encoder and mask decoder; SAM uses an image encoder based on Vision Transformer; the hint encoder is used to embed the sparse hints and dense hints provided by the user into the image through position encoding or convolution operations so as to combine them with image features; the mask decoder is responsible for combining the image embedding and the hint embedding to generate the final segmentation mask; when outputting the segmentation result, the image embedding is upsampled using transposed convolution and the final mask output is generated through a multi-layer perceptron; at the same time, the confidence of the mask is also predicted to evaluate the segmentation quality; In order to realize automatic defect recognition, the trained Yolov9 object detection results and Canny edge detection results of the image are used as prompt information and input into the Prompt Encoder; Among them, Canny edge detection converts the input image into a grayscale image and suppresses noise interference through adaptive Gaussian filtering; ； σ: standard deviation of Gaussian filtering, which determines the filtering strength, I i : grayscale value of local pixel, μ: mean value of local pixel, n: total number of pixels in the local window, k: noise sensitivity coefficient; Dynamically adjust the smoothness of Gaussian filtering to suppress noise while retaining edge details; Use the Sobel operator to calculate the x and y gradients, combine the gradient direction to perform non-maximum suppression, retain the main edge contour, set the high threshold Thigh = 100 and the low threshold Tlow = 50, and generate a binary edge map through connectivity analysis; For the mask decoder, the multi-level feature fusion decoder architecture is improved, and the single upsampling process can be expressed as: ,in is the transposed convolution kernel parameter, σ is the activation function; Multi-stage refinement expansion, when splitting a single upsampling into two stages, the operation of the i-th stage is: ； ； ； Among them E i is the encoder layer i feature, F skip Contains skip connection processing of 3×3 convolution, Convolution: adjusts the channel dimension to match the decoder. BN layer: stabilizes gradient propagation and accelerates training convergence. ReLU activation: enhances nonlinear expression capabilities. Set up the feature pyramid fusion mechanism, cross-layer feature alignment, encoder shallow feature E i and the decoder deep feature D h The integration must meet the following requirements: Resolution(D h )=Resolution(E l ); Scale alignment is achieved through bilinear interpolation: Align(D h ,E l )=I(D h ,scale=2 ∣h-l∣ ); Where I is the interpolation function; Use learnable projection convolution to adjust the channels so that the dimensions match: Guarantee C l =C d Implement element-by-element addition; Through a complete multi-level refinement process, the segmentation accuracy of fine textures is further improved; E4 layer, 1 / 4 resolution: preserves concrete surface texture details; E8 layer, 1 / 8 resolution: captures high-frequency information of crack edges.

5. The method for concrete defect detection based on multi-scale feature fusion of improved SAM according to claim 1 is characterized in that: In step S5, the steps of evaluating the effectiveness of model training using the four indicators of precision, recall, F1 value, and intersection-over-union ratio include: Accuracy: Definition: Precision is a measure of the proportion of samples that are actually positive among the samples predicted by the model to be positive. It focuses on the accuracy of the model's predictions. official: ; TP: True positive examples, that is, the number of samples correctly predicted by the model as positive; FP: False positive, that is, the number of samples that the model incorrectly predicts as positive; Recall: Definition: Recall is a measure of the proportion of samples that are correctly predicted as positive among samples that are actually positive. It focuses on the recall ability of the model. official: ; TP: True positive examples, that is, the number of samples correctly predicted by the model as positive; FN: False negative examples, that is, the number of samples that the model incorrectly predicts as negative; F1 value: Definition: F1 value is the harmonic mean of precision and recall, which is used to comprehensively measure the performance of the model; official: ; Intersection-over-Union Ratio: Definition: Intersection over Union (IoU) is a measure of the degree of overlap between the segmented regions predicted by the model and the true segmented regions. It is commonly used in image segmentation tasks. official: ; TP: the number of pixels correctly predicted by the model as positive; FP: the number of pixels that the model incorrectly predicts as positive; FN: The number of pixels that the model incorrectly predicted as negative.

Images