FRP strip erosion detection method and device based on multi-scale target detection model

Abstract

The embodiment of the invention provides an FRP strip erosion detection method and device based on a multi-scale target detection model, and the method comprises the steps: introducing an SA-C2f attention mechanism module and a BiFPN-HMC feature fusion module into a YOLOv8n network model, and constructing the multi-scale target detection model; acquiring an erosion detection image sample set of the FRP strip, wherein the sample set is marked with a damage area and a damage category; training a multi-scale target detection model based on the sample set to obtain a trained multi-scale target detection model; and acquiring a to-be-detected FRP strip image, performing erosion detection on the FRP strip image by using the trained multi-scale target detection model, and outputting the position and feature information of an erosion defect.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of civil engineering material testing. Specifically, the present disclosure relates to a method and apparatus for detecting FRP strip erosion based on a multi-scale object detection model, and a computer-readable storage medium storing a computer program. Background Art

[0002] Due to its physical and chemical properties such as high strength, corrosion resistance, and low weight, FRP (Fiber Reinforced Polymer) has shown significant advantages in structural reinforcement, and can significantly improve the bearing capacity, seismic performance, and durability of structures. Especially in the field of civil engineering reinforcement, FRP is used to reinforce and repair infrastructure such as buildings, bridges, and tunnels. Although FRP has significant advantages in civil engineering, if the FRP material itself is damaged, it will directly affect the safety and stability of the engineering structure. In extreme environments, especially in desert areas, FRP materials face the challenge of sand erosion. The fine particles contained in the sand cause friction and abrasion on the surface of the FRP material, resulting in gradual erosion of its surface, and the internal organizational structure will also be damaged, affecting the stability of the engineering structure and even potentially causing huge economic losses. Therefore, it is necessary to detect and evaluate the existing damage of FRP materials.

[0003] The erosion damage faced by FRP materials in desert areas is often very concealed and complex. Traditional detection equipment and sensors may not adapt to environmental changes such as strong winds, sandstorms, and large temperature differences, resulting in unstable or distorted detection results. Moreover, the damage caused by sand erosion has multi-scale characteristics. It may cause slight surface damage over a large area, while at the same time, it may cause relatively serious damage in local areas. The degree and form of damage vary greatly, making it difficult for traditional damage detection methods, such as manual inspections or simple visual inspections, to effectively capture these small and widely distributed damages. Different from traditional structural damages (such as cracks and breakages), the damages caused by sand erosion often manifest as fine surface damages or microcracks, which are not obvious in the early stage and are usually difficult to accurately judge by conventional methods.

[0004] With the continuous development of deep learning technology, damage detection methods based on computer vision and neural networks have become a research hotspot. Compared with traditional manual inspection methods, deep learning algorithms have obvious advantages in dealing with complex, fuzzy, and multi-scale damage features. Especially in sand erosion detection, deep learning algorithms can learn the features of different types of damage through a large amount of training data, so as to achieve accurate detection and localization of small damages. Due to problems such as extreme environments, small erosion range sizes, and unclear erosion features in FRP material erosion detection, traditional deep learning algorithms cannot effectively detect large-scale and small erosion areas simultaneously when dealing with this multi-scale erosion pattern, thus ignoring some small damages or being unable to accurately locate the positions of damages. Summary of the Invention

[0005] The embodiments described in this article provide a method, device, and computer-readable storage medium storing a computer program for FRP strip erosion detection based on a multi-scale object detection model.

[0006] According to the first aspect of the present disclosure, a method for FRP strip erosion detection based on a multi-scale object detection model is provided, including: introducing an SA-C2f attention mechanism module and a BiFPN-HMC feature fusion module into the YOLOv8n network model to construct a multi-scale object detection model; obtaining an erosion detection image sample set of FRP strips, where the sample set is labeled with damage areas and damage categories; training the multi-scale object detection model based on the sample set to obtain a trained multi-scale object detection model; and obtaining an FRP strip image to be detected, and using the trained multi-scale object detection model to perform erosion detection on the FRP strip image, and output the positions and feature information of erosion defects.

[0007] In some embodiments of the present disclosure, introducing an SA-C2f attention mechanism module and a BiFPN-HMC feature fusion module into the YOLOv8n network model to construct a multi-scale object detection model includes: introducing an SA-C2f attention mechanism module into the backbone network of the YOLOv8n network model to add spatial and channel attention to the input feature map; integrating a BiFPN-HMC feature fusion module into the neck network of the YOLOv8n network model to refine, weight, and fuse the feature map extracted by the backbone network to obtain a multi-scale feature map.

[0008] In some embodiments of the present disclosure, the SA-C2f attention mechanism module includes a spatial attention sub-module, a channel attention sub-module, and a feature recombination unit; Among them, the spatial attention sub-module includes a horizontal pooling layer, a vertical pooling layer, a splicing module, a horizontal convolution, a vertical convolution, an Fc linear transformation module and a sigmoid activation function, a horizontal sigmoid activation function and a vertical sigmoid activation function. Among them, the horizontal pooling layer is used to perform a maximum pooling operation on the input feature map in the horizontal direction to obtain a horizontal feature map; the vertical pooling layer is used to perform a maximum pooling operation on the input feature map in the vertical direction to obtain a vertical feature map; the splicing module is used to splice the horizontal feature map and the vertical feature map to obtain a fused feature map; the horizontal convolution and the vertical convolution respectively perform convolution operations on the fused feature map to generate attention weights; the Fc linear transformation module and the sigmoid activation function are used to map the convolution output to the range of [0,1] to be used as attention weights for the input feature map. The channel attention sub-module is used to weight the channel features through a multi-layer perceptron to obtain dynamically adjusted channels; the feature recombination unit is used to combine spatial and channel information through channel rearrangement and feature fusion operations.

[0009] In some embodiments of the present disclosure, the SA-C2f attention mechanism module is used for an input feature map with dimensions C×H×W, where H and W represent the height and width of the feature map, and C represents the number of channels. The feature map is divided into g groups along the channel dimension to form grouped feature maps , where ; each subgroup obtains a sub-feature group through the spatial attention module and corresponding weight coefficients; the sub-feature group applies a random unit through two branches of space and channel to simultaneously construct channel attention and spatial attention to be used as attention weights for the input feature map.

[0010] In some embodiments of the present disclosure, the BiFPN-HMC feature fusion module includes a bidirectional feature pyramid network, a multi-convolution module, a feature weighting mechanism and a residual connection; the bidirectional feature pyramid network is used to fuse feature maps of different scales through bidirectional feature flow in the up and down paths; the multi-convolution module uses depthwise separable convolution and variable dilation rate convolution to obtain erosion feature details; the feature weighting mechanism is used to assign dynamic weights to feature maps of different scales.

[0011] In some embodiments of the present disclosure, the BiFPN-HMC feature fusion module is used to obtain a feature map from the backbone network; extract feature maps of different scales through upsampling and downsampling operations; use depthwise separable convolution and variable dilation rate convolution to refine the feature maps of different scales; perform channel weighting adjustment and spatial information optimization through the channel attention module and the spatial attention module respectively, and recombine the feature maps after channel weighting adjustment and spatial information optimization to output multi-scale feature maps.

[0012] In some embodiments of the present disclosure, an erosion detection image sample set of FRP strips is obtained, and the sample set is labeled with damage regions and damage categories, including: obtaining FRP strip images simulating sand and wind erosion damage under different shooting angles and lighting conditions; labeling the damage regions and damage categories in the FRP strip images.

[0013] In some embodiments of the present disclosure, a multi-scale object detection model is trained based on the sample set to obtain a trained multi-scale object detection model, including: dividing the labeled images into a training set and a validation set, where the training set is used to adjust the parameters of the multi-scale object detection model, and the validation set is used to evaluate the performance of the multi-scale object detection model; inputting the training set into the multi-scale object detection model, and minimizing the BCEWithLogitsLoss loss function through backpropagation to update the model parameters, obtaining a trained multi-scale object detection model, and the BCEWithLogitsLoss damage function is: ; In the formula, is the number of samples, is the true label of the th sample, taking values of 0 or 1, is the predicted value of the model for the th sample without passing through the activation function.

[0014] According to a second aspect of the present disclosure, there is provided an FRP strip erosion detection device based on a multi-scale object detection model, including at least one processor; and at least one memory storing a computer program. When the computer program is executed by the at least one processor, the device: introduces an SA-C2f attention mechanism module and a BiFPN-HMC feature fusion module into the YOLOv8n network model to construct a multi-scale object detection model; obtains an erosion detection image sample set of FRP strips, and the sample set is labeled with damage regions and damage categories; trains the multi-scale object detection model based on the sample set to obtain a trained multi-scale object detection model; and obtains an FRP strip image to be detected, and uses the trained multi-scale object detection model to perform erosion detection on the FRP strip image, and outputs the position and characteristic information of the erosion defect.

[0015] According to a third aspect of the present disclosure, there is provided a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the FRP strip erosion detection method based on a multi-scale object detection model according to the first aspect of the present disclosure.

[0016] The FRP strip erosion detection method based on a multi-scale object detection model according to an embodiment of the present disclosure effectively enhances the model's ability to capture local features by introducing an attention mechanism, and effectively combines the spatial attention mechanism and the channel attention mechanism using the SA-C2f module, thereby more comprehensively capturing the relationships between features and enhancing the channel and spatial information of the feature map; based on the BiFPN-HMC feature fusion module, effective information transfer and fusion can be performed between features of different scales, and large-scale and small-scale erosion damages can be detected more accurately. Compared with existing model detection methods, it has higher accuracy, robustness, and multi-scale detection capabilities, and can also provide accurate position and damage classification information, significantly improving the efficiency and accuracy of FRP strip erosion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To briefly describe the technical solutions of the embodiments of the present disclosure more clearly, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the following described drawings only relate to some embodiments of the present disclosure and do not limit the present disclosure, where: Figure 1 is a schematic structural diagram of the Yolov8n model according to an embodiment of the present disclosure; Figure 2 shows an exemplary flowchart of the FRP strip erosion detection method 200 based on a multi-scale object detection model according to an embodiment of the present disclosure; Figure 3 is a schematic structural diagram of the SA-C2f attention mechanism module according to an embodiment of the present disclosure; Figure 4 is a schematic structural diagram of the BiFPN-HMC feature fusion module according to an embodiment of the present disclosure; Figure 5 is an application environment diagram of FRP strip erosion detection according to an embodiment of the present disclosure; Figure 6 is a schematic diagram of an FRP strip according to an embodiment of the present disclosure; Figure 7 is a schematic block diagram of the FRP strip erosion detection device 700 based on a multi-scale object detection model according to an embodiment of the present disclosure.

[0018] It should be noted that the elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of the present disclosure without creative efforts also fall within the scope of protection of the present disclosure.

[0020] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the subject matter of the present disclosure pertains. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the specification and the relevant art, and will not be interpreted in an idealized or overly formal form unless expressly defined otherwise herein.

[0021] The Yolov8n model is a lightweight version in the YOLO (You Only Look Once) series and consists of four main parts: the input end, the backbone network, the neck network, and the detection head. Figure 1 It is a schematic structural diagram of the Yolov8n model according to an embodiment of the present disclosure. Referring to Figure 1 As shown, the SPPF (Spatial Pyramid Pooling) is introduced in the Input input end and the Backbone backbone network, aiming to capture features at different scales through pooling operations of different sizes, helping the model aggregate information at different spatial scales to enhance the model's detection ability for multi-scale objects. The Backbone backbone network also includes multiple CBS convolution modules (Convolution + BatchNorm + SiLU) and C2f modules (CSP + 2 Feedforward Networks). The C2f module combines CSP (Cross Stage Partial connections) and two feedforward neural networks (Feedforward Networks) to further extract and process features. The Neck neck network is responsible for further processing and fusing the features from the backbone network, using the Feature Pyramid Network (FPN) to fuse information at multiple scales to enhance the model's detection ability for objects of different sizes. The Head detection head is responsible for the final object classification, localization, and confidence scoring. Object detection is achieved by directly predicting bounding boxes, class labels, and confidences.

[0022] To solve the problem of low accuracy in multi-scale erosion detection of FRP materials due to extreme environments, small erosion range sizes, and unclear erosion characteristics, the embodiments of the present disclosure improve the object detection model based on YOLOv8n. By introducing the SA-C2f attention mechanism module and the BiFPN-HMC feature fusion module into the feature extraction network of the YOLOv8n model, it is possible to handle multi-scale erosion detection application scenarios.

[0023] Figure 2 FIG. 4 shows an exemplary flowchart of an FRP strip erosion detection method 200 based on a multi-scale object detection model according to an embodiment of the present disclosure.

[0024] At Figure 2 block S202, the SA-C2f attention mechanism module and the BiFPN-HMC feature fusion module are introduced into the YOLOv8n network model to construct a multi-scale object detection model.

[0025] In some embodiments of the present disclosure, in order to improve the accuracy of multi-scale object detection, first, the backbone network C2f module of the YOLOv8n deep learning model is replaced with the SA-C2f module to add spatial and channel attention to the input feature map. The BiFPN-HMC feature fusion module is integrated into the neck network of the YOLOv8n network model to refine, weight, and fuse the feature maps extracted by the backbone network to obtain multi-scale feature maps.

[0026] Among them, the SA-C2f module combines the spatial attention mechanism and the channel attention mechanism, making the network more intelligent in feature extraction. The spatial attention mechanism helps the network automatically focus on important spatial regions in the image, strengthen the features of these regions, and at the same time suppress unimportant background parts. The channel attention mechanism helps the model focus on the most representative feature channels by assigning weights to each channel and suppressing irrelevant channels. In the BiFPN module, the HMC (Hybrid Multi-scale Cross Attention) module is integrated. This feature fusion method further optimizes the fusion method of different scale information by introducing the multi-scale cross-attention mechanism (CrossAttention). Specifically, HMC uses a multi-level cross-attention mechanism to weight and refine features of different scales to ensure that features of each scale can be properly represented in the final output.

[0027] In some embodiments of the present disclosure, the SA-C2f module includes: a spatial attention sub-module, a channel attention sub-module, and a feature recombination unit. The spatial attention sub-module includes a horizontal pooling layer, a vertical pooling layer, a splicing module, a horizontal convolution, a vertical convolution, an Fc linear transformation module and a sigmoid activation function, a horizontal sigmoid activation function, and a vertical sigmoid activation function.

[0028] Among them, the horizontal pooling layer is used to perform a maximum pooling operation on the input feature map in the horizontal direction to obtain a horizontal direction feature map. Similar to the horizontal pooling, the vertical pooling layer is used to perform a maximum pooling operation on the input feature map in the vertical direction to obtain a vertical direction feature map. The splicing module is used to splice the horizontal direction feature map and the vertical direction feature map to obtain a fused feature map. The horizontal convolution and the vertical convolution respectively perform convolution operations on the fused feature map to learn the attention weights in the horizontal and vertical directions, helping the model to focus on key information in different directions. The Fc linear transformation module and the sigmoid activation function are used to map the convolution output to the range [0,1] to be used as the final attention weight for application to the input feature map. The horizontal sigmoid activation function and the vertical sigmoid activation function are used to control the emphasis degree of the spatial features, and the role is to generate the final spatial attention map according to the attention in the horizontal and vertical directions.

[0029] The channel attention sub-module is used to perform weighted processing on the channel features through a multi-layer perceptron to obtain dynamically adjusted channels. The outputs of each channel are weighted through the multi-layer perceptron to generate the attention weights of the channels. Finally, the channel attention mechanism assigns an importance value to each channel, enabling the network to focus more on specific and key channel features and enhancing the discriminative ability of the model. The feature recombination unit is used to combine spatial and channel information through channel rearrangement and feature fusion operations. The channel rearrangement operation is performed to change the channel order so that the information between channels can be better combined, making the finally output features more comprehensive and containing more information.

[0030] Figure 3 is a schematic structural diagram of the SA-C2f attention mechanism module according to an embodiment of the present disclosure. Refer to Figure 3 As shown, for an input feature map X with a given dimension of C×H×W, where H and W represent the height and width of the feature map, and C represents the number of channels, the SA-C2f attention mechanism module can divide the feature map X along the channel dimension into g groups to form grouped feature maps , where , each subgroup has a channel number of C / g; each subgroup Obtain a sub - feature group through a Spatial Attention (SA) module and corresponding attention weight coefficients; the sub - feature group applies a random unit through two branches, spatial and channel, to simultaneously construct channel attention and spatial attention, and combines the attention information of these two dimensions to weight the input feature map. This weighting mechanism enables the model to better focus on important spatial regions and key channels, thereby improving the model's performance on specific tasks.

[0031] In some embodiments of the present disclosure, the BiFPN - HMC feature fusion module includes a bidirectional feature pyramid network, a multi - convolution module, a feature weighting mechanism, and a residual connection. The bidirectional feature pyramid network BiFPN is used to fuse feature maps of different scales through bidirectional feature flow in the up - and - down paths. This means that feature maps can not only be fused between different scales but also strengthen feature expressions at multiple levels from fine - grained to coarse - grained. The multi - convolution module uses depth - separable convolution and dilated convolution with variable dilation rates to obtain erosion feature details. Using depth - separable convolution to decompose the conventional convolution operation into depth convolution and point - wise convolution can reduce the computational amount, thus greatly improving the computational efficiency. Using dilated convolution, by adjusting the dilation rate of the convolution kernel to increase the receptive field, this helps capture more context information, which is particularly important for fine - grained features (such as erosion features). The feature weighting mechanism is used to assign dynamic weights to feature maps of different scales. By adaptively adjusting the weights of features at each scale, it can ensure that key features are fully utilized, thereby optimizing the information flow in the feature fusion process. The residual connection in the BiFPN module can effectively alleviate the problem of gradient vanishing, ensuring that information can be smoothly transmitted in the deep network, thereby improving the stability of training.

[0032] Figure 4 is a schematic diagram of the BiFPN - HMC feature fusion module structure according to an embodiment of the present disclosure. Refer to Figure 4 As shown, the feature maps extracted from the Backbone backbone network part are input into the BiFPN - HMC feature fusion module, and different - scale feature maps (C3, C4, C5, C6, C7) are extracted through up - sampling and down - sampling operations. This helps ensure that features of different scales can be fully fused and avoid feature loss caused by scale differences.

[0033] The multi-convolution module DDS uses depthwise separable convolutions (k = 1, d = 1, s = 1) and dilated convolutions with variable dilation rates (k = 3, d = 3, s = 1) to refine features of feature maps at different scales. Through a channel attention module (CA module (Channel Attention Module)) and a spatial attention module (SAM module (Spatial Attention Module)), channel weighting adjustment and spatial information optimization are respectively performed to enhance the expression of key features, optimize the spatial information of the FRP strip erosion image, highlight key regions, and suppress background noise, thereby improving the detection accuracy of erosion features. The feature maps after channel weighting adjustment and spatial information optimization are recombined. Using the channel invariance and spatial specificity of involution, convolution kernels are generated for each position. To reduce parameters, grouping is performed by channel, and the channel parameters within the group are shared. Finally, multi-scale feature maps (P3, P4, P5, P6, P7) are output.

[0034] Return Figure 2 As shown, subsequently, in block S204, an erosion detection image sample set of FRP strips is obtained, and the sample set is labeled with damage regions and damage categories.

[0035] The images in the sample set simulate the damage caused by sandstorm erosion to fiberglass-reinforced plastic (FRP) strips. FRP strip images simulating sandstorm erosion damage can be obtained at different shooting angles and lighting conditions. Then, the damage regions and damage categories are labeled in the FRP strip images. For example, professional labeling tools are used to annotate the samples to ensure that the eroded damage parts in each image can be accurately identified. Usually, these labeling tools allow users to draw boxes, contours, etc. on the images and label the damage regions. The labeling tools will label the eroded damage regions involved in the images, such as cracks, scratches, local corrosion, etc. Each damage region will be assigned a corresponding label indicating the damage type and location.

[0036] Subsequently, in block S206, the multi-scale object detection model is trained based on the sample set to obtain a trained multi-scale object detection model.

[0037] In some embodiments of the present disclosure, the labeled images are divided into a training set and a validation set. The training set is used to adjust the parameters of the multi-scale object detection model, and the validation set is used to evaluate the performance of the multi-scale object detection model. To train and verify the effectiveness of the model, the sample set is randomly divided in a ratio of 8:2. That is, 80% of the samples are used for the training set, and 20% of the samples are used for the validation set.

[0038] The training set is input into the multi-scale object detection model. By minimizing the BCEWithLogitsLoss loss function through backpropagation, the model parameters are updated to obtain a trained multi-scale object detection model. BCEWithLogitsLoss is a loss function for binary classification tasks, suitable for processing class predictions for each pixel or region in object detection. It combines the Sigmoid activation function and binary cross-entropy loss. The BCEWithLogitsLoss function is as follows: ; In the formula, is the number of samples, is the true label of the th sample, taking values of 0 or 1, is the prediction value of the model for the th sample without passing through the activation function, is the output after Sigmoid activation.

[0039] After the training is completed, the validation set can be used to evaluate the trained multi-scale object detection model to check its performance in the actual sand erosion damage detection task. The evaluation metrics usually include Accuracy, Recall, F1-score, mean Average Precision (mAP), etc., and the specific selection depends on the task requirements. The trained model is saved as a file, such as duoopti_yolo_model.pth, for real-time detection of sand erosion damage by loading the model in actual applications.

[0040] Finally, in box S208, the FRP strip image to be detected is obtained, and the trained multi-scale object detection model is used to perform erosion detection on the FRP strip image, and the positions and characteristic information of the erosion defects are output.

[0041] The FRP strip erosion detection based on the multi-scale object detection model provided by the embodiments of this application can be applied to the application environment as shown in Figure 5 shown. Figure 5It is an application environment diagram for FRP strip erosion detection according to an embodiment of the present disclosure. Among them, the image information of the terminal 102 is sent to the server 104 by means such as drone shooting. The data storage system can be set up separately, integrated on the server 104, placed in the cloud or on other servers. After receiving the FRP material image, for the FRP strip image, the server 104 uses the trained multi-scale object detection model to detect the erosion impact on the FRP material. In addition, in some embodiments, the method for FRP erosion damage detection using the trained multi-scale object detection model can also be implemented independently by the server 104. The server 104 can obtain the FRP strip image from the data storage system and perform erosion detection on the FRP strip image. Among them, the terminal 102 can be, but is not limited to, machinery or building structures such as ships, airplanes, bridges, and wind turbine blades.

[0042] Figure 6 It is a schematic diagram of an FRP strip according to an embodiment of the present disclosure. The FRP strip is a strip-shaped structure made of Fiber Reinforced Polymer (FRP for short). FRP is a composite material mainly composed of high-strength fibers (such as glass fibers, carbon fibers, or aramid fibers) and a polymer matrix (such as epoxy resin, polyester resin, etc.). In addition, it also includes reinforcing sheets and steel plates. This combination makes the FRP material have excellent properties such as high strength, light weight, corrosion resistance, and high temperature resistance.

[0043] The method for FRP strip damage erosion detection based on the multi-scale object detection model provided in this embodiment can be applied to the inspection scenarios of mechanical or civil engineering structures. The inspection scenarios of mechanical or building structures include an image acquisition link and an image recognition link. The method for FRP material erosion detection based on the DuoOpti-YOLO deep learning algorithm provided in this embodiment belongs to the image recognition link.

[0044] In some embodiments of the present disclosure, first, obtain the FRP strip images to be detected. These images can come from various types of sensors or devices, such as high-definition cameras, industrial detection systems, or drones. At this stage, ensure that the image quality is high and can clearly display the surface features of the FRP strip for subsequent erosion detection. Before processing, the image is usually subjected to certain preprocessing operations, such as denoising, image enhancement, contrast adjustment, etc., to improve the clarity and detail information of the image and ensure that the erosion defects can be more prominent.

[0045] Then, load the trained multi-scale object detection model. This model has been trained based on the annotated image sample set and learned different types of erosion defects (such as cracks, corrosion, delamination, etc.) and their features at different scales. The model includes the SA-C2f attention mechanism module and the BiFPN-HMC feature fusion module, which enables it to effectively extract feature information at different scales in the image to be detected and focus on important regions.

[0046] Input the FRP strip image to be detected into the trained model. After the image is processed, the model will automatically perform feature extraction, object detection, and analyze the potential damage features in different regions. The position of the erosion area (the upper left and lower right coordinates) is calibrated in the form of a bounding box. These bounding boxes surround the erosion defects and indicate the regions where they are located. In addition to the position, the model will also identify different types of erosion defects and provide corresponding feature information. This feature information may include: Damage type: such as corrosion, crack, delamination, etc. Damage degree: such as mild corrosion, moderate corrosion, severe corrosion, etc., usually judged by the size and shape of the defect. Area or size of the defect: the area of the detection region, usually proportional to the damage degree. Morphology of the damage: such as the length and width of the crack, providing a detailed description of the damage morphology. This information can be presented in a visual way, for example, the position of the erosion defect is marked with a bounding box, and the damage type and severity are marked on the image.

[0047] To verify the effectiveness and superiority of the multi-scale object detection model provided by the embodiments of the present disclosure, the improved model is compared with the detection effect of the current YOLOv8n model. After verification, the algorithm proposed in this application is superior to the YOLOv8n model in terms of detection accuracy. Its detection reaches an average accuracy of 95.2%, an improvement of 3.3%, a recall improvement of 3.7%, an F1 score improvement of 3.5%, and an mAP improvement of 3.1%.

[0048] Figure 7 is a schematic block diagram of an FRP strip erosion detection device 700 based on a multi-scale object detection model according to an embodiment of the present disclosure. As Figure 7 shown, the device 700 may include a processor 710 and a memory 720 storing a computer program. When the computer program is executed by the processor 710, the device 700 can be made to execute as Figure 2Steps of the FRP strip erosion detection method 200 based on the multi-scale object detection model as shown. In one example, the device 700 may be a computer device or a cloud computing node. The device 700 may: introduce the SA-C2f attention mechanism module and the BiFPN-HMC feature fusion module into the YOLOv8n network model to construct a multi-scale object detection model; obtain a set of erosion detection image samples of the FRP strip, and the sample set is labeled with damage areas and damage categories; train the multi-scale object detection model based on the sample set to obtain a trained multi-scale object detection model; and obtain the FRP strip image to be detected, and use the trained multi-scale object detection model to perform erosion detection on the FRP strip image, and output the position and its feature information of the erosion defect.

[0049] In an embodiment of the present disclosure, the processor 710 may be, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a processor based on a multi-core processor architecture, etc. The memory 720 may be any type of memory implemented using data storage technology, including but not limited to random access memory, read-only memory, semiconductor-based memory, flash memory, disk memory, etc.

[0050] In addition, in an embodiment of the present disclosure, the device 700 may also include an input device 730, such as a keyboard, a mouse, etc., for inputting a set of erosion detection image samples of the FRP strip and the FRP strip image to be detected. Additionally, the device 700 may further include an output device 740, such as a display, etc., for outputting the position and its feature information of the erosion defect.

[0051] In other embodiments of the present disclosure, there is also provided a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, can implement the steps of the FRP strip erosion detection method 200 based on the multi-scale object detection model as shown in Figure 2 Steps.

[0052] In summary, according to the FRP strip erosion detection method based on the multi-scale object detection model in the embodiments of the present disclosure, by introducing the attention mechanism, the model's ability to capture local features is effectively enhanced. The SA-C2f module effectively combines the spatial attention mechanism and the channel attention mechanism, thereby more comprehensively capturing the relationship between features and enhancing the channel and spatial information of the feature map. Based on the BiFPN-HMC feature fusion module, effective information transfer and fusion can be performed between features of different scales, and large-scale and small-scale erosion damages can be detected more accurately. Compared with the existing model detection methods, it has higher accuracy, robustness, multi-scale detection ability, and can also provide accurate position and damage classification information, significantly improving the efficiency and accuracy of FRP strip erosion detection.

[0053] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus and methods according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0054] Unless the context clearly indicates otherwise, the singular forms of the words used in this specification and the appended claims include the plural, and vice versa. Thus, when reference is made to the singular, the corresponding plural is generally included. Similarly, the terms "comprising" and "including" shall be construed as inclusive rather than exclusive. Likewise, the term "or" shall be construed as inclusive, unless such construction is clearly prohibited in this application. Where the term "example" is used in this application, particularly when it is followed by a list of terms, the "example" is merely illustrative and explanatory, and should not be considered exclusive or exhaustive.

[0055] Further aspects and scopes of adaptability become apparent from the description provided herein. It should be understood that the various aspects of the present application may be implemented alone or in combination with one or more other aspects. It should also be understood that the description herein and the specific embodiments are intended for illustrative purposes only and are not intended to limit the scope of the present application.

[0056] The above has described in detail several embodiments of the present disclosure. Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present disclosure without departing from the spirit and scope of the present disclosure. The protection scope of the present disclosure is defined by the appended claims.

Claims

1. A method for detecting FRP strip erosion based on a multi-scale object detection model, characterized in that Including: Introduce the SA-C2f attention mechanism module and the BiFPN-HMC feature fusion module into the YOLOv8n network model to construct a multi-scale object detection model; Obtain a set of erosion detection image samples of FRP strips, where the sample set is labeled with damage areas and damage categories; Train the multi-scale object detection model based on the sample set to obtain a trained multi-scale object detection model; and Obtain the FRP strip image to be detected, and use the trained multi-scale object detection model to perform erosion detection on the FRP strip image, and output the location and its feature information of the erosion defect.

2. The FRP strip erosion detection method based on a multi-scale object detection model according to claim 1, wherein The introducing the SA-C2f attention mechanism module and the BiFPN-HMC feature fusion module into the YOLOv8n network model to construct a multi-scale object detection model includes: Introduce the SA-C2f attention mechanism module into the backbone network of the YOLOv8n network model to add spatial and channel attention to the input feature map; Integrate the BiFPN-HMC feature fusion module into the neck network of the YOLOv8n network model to refine, weight, and fuse the feature maps extracted by the backbone network to obtain multi-scale feature maps.

3. The FRP strip erosion detection method based on the multi-scale object detection model according to claim 1 or 2, characterized in that, The SA-C2f attention mechanism module includes a spatial attention sub-module, a channel attention sub-module, and a feature recombination unit; The spatial attention sub-module includes a horizontal pooling layer, a vertical pooling layer, a splicing module, a horizontal convolution, a vertical convolution, an Fc linear transformation module and a sigmoid activation function, a horizontal Sigmoid activation function and a vertical Sigmoid activation function. Among them, the horizontal pooling layer is used to perform a maximum pooling operation on the input feature map in the horizontal direction to obtain a horizontal feature map; the vertical pooling layer is used to perform a maximum pooling operation on the input feature map in the vertical direction to obtain a vertical feature map; the splicing module is used to splice the horizontal feature map and the vertical feature map to obtain a fused feature map; the horizontal convolution and the vertical convolution respectively perform convolution operations on the fused feature map to generate attention weights; the Fc linear transformation module and the sigmoid activation function are used to map the convolution output to the range of [0,1] to be used as attention weights for the input feature map; The channel attention sub-module is used to weight the channel features through a multi-layer perceptron to obtain dynamically adjusted channels; The feature recombination unit is used to combine spatial and channel information through channel rearrangement and feature fusion operations.

4. The FRP strip erosion detection method based on a multi-scale object detection model according to claim 1, characterized in that The SA-C2f attention mechanism module is used to divide the input feature map with dimensions of C×H×W, where H and W represent the height and width of the feature map, and C represents the number of channels, into g groups along the channel dimension to form grouped feature maps. , where ; each subgroup obtains a sub-feature group through a spatial attention module and corresponding weight coefficients; the sub-feature group applies a random unit through two branches of space and channel to simultaneously construct channel attention and spatial attention, which are used as attention weights for the input feature map.

5. The FRP strip erosion detection method based on the multi-scale object detection model according to claim 1, characterized in that The BiFPN-HMC feature fusion module includes a bidirectional feature pyramid network, a multi-convolution module, a feature weighting mechanism, and a residual connection; The bidirectional feature pyramid network is used to fuse feature maps of different scales through bidirectional feature flow in the up and down paths; the multi-convolution module uses depthwise separable convolution and variable dilation rate convolution to obtain erosion feature details; the feature weighting mechanism is used to assign dynamic weights to feature maps of different scales.

6. The FRP strip erosion detection method based on the multi-scale object detection model according to claim 5, wherein The BiFPN-HMC feature fusion module is used to obtain feature maps from the backbone network; extract feature maps of different scales through upsampling and downsampling operations; adopt depthwise separable convolution and dilated convolution with variable dilation rates to refine the feature maps of different scales; respectively perform channel weighting adjustment and spatial information optimization through the channel attention module and the spatial attention module, and recombine the feature maps after channel weighting adjustment and spatial information optimization to output multi-scale feature maps.

7. The FRP strip erosion detection method based on a multi-scale object detection model according to claim 1, characterized in that, The method for obtaining the erosion detection image sample set of the FRP strip, and the sample set is labeled with damage areas and damage categories, including: Obtain FRP strip images simulating wind-sand erosion damage under different shooting angles and lighting conditions; Label the damage areas and damage categories in the FRP strip images.

8. The FRP strip erosion detection method based on the multi-scale object detection model according to claim 1, characterized in that The method for training the multi-scale object detection model based on the sample set to obtain a trained multi-scale object detection model includes: Divide the labeled images into a training set and a validation set. The training set is used to adjust the parameters of the multi-scale object detection model, and the validation set is used to evaluate the performance of the multi-scale object detection model; Input the training set into the multi-scale object detection model, update the model parameters by minimizing the BCEWithLogitsLoss loss function through backpropagation, and use the validation set to evaluate the performance of the model to obtain a trained multi-scale object detection model. The BCEWithLogitsLoss loss function is: ； Wherein, is the number of samples, is the true label of the th sample, taking values of 0 or 1, is the predicted value of the model for the th sample without passing through the activation function.

9. An FRP strip erosion detection device based on a multi-scale object detection model, characterized in that, Including: At least one processor; And At least one memory storing a computer program; Wherein, when the computer program is executed by the at least one processor, the device is caused to execute the steps of the FRP strip erosion detection method based on the multi-scale object detection model according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the FRP strip erosion detection method based on the multi-scale object detection model according to any one of claims 1 to 8.

Images