A method for detecting fluorescent cracks of pull rivets based on an improved YOLOV5 model
By combining the Soft-NMS and ECSAM attention modules, the improved YOLOV5 model solves the problem of low accuracy in complex environments for rivet fluorescence crack detection, achieving higher detection accuracy and recall.
Patent Information
- Application Number
- CN202310623442.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2043-05-30
AI Technical Summary
Existing traditional methods are difficult to effectively detect fluorescent cracks in rivets, especially in complex environments where the dense cracks, irregular shapes, and multi-scale cracks within the same category lead to low detection accuracy.
By combining Soft-NMS and YOLOv5 models, and employing the ECSAM attention module and segmentation prediction layer, we enhance bounding box preservation and multi-scale detection capabilities while reducing background interference through improved annotation methods and feature extraction processes.
It improves the accuracy and recall of fluorescent crack detection, especially in the detection of dense and irregular crack areas, thus enhancing detection precision.
Smart Images
Figure CN116664511B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of computer vision, and particularly relates to a fluorescent crack detection method based on an improved YOLOV5 model. BACKGROUND
[0002] With the continuous development of railway trains in the direction of high speed and heavy load, rivets and pins used to connect parts are widely used in railway vehicles. Rivets have high fastening force, never loosen and high shear performance. It is particularly advantageous to use rivets to solve the problems caused by vibration and swing. During the casting process, some cracks at the head of the rivet reduce the performance of the product due to technical reasons. The load and stress applied on the rivet can cause these small cracks to expand, resulting in failure. Currently, the industry finds defective rivets through manual observation, which not only requires a high degree of concentration, but also is prone to missed detection. An intelligent defect detection system improves standardization and work efficiency. It provides an intelligent detection solution and broad application prospects.
[0003] The most widely used traditional machine learning methods in the field of crack detection are mainly support vector machines, artificial neural networks, decision trees and K-means clustering algorithms. These traditional methods require complex image preprocessing. In addition, the sample quality has a great influence on the detection results, making it difficult to meet the needs of industrial product detection. Currently, convolutional neural networks (CNN) can learn useful features extracted from large datasets. Crack detection based on deep learning has been widely applied in many fields. In manufacturing inspection, there are few studies on rivet fluorescent crack detection. Compared with natural scenes, the images of rivet fluorescent cracks have the characteristics of dense cracks, irregular shapes and multi-scale within crack classes. In addition, uneven spraying of fluorescent magnetic suspension can cause fluorescence to gather, which will interfere with crack recognition and make defect recognition difficult.
[0004] Therefore, how to avoid the problems of dense cracks, irregular shapes and multi-scale within crack classes when detecting fluorescent cracks in complex environments and improve the accuracy of fluorescent crack detection is a problem that technicians in the field need to solve. SUMMARY
[0005] In order to solve the problem of low precision caused by dense cracks, multi-scale distribution and irregular shape in fluorescent crack detection, the present application proposes to combine Soft-NMS with Yolov5, as well as a new dense crack annotation method to retain more bounding boxes in dense areas. ECSAM is proposed to adaptively fuse fluorescent crack feature maps of different levels. In addition, Yolov5 adds a segmentation head to guide the model to thoroughly learn the semantic and spatial features of cracks, reducing the interference of fluorescence gathering in the background. This method can effectively detect fluorescent cracks in complex backgrounds and improve the accuracy of fluorescent crack detection.
[0006] The object of the present application is achieved by the technical solutions as follows:
[0007] A method for detecting rivet fluorescent cracks based on an improved YOLOV5 model, comprising the following steps:
[0008] S1, acquiring a fluorescent crack image dataset, labeling all cracks in the fluorescent crack image dataset using a label box, and giving a crack category to obtain a labeled image;
[0009] S2, inputting the labeled image into the improved YOLOV5 model for training to obtain a trained improved YOLOV5 model; the improved YOLOV5 model comprises: embedding an ECSAM attention mechanism for multi-scale fluorescent crack detection in a feature extraction part; adding a segmentation prediction layer; using Soft-NMS for post-processing of a prediction box; the ECSAM attention module is an attention module formed by connecting an ECA attention module and a SA spatial attention module in series;
[0010] S3, inputting an image to be detected into the trained improved YOLOV5 model to obtain a detection result of the fluorescent crack.
[0011] Further, when labeling all cracks in the fluorescent crack image dataset, the following labeling method is used:
[0012] When two cracks are close in space, the overlapping area of two label boxes exceeds the label box overlapping area threshold, the label box is changed so that it only contains the respective significant feature parts of the two cracks, highlighting the features of the non-overlapping part cracks, and at the same time reducing the overlapping area of the two label boxes or making them not overlap.
[0013] Further, in step S2, the sub-step of using Soft-NMS for post-processing of the prediction box is as follows:
[0014] Soft-NMS retains the bounding boxes adjacent to the selected bounding box M by reducing the confidence of the bounding boxes; when the IOU between these adjacent bounding boxes and the selected bounding box M is greater than a specified threshold N t , the confidence of the adjacent bounding boxes is multiplied by a Gaussian function, thereby reducing the confidence of these adjacent bounding boxes:
[0015]
[0016] In the formula, M represents the selected bounding box with the highest confidence, N t is the specified threshold in NMS, b i represents the remaining bounding boxes, and s iclassification confidence of the corresponding bounding box; iou(·) represents a formula for calculating the intersection over union between two bounding boxes, and σ represents the standard deviation in the Gaussian function.
[0017] Further, in the S2, the processing process of the ECSAM attention module on the input feature map includes the following sub-steps: (1) the input feature map with a size of HxWxC is subjected to 1x1 convolution;
[0018] (2) batch normalization is performed on the feature map output after convolution;
[0019] (3) a hardswish activation function is used to activate the normalized feature map to obtain a feature map χ;
[0020] (4) the ECA attention module compresses the feature map χ through spatial information to obtain a 1x1xC vector;
[0021] (5) one-dimensional convolution with adaptive convolution kernel size k is used to realize information exchange between each channel and its k nearest neighbors;
[0022] (6) an activation operation is used to obtain interactive channel weights;
[0023] (7) the feature map χ and the interactive channel weights are spliced to obtain an output feature map
[0024] (8) the output feature map is input into the spatial attention module, and the spatial attention module obtains the spatial weight of the output feature map by performing average and maximum pooling along the channel axis;
[0025] (9) the output feature map and its spatial weight are spliced to obtain the feature map output by the ECSAM attention module, and the feature map is applied to the convolution feature extraction in the later stage of the network.
[0026] Further, the S2 includes the following sub-steps:
[0027] (2.1) the image with completed labeling is subjected to a 6x6 convolution layer to obtain a feature map Feature_C0;
[0028] (2.2) the feature map Feature_C0 is input into the ECSAM attention mechanism module, and then passes through the C3 module to enhance the effective features of the crack and output a feature map Feature_C1;
[0029] (2.3) the output feature map Feature_C1 is input again to the ECSAM attention mechanism module, and then passes through a C3 module to obtain a feature map Feature_C2; the feature map Feature_C2 is input again to the ECSAM attention mechanism module, and then passes through a C3 module to obtain a feature map Feature_C3;
[0030] (2.4) the feature map Feature_C3 passes through the ECSAM attention mechanism module and the C3 module, and then the feature map undergoes spatial pyramid pooling (SPPF) to obtain a feature map Feature_C4 as an input of a multi-scale feature fusion module.
[0031] (2.5) the feature map Feature_C4 is obtained by 3x3 convolution and up-sampling operation to obtain a feature map Feature_Up1;
[0032] (2.6) the feature map Feature_Up1 is fused with the feature map Feature_C3, and then passes through a C3 module and 3x3 convolution to obtain a fused feature map Feature_Fuse1; the feature map Feature_Fuse1 is obtained by up-sampling operation to obtain a feature map Feature_Up2;
[0033] (2.7) the feature map Feature_Up2 is fused with the feature map Feature_C2 to obtain a feature map Feature_Fuse2, and a feature map Feature_C5 is obtained by a C3 module, and a feature map Feature_C6 is obtained by 3x3 convolution;
[0034] (2.8) the feature map Feature_C6 is fused with the feature map Feature_Fuse1 to obtain a feature map Feature_Fuse3, a feature map Feature_C7 is obtained by a C3 module, and a feature map Feature_C8 is obtained by 3x3 convolution,
[0035] (2.9) the feature map Feature_C8 is fused with the feature map Feature_C4 obtained by 3x3 convolution to obtain a feature map Feature_Fuse4, and a detection layer Feature_D0 is obtained by a C3 module and 1x1 convolution;
[0036] (2.10) the feature map Feature_C7 is obtained by 1x1 convolution to obtain a detection layer Feature_D1; the feature map Feature_C5 is obtained by 1x1 convolution to obtain a detection layer Feature_D2;
[0037] (2.11) The Feature_Fuse2 is sequentially subjected to a 3*3 convolution, an upsampling operation, a C3 module, a 3*3 convolution, an upsampling operation, a 3*3 convolution, a C3 module, an upsampling operation and a 1*1 convolution to obtain a segmentation layer Feature_S0.
[0038] The beneficial effects of the present application are as follows:
[0039] 1. In view of the fact that the crack of the pull rivet exists in a densely distributed area, the present application combines Yolov5 with Soft-NMS, and retains more bounding boxes in the densely distributed crack area, which is conducive to the detection result; on this basis, a labeling method for densely distributed crack areas is proposed, so that the bounding box can be accurately regressed to the true label, thereby improving the recall rate of the improved Yolov5 model.
[0040] 2. In view of the fact that the crack of the pull rivet exists in a densely distributed area, the present application combines Yolov5 with Soft-NMS, and retains more bounding boxes in the densely distributed crack area, which is conducive to the detection result; on this basis, a labeling method for densely distributed crack areas is proposed, so that the bounding box can be accurately regressed to the true label, thereby improving the recall rate of the improved Yolov5 model.
[0041] 3. In view of the fact that the shape of the pull rivet crack is irregular, the present application adds a segmentation layer in Yolov5 to guide the network to supplement the learning of crack semantic features. The segmentation layer reduces the interference in the background and improves the accuracy of crack detection. Through comparative analysis, it is found that instance segmentation is more suitable for the segmentation of fluorescent cracks than semantic segmentation. BRIEF DESCRIPTION OF DRAWINGS
[0042] Figure 1 The flowchart of the fluorescent crack detection method based on the improved YOLOV5 model provided by the embodiment of the present application.
[0043] Figure 2 The improved schematic diagram of the densely distributed crack labeling method provided by the embodiment of the present application, wherein (a) is the existing method, and (b) is the densely distributed crack labeling method of the embodiment of the present application.
[0044] Figure 3 The structural schematic diagram of the improved YOLOV5 convolutional neural network provided by the embodiment of the present application.
[0045] Figure 4 The model schematic diagram of the ECSAM attention module provided by the embodiment of the present application.
[0046] Figure 5 The example diagram of the comparison of improving the detection of densely distributed crack areas by using Soft-NMS provided by the embodiment of the present application; wherein (a) is an example diagram of detection by using Yolov5; and (b) is an example diagram of detection by using Yolov5+Soft-NMS.
[0047] Figure 6 The detection contrast example using the improved labeling method provided by the embodiment of the present application is provided, wherein figure (a) is an example of detection using Yolov5; figure (b) is an example of detection using Yolov5+ improved labeling method.
[0048] Figure 7 The ECSAM attention module detection contrast example provided by the embodiment of the present application is provided, wherein figure (a) is an example of detection using Yolov5; figure (b) is an example of detection using Yolov5+ECSAM.
[0049] Figure 8 The fluorescent crack instance segmentation detection contrast example provided by the embodiment of the present application is provided, wherein figure (a) is an example of detection using Yolov5; figure (b) is an example of detection using instance segmentation. DETAILED DESCRIPTION
[0050] The purpose and effect of the present application will become more apparent from the following detailed description of the drawings and preferred embodiments, and it should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.
[0051] The fluorescent crack detection method based on the improved YOLOV5 model of the present application first introduces Soft-NMS, which retains more boundary boxes in the dense crack area, which is beneficial to the detection result. And a new dense crack region labeling method is proposed, which can accurately regress the boundary box to the true label. The above two methods effectively improve the recall rate; secondly, the attention mechanism ECSAM of multi-scale fluorescent crack detection is proposed to enhance the channel and spatial information of the feature layer and improve the model's ability to predict multi-scale cracks; finally, the segmentation prediction layer is added in Yolov5 to guide the network to supplement the learning of crack semantic features, reduce the interference of fluorescent in the background, and improve the accuracy of crack detection. The above method effectively improves the problems of dense cracks, multi-scale defects within the class and irregular crack shape in the fluorescent crack image, and improves the accuracy of fluorescent crack detection.
[0052] The implementation steps of the present application are as shown in Figure 1 , including the following steps:
[0053] S1, obtain the fluorescent crack image data set, label all the cracks in the fluorescent crack image data set using the labeling box, and give the crack category to obtain the labeled image;
[0054] S2, input the labeled image into the improved YOLOV5 model for training to obtain the trained improved YOLOV5 model; the improved YOLOV5 model comprises: embedding an ECSAM attention module of multi-scale fluorescent crack detection in a feature extraction part; adding a segmentation prediction layer; using Soft-NMS for post-processing of a prediction frame; the ECSAM attention module is an attention module formed by connecting an ECA attention module (Efficient Channel Attention) and an SA spatial attention module in series. The improved YOLOV5 target detection model is optimized when the boundary frame is regressed by introducing Soft-NMS (non-maximum suppression); the channel and spatial features with key information are enhanced by introducing the proposed efficient channel and spatial attention mechanism (ECSAM); the model is guided to fully learn the semantic and spatial features of the fluorescent crack by adding the segmentation prediction layer.
[0055] S3, input the image to be detected into the trained improved YOLOV5 model to obtain the detection result of the fluorescent crack.
[0056] In the above step S1, according to the characteristics of the dense fluorescent crack, the data is labeled using the dense fluorescent crack region labeling method. When two cracks are close in space, the conventional labeling method will make the overlapping area of two annotation boxes too large, or even one box completely contains another box. The new labeling method emphasizes the features of the non-overlapping part of the dense crack region, and reduces or eliminates the overlapping area between two different labeled boxes. In this embodiment, the dense fluorescent crack region labeling method is as follows: when the conventional method is used to label two dense cracks and the height of the labeled box is overlapped, the most significant feature part of each crack is highlighted, the non-overlapping crack feature is highlighted, and the overlapping area of the two labeled boxes is reduced or eliminated. By improving the labeling method, the distance between adjacent boxes is increased, while the most significant features of the cracks are retained. Yolov5 uses a grid corresponding to the center point of the anchor, and the offset of the left upper corner coordinate of the grid is calculated for regression. The new labeling method assigns dense cracks to different grids for prediction and regression, as shown in Figure 2 , which realizes more accurate prediction of cracks in dense regions.
[0057] b x = 2σ(t x 0-0.5+C x
[0058] b y = 2σ(t y )-0.5+C y
[0059] b w = Pw (2σ(t h )) 2
[0060] b h =P h (2σ(t w )) 2
[0061] where t x t y t w t h is the predicted offset, P w is the width of the prior box, b h is the height of the prior box C x , C y is the distance between the center of the annotated box and the top-left corner of the grid.
[0062] In the above step S2, the data-labeled image is input into the improved YOLOV5 model for training to obtain the trained improved YOLOV5 model; in the embodiment of the application, the improved YOLOV5 model comprises: 1. using Soft-NMS for post-processing of the predicted box; 2. embedding the ECSAM attention module of multi-scale fluorescent crack detection in the feature extraction part; 3. increasing the segmentation prediction layer.
[0063] ①Soft-NMS post-processing part of prediction
[0064] NMS is a component of object detection, which sorts the bounding boxes according to their confidence scores. The bounding box with the highest confidence score will be selected. The remaining bounding boxes that intersect with the selected box and exceed a specified threshold N t will be discarded. In NMS, removing bounding boxes near the selected box will lead to missed detection in dense crack regions. In the formula, M represents the selected bounding box with the highest confidence score, N t is the threshold specified in NMS, b i represents the remaining bounding boxes, s i represents the classification confidence of the corresponding bounding box. iou(·) represents the formula for calculating the intersection on the union of two bounding boxes. σ represents the standard deviation in the Gaussian function.
[0065]
[0066] In order to alleviate the missed detection problem caused by directly discarding bounding boxes with low confidence in NMS, Soft-NMS is used for post-processing of the predicted box. Soft-NMS retains the bounding boxes adjacent to the selected bounding box M by reducing the confidence of the bounding boxes. When the IOU between the bounding box and the selected box is greater than a specified threshold N tAt this time, the confidence of the adjacent bounding box will be multiplied by the Gaussian function. The confidence of these adjacent bounding boxes will decay instead of being directly set to 0.
[0067]
[0068] Yolov5 is a single-stage algorithm. Each grid in the prediction layer generates nine anchor boxes. For crack-dense areas, when several cracks are clustered in one grid, it is easy to cause missed detection and low recall rate. In addition, if only one bounding box is retained in the dense crack area according to the confidence, and the bounding box adjacent to M is discarded, the recall rate will be reduced. In order to solve the above problems, Soft-NMS is used instead of NMS in Yolov5, which is conducive to improving the detection performance of dense crack areas.
[0069] ②ECSAM attention mechanism part
[0070] The main purpose of feature extraction is to learn the mapping relationship between high-resolution images and low-resolution images using a convolutional neural network. In the embodiment of the application, in the feature extraction module, it is mainly composed of a convolutional layer (Conv), a C3 module, an ECSAM module and a feature pyramid module (SPPF). As shown in Figure 4 , the ECSAM is composed of ECA and spatial attention, which can effectively highlight the effective feature descriptors of the information area of the fluorescent crack image. As shown in Figure 3 , the specific process of inputting the labeled image into the improved YOLOV5 model for training is as follows:
[0071] (1) After the labeled image is completed, the feature map Feature_C0 is obtained after passing through the 6x6 convolutional layer;
[0072] (2) The feature map Feature_C0 is input into the ECSAM attention module, and then passes through the C3 module. The C3 module has three convolution modules in addition to the bottleneck, which reduces the parameter amount to enhance the effective features of the cracks, and outputs the feature map Feature_C1;
[0073] (3) The output feature map Feature_C1 is input into the ECSAM attention mechanism module again, and then passes through the C3 module to obtain the feature map Feature_C2; the feature map Feature_C2 is input into the ECSAM attention mechanism module again, and then passes through the C3 module to obtain the feature map Feature_C3;
[0074] (4) Feature_C3 is input into the ECSAM attention mechanism module and the C3 module, and then the feature map is subjected to spatial pyramid pooling (SPFF) to obtain a feature map Feature_C4 as an input of a multi-scale feature fusion module. The SPPF reduces the amount of calculation under the condition of ensuring the accuracy to improve the speed, and uses three 5x5 maximum pooling operations.
[0075] (5) The feature map Feature_C4 obtained by the feature extraction part is input into the multi-scale feature fusion module, and the feature map Feature_C4 is subjected to a 3x3 convolution and an upsampling operation to obtain a feature map Feature_Up1;
[0076] (6) The feature map Feature_Up1 is fused with the feature map Feature_C3, and then the C3 module and the 3x3 convolution are performed to obtain a fused feature map Feature_Fuse1. The feature map Feature_Fuse1 is subjected to an upsampling operation to obtain a feature map Feature_Up2;
[0077] (7) The feature map Feature_Up2 is fused with the feature map Feature_C2 to obtain a feature map Feature_Fuse2. The C3 module is used to obtain a feature map Feature_C5, and the 3x3 convolution is used to obtain a feature map Feature_C6;
[0078] (8) The feature map Feature_C6 is fused with the feature map Feature_Fuse1 to obtain a feature map Feature_Fuse3. The C3 module is used to obtain a feature map Feature_C7, and the 3x3 convolution is used to obtain a feature map Feature_C8;
[0079] (9) The feature map Feature_C8 is fused with the feature map Feature_C4 obtained by the 3x3 convolution to obtain a feature map Feature_Fuse4. The C3 module and the 1x1 convolution are used to obtain a detection layer Feature_D0, and the feature size of Feature_D0 is 1 / 4 of the original image. Feature_D0 is used as a prediction feature layer to predict the to-be-detected crack;
[0080] (10) The feature map Feature_C7 is subjected to the 1x1 convolution to obtain a detection layer Feature_D1, and the feature size of Feature_D1 is 1 / 8 of the original image. Feature_D1 is used as a prediction feature layer to predict the to-be-detected crack. The feature map Feature_C5 is subjected to the 1x1 convolution to obtain a detection layer Feature_D2, and the feature size of Feature_D2 is 1 / 16 of the original image. Feature_D2 is used as a prediction feature layer to predict the to-be-detected crack;
[0081] (11) Feature_Fuse2 is sequentially subjected to 3x3 convolution, upsampling operation, C3 module, 3x3 convolution, upsampling operation, 3x3 convolution, C3 module, upsampling operation and 1x1 convolution to obtain a segmentation layer Feature_S0, which has a feature size of 1 / 4 of the original image, as a prediction feature layer for semantic segmentation of the detected cracks. A total of three target detection feature layers and one semantic segmentation feature layer are used to detect the detected cracks. Through the combination of segmentation and detection, the network can more fully learn the irregular features of the cracks, and the detection accuracy can be improved. When the training batch size is set to 4, the baseline Yolov5 has the best result. The loss function gradually converges within 100 training rounds. In the following experiments, these two parameters are set to this setting by default.
[0082] Yolov5 is used as the baseline. In order to evaluate the effectiveness of each improved method, an ablation experiment is designed for verification. As shown in the following table, the performance of different improved methods is compared.
[0083] Compared with the baseline, the new annotation method and Soft-NMS improve the model's ability to detect dense crack regions and increase the recall rate. In addition, ECSAM enhances the weight of key channels and spatial information in each feature map. Compared with CBAM and BiFPN, ECSAM outperforms the combination of CBAM attention mechanism and BiFPN weighted feature fusion, verifying the effectiveness of ECSAM. A segmentation prediction layer is added to Yolov5, which fully utilizes the semantic features of the cracks. The precision is improved to 90.1%, higher than that of Yolov5. The best overall performance is achieved by the group that integrates all the improved methods. The recall rate and mAP are 86.2% and 89.7% respectively, and the performance is better than the baseline.
[0084] Table 1 Ablation experiment based on improved Yolov5 algorithm
[0085]
[0086] In addition, the results of other models are also compared, as shown in Table 2. In order to fairly compare the performance of different models, the same training conditions are set for the model parameters. Table 2 shows that the improved model has better performance than the Faster RCNN and YOLOX algorithms. Figures 5 to 8
[0087]
[0088]
[0089]
[0090] As shown in Table 3, the embodiment also compares the mIoU of semantic segmentation and instance segmentation. Compared with deeplabv3+, Yolov5 with segmentation layer and Mask-RCNN significantly improve the mIoU of fluorescent crack segmentation, which increases by 12% and 8% respectively. The experiment proves the advantage and effectiveness of instance segmentation on the fluorescent crack dataset.
[0091] Table 3 Comparison of crack segmentation performance
[0092]
[0093] Those skilled in the art can understand that the above description is only preferred examples of the application and is not used to limit the application, although the application is described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions recorded in the foregoing examples or make equivalent replacement for part of the technical features. Any modification, equivalent replacement, etc. within the spirit and principles of the application shall be included in the protection scope of the application.
Claims
1. A method for detecting fluorescent cracks of a pull rivet based on an improved YOLOV5 model, characterized in that, The method comprises the following steps: S1, acquiring a fluorescent crack image data set, labeling all cracks in the fluorescent crack image data set with a label box, and giving a crack category to obtain a labeled image; S2, inputting the labeled image into an improved YOLOV5 model for training to obtain a trained improved YOLOV5 model; the improved YOLOV5 model comprises: embedding an ECSAM attention mechanism for multi-scale fluorescent crack detection in a feature extraction part; adding a segmentation prediction layer; using Soft-NMS for post-processing of a prediction box; the ECSAM attention module is an attention module formed by connecting an ECA attention module and an SA spatial attention module in series; S3, inputting a to-be-detected image into the trained improved YOLOV5 model to obtain a detection result of the fluorescent crack; In S2, the processing process of the ECSAM attention module on the input feature map comprises the following sub-steps: (1) the input feature map with a size of HxWxC is subjected to 1x1 convolution; (2) batch normalization is performed on the output feature map after convolution; (3) using a hardswish activation function to activate the normalized feature map to obtain a feature map ; (4) The ECA attention module obtains a feature map obtained by spatial information compression; (5) one-dimensional convolution with an adaptive convolution kernel size k is used to realize information exchange between each channel and its k nearest neighbors; (6) interactive channel weights are obtained after an activation operation; (7) concatenating the feature map and the interactive channel weight to obtain an output feature map ; (8) the output feature map is input into the spatial attention module, and the spatial attention module obtains the spatial weight of the output feature map by performing average and maximum pooling along the channel axis. (9) the output feature map and its spatial weight are spliced to obtain the feature map output by the ECSAM attention module, and applied to the convolution feature extraction in the later stage of the network.
2. The method for detecting the fluorescent crack of the rivet based on the improved YOLOV5 model according to claim 1, characterized in that, When two cracks are close in space, if the overlapping area of two label boxes exceeds a label box overlapping area threshold, the label boxes are changed so as to contain only the parts with significant features of the two cracks, highlight the features of the non-overlapping cracks, and reduce the overlapping area of the two label boxes or make them not overlap. In step S2, the sub-steps of using Soft-NMS for post-processing of a prediction box are as follows:
3. The method for detecting the fluorescent crack of the rivet based on the improved YOLOV5 model according to claim 1, characterized in that, In S2, the following sub-steps are included: Soft-NMS retains bounding boxes adjacent to the selected bounding box M by reducing their confidence; when the IOU between these adjacent bounding boxes and the selected bounding box M is greater than a specified threshold the confidence of the adjacent bounding boxes is multiplied by a Gaussian function, thereby reducing the confidence of these adjacent bounding boxes: ； In the formula, M represents the selected bounding box with the highest confidence, is a threshold specified in NMS, b 𝑖 represents the remaining bounding boxes, s i represents the classification confidence of the corresponding bounding box. represents the formula for calculating the intersection over union between two bounding boxes, and σ represents the standard deviation in the Gaussian function.
4. The method for detecting the fluorescent crack of the rivet based on the improved YOLOV5 model according to claim 1, characterized in that, (2.1) the labeled image is subjected to a 6x6 convolution layer to obtain a feature map Feature_C0; (2.2) the feature map Feature_C0 is input into the ECSAM attention mechanism module, and then passes through a C3 module to enhance the effective features of the cracks and output a feature map Feature_C1; (2.3) the output feature map Feature_C1 is input again into the ECSAM attention mechanism module, and then passes through a C3 module to obtain a feature map Feature_C2; the feature map Feature_C2 is input again into the ECSAM attention mechanism module, and then passes through a C3 module to obtain a feature map Feature_C3; (2.4) after the feature map Feature_C3 passes through the ECSAM attention mechanism module and the C3 module, the feature map undergoes spatial pyramid pooling SPPF to obtain a feature map Feature_C4 as the input of a multi-scale feature fusion module; (2.5) the feature map Feature_C4 is subjected to 3x3 convolution and an upsampling operation to obtain a feature map Feature_Up1; (2.6) Fuse the feature map Feature_Up1 with the feature map Feature_C3, and then pass through a C3 module and a 3x3 convolution to obtain a fused feature map Feature_Fuse1, and perform an upsampling operation on the feature map Feature_Fuse1 to obtain a feature map Feature_Up2; (2.7) Fuse the feature map Feature_Up2 with the feature map Feature_C2 to obtain a feature map Feature_Fuse2, pass through a C3 module to obtain a feature map Feature_C5, and then pass through a 3x3 convolution to obtain a feature map Feature_C6; (2.8) Fuse the feature map Feature_C6 with the feature map Feature_Fuse1 to obtain a feature map Feature_Fuse3, pass through a C3 module to obtain a feature map Feature_C7, and then pass through a 3x3 convolution to obtain a feature map Feature_C8, (2.9) Fuse the feature map Feature_C8 with the feature map Feature_C4 after the 3x3 convolution to obtain a feature map Feature_Fuse4, pass through a C3 module and a 1x1 convolution to obtain a detection layer Feature_D0; (2.10) Perform a 1x1 convolution on the feature map Feature_C7 to obtain a detection layer Feature_D1, and perform a 1x1 convolution on the feature map Feature_C5 to obtain a detection layer Feature_D2; (2.11) Fuse the feature map Feature_Fuse2 with a 3x3 convolution, an upsampling operation, a C3 module, a 3x3 convolution, an upsampling operation, a 3x3 convolution, a C3 module, an upsampling operation, and a 1x1 convolution in sequence to obtain a segmentation layer Feature_S0.