Clamp defect detection model and clamp detection method based on YOLOv51 framework

Abstract

The clip defect detection model and clip detection method based on the YOLOv51 framework extract the Backbone part features of YOLOv51 in the aerial image; the C3 module and the down-sampling convolution layer in the Backbone part are replaced by using a multi-level cascade module and a down-sampling excessive module, and a pyramid compression attention is introduced to reconstruct the Backbone part; a CSPSPP module is proposed to replace the SPPF module of YOLOv51; and the Backbone part and the Neck module are further reconstructed by using the residual idea; the CIoU positioning loss of YOLOv51 is replaced by using the loss function SIoULoss, so that the prediction frame can better regress small targets. First, based on the position correlation of the aerial image, the fitting and the clip, the fitting position detected by the YOLOv51 detection model is used, and the predicted fitting image is cropped out; then, the cropped fitting image is transmitted to the improved YOLOv51 detection model for clip detection, and the final clip image is obtained. The model and method detect the clip based on the improved YOLOv51 algorithm, and improve the efficiency and accuracy of the clip defect recognition.

Description

Technical Field

[0001] This invention relates to the field of wire clamp defect identification technology, specifically to a wire clamp defect detection model and method based on the YOLOv51 framework. Background Technology

[0002] The primary task of wire clip inspection is to detect the wire clips. The mainstream target detection algorithms of the past decade have mainly evolved from deep learning, and can be categorized into single-stage and two-stage algorithms based on their underlying principles. Single-stage models offer faster inference speeds and only regress the result once throughout the entire process. The YOLO algorithm, as the first single-stage detector in the deep learning era, directly regresses coordinates and category after feature extraction. Later, researchers proposed versions lv2 and v3, and other researchers also modified YOLO, including YOLOv4, YOLOX, PPYOLO, and YOLOv5. Although YOLO's detection speed is significantly improved compared to two-stage algorithms, its accuracy in locating small targets remains insufficient.

[0003] Subsequently, scholars proposed the SSD detector, which introduced multi-resolution detection technology and improved the detection accuracy of single-stage detectors. In 2017, T-YLin et al. proposed the RetinaNet model in their paper, which introduced a new loss function, Focal-Loss, to help the detector focus on hard-to-classify samples during training, addressing the problem of extreme foreground and background sample imbalance during the training process.

[0004] Cable clamp defect identification requires the model to first clearly detect and accurately regress the location of the cable clamp in aerial images (6280×3040 or 4000×2010). Cable clamps occupy only 0.2% to 3% of the area in aerial images. Since cable clamps occupy a very small proportion in aerial images, this poses a significant challenge to the model's feature extraction and regression strategies. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a wire clamp defect detection model and method based on the YOLOv51 framework. This model and method detect wire clamps based on an improved YOLOv51 algorithm, thereby improving the efficiency and accuracy of wire clamp defect identification.

[0006] The technical solution adopted in this invention is as follows:

[0007] A wire clamp defect detection model based on the YOLOv51 framework is established, and the method includes the following steps:

[0008] Step 1: Extract the Backbone features of YOLOv51;

[0009] Step 2: Replace the C3 module and downsampling convolutional layer in the Backbone part with a multi-layer cascaded module and a downsampling transition module, and reconstruct the Backbone part by introducing pyramid compressed attention;

[0010] Step 3: Propose the CSPSPP module to replace the SPPF module of YOLOv51; and further refactor the Babbone part and Neck module using the residual concept.

[0011] Step 4: Replace the CIoU localization loss of YOLOv51 with the loss function SIoULoss to enable the predicted bounding box to better regress small targets.

[0012] In step 1, firstly, the MaxPooLing layer input from the 640×640 aerial image is downsampled to a MaxPooLing layer size of 160×160 through two convolutions; then, feature extraction is performed through multiple C3 modules; and then downsampling and upscaling are performed separately through convolutions with a stride of 2, finally scaling to a size of 20×20, thus completing the feature extraction of the Backbone part of the aerial image.

[0013] In step 2, a multi-layer cascaded module structure is first proposed. This module adds a residual connection after each convolution, employing six different paths. The shortest path contains only one convolutional layer, while the longest path contains five convolutional layers. Then, a downsampling transition module is proposed. The left branch consists of a 2×2 max-pooling layer followed by a 1×1 convolution, and the right branch consists of a 1×1 convolution followed by a 3×3 kernel convolution with a stride of 2. In step 2, pyramid compressed attention is developed from the SE module. The SE module consists of compression and excitation, used to encode global information and adaptively recalibrate channel relationships, respectively. Global average pooling (GAP) is used to generate channel statistics g. c This allows global spatial information to be embedded into a channel descriptor, and the specific calculation process is shown in equations (1) and (2):

[0014]

[0015] ω c =σ(W1δ(W0(g) c ))) (2);

[0016] Where: H, W, and C represent height, width, and number of input channels, respectively; x c (i,j) represents the horizontal and vertical coordinates of the feature map; i represents the horizontal coordinate; j represents the vertical coordinate; W0(g c) represents the channel information generated by the first fully connected operation; δ represents the ReLU nonlinear activation function; W0 and W1 refer to the two fully connected operations respectively; σ is the activation function, using the Sigmoid function, through which the weight ω of each channel can be obtained. c Then, a weighted operation is performed on the original feature map to obtain a new feature map. This process of generating attention weights is called the SE weight module.

[0017] The implementation of pyramid-compressed attention includes the following steps:

[0018] S2.1: First, multi-scale feature maps along the channel direction are obtained through compression cascade modules;

[0019] S2.2: Secondly, by using the SE weighting module to extract the weight coefficients of the feature maps at different scales, the channel direction attention vector is obtained;

[0020] S2.3: Third, by using Softmax to recalibrate the channel orientation attention vector, the recalibrated weights of the multi-scale channels are obtained; Softmax can convert linear predictions into class probabilities, and the calculation formula is shown in Equation (3):

[0021]

[0022] in: This indicates exponential calculation; x represents the input value; i and j represent the horizontal and vertical directions respectively; Max ori This represents the maximum value of all inputs;

[0023] S2.4: Fourth, the recalibrated weights and feature maps are multiplied element-wise to finally output a refined feature map that incorporates multi-scale information.

[0024] In step 2, in the compression cascade module, the feature map with channel dimension C is first subdivided into four parts, with each branch having a dimension of C / 4. Each branch, from top to bottom, uses 3×3, 5×5, 7×7, and 9×9 grouped convolutions to obtain receptive fields at different scales and extract information at different scales. These four branches are operated in parallel, and different convolution kernel operations constitute a pyramid structure. As shown in formula (4):

[0025] F i =Conv(k i ×k i G i (X) i (4);

[0026] Wherein: F i Denotes the multi-scale feature mapping generation function for the i-th branch; k iG represents the size of the i-th convolutional kernel; i X represents the number of groups in the i-th branch; i This represents the original feature map of the i-th branch;

[0027] Finally, the results of the four branches are concatenated along the channel dimension to obtain the feature map F, as shown in Equation (5):

[0028] F = Cat([F0,F1,...,F) S-1 ]) (5);

[0029] Where: F0, F1, ..., F S-1 Represent the multi-scale feature mapping generation function of the 0-(s-1) branch respectively; s represents the channel dimension; Cat([F0,F1,...,F...)] S-1 ]) indicates that the multi-scale feature mapping generation function of the 0-(s-1) branch is concatenated in the channel dimension.

[0030] The new feature map F is fed into the SE weight module to obtain the attention weights of feature maps at different scales. The relevant formula is shown in equation (6):

[0031] Z i =SEWeight(F i ),i=1,2...,S-1 (6);

[0032] Among them: Z i For attention weights, the SEWeight module is used to extract attention weights from input feature maps at different scales.

[0033] The improved YOLOv51 model incorporates pyramid-compressed attention, fusing contextual information at different scales to generate better upward attention. The pyramid-compressed attention is placed in the first multi-layer cascaded module, replacing the first 3×3 convolutional layer. The last multi-layer cascaded module in the new backbone retains the original number of channels.

[0034] In step 3, the Neck module is reconstructed, as follows:

[0035] In YOLOv51, the Neck module is built using the SPPF module, convolutional blocks, and C3 module. The SPPF module is further improved using the concepts of CSPSPP module and pyramid, and the structure of the SPPF module and the overall structure of the Neck module are adjusted.

[0036] The Neck module serially passes the input through multiple 5×5 MaxPpooling layers. The CSPSPP module replaces the original YOLOv51 SPPF module. The CSPSPP module adopts a two-branch structure, including a left branch and a right branch. The right branch is a pyramid pooling operation, which uses a 5×5, 9×9 and 13×13 max pooling layer and a no-processing layer for concatenation. Finally, it is concatenated with the information flow of the left branch in the channel dimension.

[0037] In addition, a large number of C3 modules in the Neck module are replaced with multi-layer cascaded modules and downsampling modules, and the downsampling convolution module Conv-BN-SiLU is replaced with a downsampling over-module.

[0038] In step 4, the loss function SIoULoss is used instead of the CIoU localization loss CIoULoss of YOLOv51. Specifically:

[0039] The SIoULoss angular loss function minimizes the number of distance-dependent variables by shifting the predicted bounding box to the nearest X or Y axis and continuing to approach along the relevant axis. To get closer to the ground truth bounding box B... GT The predicted bounding box B needs to minimize α or β first, and convergence is achieved by choosing an angle less than π / 4.

[0040] Choose angle α as the first convergence angle, then β = π / 2 - α; the angle loss is defined by equation (8), and equation (9) explains x, σ, and c. h The calculation method;

[0041]

[0042] Where Λ represents angle cost, x represents the ratio of the opposite side to the hypotenuse in a right triangle, and c h It represents the height of the smallest bounding rectangle between the ground truth bounding box and the predicted bounding box.

[0043] x = sin(α) is the ratio of the opposite side to the hypotenuse in a right triangle; σ represents the distance between the center point of the true bounding box and the center point of the predicted bounding box. and The coordinates of the center of the true bounding box; and These are the coordinates of the center of the prediction box.

[0044] When considering angle cost, distance cost needs to be redefined. Distance loss takes into account the minimum bounding rectangle between two boxes. Equation (10) describes the method for measuring distance loss, and Equation (11) explains ρ. x ρ y And the calculation method of γ,

[0045]

[0046] Where: △ represents distance cost, t represents direction, x represents the x-axis direction, y represents the y-axis direction, γ represents distance coefficient, and ρ represents angle coefficient. -γρt e represents the total rotation angle. -γρx e represents the angle of rotation about the x-axis. -γρy Indicates the angle of rotation about the y-axis; Represents the x-coordinates of the ground truth and predicted bounding boxes. Represents the y-coordinates of the actual and predicted bounding boxes.

[0047] c w and c h Let α be the width and height of the minimum bounding rectangle of the ground truth box and the predicted box; when α is small, the corner cost Λ will also decrease and the corresponding γ will increase, thus reducing the distance cost.

[0048] In addition, the loss function SIoULoss also considers the shape loss between the predicted box and the ground truth box, and the shape loss Ω is shown in Equation (12):

[0049]

[0050] Where: w and h are the width and height of the prediction box, respectively; h gt w gt These are the width and height of the prediction box, respectively; ω t ω represents the relative difference function; w ω represents the relative difference between the predicted bounding box width and the true bounding box width; h This represents the relative difference between the predicted bounding box height and the actual bounding box height. This represents the relative difference coefficient. This represents the relative difference coefficient between the predicted bounding box width and the actual bounding box width. This represents the relative difference coefficient between the predicted bounding box height and the actual bounding box height; max(w,w) gt ) represents the larger of the predicted bounding box width and the actual bounding box width; max(h,h) gt ) represents the larger of the predicted bounding box height and the actual bounding box height; θ controls the degree of attention paid to shape loss;

[0051] The final loss function includes angle loss, distance loss, shape loss and IoU loss; as shown in Equation (14), Equation (15) describes how IoU is calculated.

[0052]

[0053] Where: L locLet IoU represent the loss function, B represent the loss cost, and B represent the prediction box. GT Represents a real bounding box.

[0054] The wire clamp defect detection method based on the YOLOv51 framework first uses the YOLOv51 detection model to detect the position of the hardware and the wire clamp based on the positional correlation of the aerial image, the hardware, and the wire clamp, and then crops out the predicted hardware image. Then, the cropped hardware image is transmitted to the improved YOLOv51 detection model for wire clamp detection to obtain the final wire clamp image.

[0055] This invention provides a wire clamp defect detection model and method based on the YOLOv51 framework, with the following technical advantages:

[0056] 1) In step 2 of this invention: First, the application of multi-layer cascaded modules and downsampling modules: By introducing multi-layer cascaded modules and downsampling transition modules to replace the original C3 module in the backbone structure, features can be extracted more effectively, and the shortest and longest gradient paths of the network can be controlled, enabling the network to learn and converge more effectively. Second, the extension of the residual idea: By extending the residual idea, the C3 module and downsampling convolutional layers are reconstructed, adding convolutional layers with different paths, enabling the network to learn more features at different levels, thereby improving the accuracy of the model. In addition, pyramid compressed attention is introduced: A pyramid compressed attention mechanism is added, aiming to capture spatial information at different scales to enrich the feature space. This is particularly important for the recognition of small targets, because small targets occupy a small proportion in the image and require sufficiently detailed feature information for accurate recognition.

[0057] 2) In step 3 of this invention: First, the SPPF module is improved: by proposing the CSPSPP module to replace the original SPPF module, this invention utilizes the ideas of CSPNet and pyramid pooling to enhance the receptive field of the network and improve the computational speed. Second, the residual idea is further applied: in step 3, the residual idea is further used to reconstruct the Backbone and Neck parts, enhancing the network's feature learning ability and helping to improve the model's recognition accuracy for small targets. In addition, the most important aspect is the optimization of the Neck module: by replacing a large number of C3 modules in the Neck module with multi-layer cascaded modules and downsampling modules, the feature fusion method is improved, which helps to improve the model's detection performance for targets of different scales.

[0058] 3) In step 4 of this invention: First, a novel loss function, SIoULoss, is introduced: This invention uses the SIoULoss loss function to replace the CIoU localization loss used in YOLOv51. SIoULoss is a loss function based on binary cross entropy. It introduces a dynamic scaling factor, which can dynamically reduce the weight of easily distinguishable samples during training, thereby focusing the training focus on difficult-to-distinguish samples. Second, it solves the problem of imbalanced sample classification and improves the localization accuracy of small targets: The purpose of SIoULoss is to solve the problem of imbalanced sample classification, especially when faced with a large number of easily classifiable samples. It reduces the impact of these samples on model training, allowing the model to focus more on difficult-to-distinguish samples and improve the recognition accuracy of these samples. With the introduction of SIoULoss, the improved YOLOv51 model can better regress small targets, which is particularly important for wire clip defect detection, because wire clips account for a very small proportion in aerial images, placing higher demands on the model's localization accuracy. In addition, the convergence speed and effectiveness of the model's predicted bounding boxes were improved: the use of the SIoULoss loss function improved the convergence speed and effectiveness of the model's predicted bounding boxes, thereby improving the overall detection accuracy.

[0059] 4) The wire clamp detection scheme proposed in this invention not only improves upon the traditional YOLOv51 detection framework but also employs a cascaded model detection scheme. This bottom-up detection method helps to progressively identify the parts of interest, improving the accuracy of target detection and defect identification. Furthermore, experimental verification shows that the improved YOLOv51 model outperforms the original YOLOv51 model and other existing algorithms in key metrics, demonstrating the effectiveness of the improvements. Attached Figure Description

[0060] The present invention will be further described below with reference to the accompanying drawings and examples;

[0061] Figure 1 This is a schematic diagram of the original structure of the Backbone section of YOLOv51.

[0062] Figure 2 This is a diagram of the CSPrayer architecture.

[0063] Figure 3 This is a diagram of a multi-layered cascaded module structure.

[0064] Figure 4 This is a structural diagram of the downsampling transition module, Transition-Block.

[0065] Figure 5 This is a diagram illustrating pyramid-shaped attention compression.

[0066] Figure 6This is a schematic diagram of a compression cascade module.

[0067] Figure 7 This is a schematic diagram of the SE weight module.

[0068] Figure 8 To recalibrate the weight structure diagram.

[0069] Figure 9 New Backbone section structure diagram.

[0070] Figure 10 This is a structural diagram of the Neck module in YOLOv51.

[0071] Figure 11 This is a structural diagram of the CSPSPP module.

[0072] Figure 12 This is a diagram of the SPPF module structure.

[0073] Figure 13 This is a diagram of the new Neck module structure.

[0074] Figure 14 This is a schematic diagram of SIoU Loss - angle loss.

[0075] Figure 15 This is a flowchart of the overall process for wire clamp testing.

[0076] Figure 16 This is a flowchart for direct detection of wire clamps using a single model. Detailed Implementation

[0077] A wire clamp defect detection model based on the YOLOv51 framework is established, and the method includes the following steps:

[0078] Step 1: Extract the Backbone features of YOLOv51;

[0079] Step 2: Replace the C3 module and downsampling convolutional layer in the Backbone part with a multi-layer cascaded module and a downsampling over-module, and introduce pyramid compressed attention to enable the model to learn feature information at different scales, and reconstruct the Backbone part;

[0080] Step 3: Improve the SPPF module based on CSPNet and the pyramid idea, and propose the CSPSPP module to replace the SPPF module of YOLOv51; and further reconstruct the Babbone part and Neck module using the residual idea;

[0081] Step 4: Replace the CIoU localization loss of YOLOv51 with the loss function SIoULoss to enable the predicted bounding box to better regress small targets.

[0082] In step 1, the Backbone portion of the YOLOv51 network is as follows: Figure 1 As shown, firstly, the MaxPooLing layer input from the 640×640 aerial image is downsampled to a 160×160 MaxPooLing layer size through two convolutions; then, feature extraction is performed through multiple C3 modules; finally, downsampling and dimensionality upsampling are performed separately through convolutions with a stride of 2 (such as P2-P5), and finally, the image is scaled to a size of 20×20, thus completing the feature extraction of the backbone portion of the aerial image.

[0083] CSPrayer in the C3 module, such as Figure 2 As shown, it utilizes the CSNet network architecture to split the original stacked residual blocks into two parts, left and right. One part outputs the result through a convolution, while the other part goes through stacked BottleNeck convolutional layers to finally splice the result.

[0084] In step 2, to further enhance the model's feature capabilities, this invention extends the residual concept by reconstructing the C3 module and the downsampling convolutional layer. Simultaneously, it controls the shortest and longest gradient paths to enable the network to learn and converge more effectively. Unlike previous P3-P5 network layers, this invention implements the downsampling and dimensionality upsampling operations separately.

[0085] like Figure 3 As shown, this invention first proposes a multi-layer cascaded module structure, such as... Figure 3 As shown, this module adds a residual connection after each convolution, employing a total of six different paths. The shortest path contains only one convolutional layer, while the longest path contains five. This allows gradients at different levels to learn more diverse features, thereby improving accuracy. Furthermore, this multi-cascaded structure corresponds to denser residual connections. Residual networks are characterized by being easier to optimize, and their accuracy can be further improved by increasing their depth.

[0086] A downsampling over-module is proposed, such as Figure 4 As shown, existing models all use a convolutional kernel of size 3 and a stride of 2 or a pooling layer with a stride of 2 for downsampling. This invention restructures this method, integrating the two transition modules. For example... Figure 4 As shown, the left branch consists of a max pooling layer with a stride of 2×2 followed by a 1×1 convolution, while the right branch consists of a 1×1 convolution followed by a convolution with a kernel size of 3×3 and a stride of 2. The results of these two branches are stacked at output, halving the tensor size without affecting the channel dimension. This method can reduce the information loss caused by model downsampling.

[0087] In step 2, considering that the hardware and wire clamps occupy a very small area in the image, sufficiently detailed feature information is required. This invention incorporates pyramid compression attention to capture spatial information at different scales to enrich the feature space.

[0088] Pyramid compressed attention is developed based on the classic SE attention model. SE attention is a channel-based attention model that weights channels according to their importance to obtain more information in the output. Here, H, W, and C represent height, width, and the number of input channels, respectively. The SE module consists of two parts: compression and excitation. Compression is used to encode global information, and excitation is used to adaptively recalibrate channel relationships. Global average pooling (GAP) is typically used to generate channel statistics (g). c This allows global spatial information to be embedded into a channel descriptor, and the specific calculation process is shown in equations (1) and (2):

[0089]

[0090] Where: H, W, and C represent height, width, and number of input channels, respectively; x c (i,j) represents the horizontal and vertical coordinates of the feature map; i represents the horizontal coordinate; j represents the vertical coordinate; W0(g c The ) represents the channel information generated by the first fully connected operation. δ represents the ReLU nonlinear activation function; W0 and W1 refer to the two fully connected operations respectively; σ is the activation function, which is usually the Sigmoid function used in practice. The weight ω of each channel can be obtained through the Sigmoid function. c Then, a weighted operation is performed on the original feature map to obtain a new feature map. This process of generating attention weights is called the SE weight module.

[0091] The implementation of pyramid-compressed attention includes the following steps, such as... Figure 5 As shown:

[0092] S2.1: First, through, as follows Figure 5 The compression cascaded module shown obtains multi-scale feature maps along the channel direction;

[0093] S2.2: Next, by using the SE weighting module to extract the weight coefficients of the feature maps at different scales, the channel orientation attention vector is obtained. The process is as follows: Figure 7 As shown, the calculation process is shown in equations (1) to (2);

[0094] S2.3: Third, by using Softmax to recalibrate the channel orientation attention vector, the recalibrated weights of the multi-scale channels are obtained; the Softmax function has differentiable properties and can be used with optimization algorithms such as gradient descent. The Softmax function uses the natural exponent as a nonlinear function to amplify small differences in input values, thereby increasing the accuracy of classification. As a normalized exponential function, Softmax can convert linear predictions into class probabilities, and the calculation formula is shown in equation (3):

[0095]

[0096] in: This indicates exponential calculation; x represents the input value; i and j represent the horizontal and vertical directions respectively; Max ori This represents the maximum value of all inputs. When implementing Softmax in hardware, to avoid exponent overflow, a normalization method is generally used, which is to subtract the maximum value of all inputs from the input x, thereby limiting the range of the exponent to the interval (0,1].

[0097] S2.4: Fourth, the recalibrated weights and feature maps are multiplied element-wise to finally output a refined feature map that incorporates multi-scale information. The process is as follows: Figure 8 As shown.

[0098] In S2.1, in the compression cascade module, the feature map with channel dimension C is first subdivided into four parts, each branch with dimension C / 4. Each branch uses 3×3, 5×5, 7×7 and 9×9 grouped convolutions from top to bottom to obtain receptive fields at different scales and extract information at different scales. These four branches are operated in parallel, and different convolution kernel operations constitute a pyramid structure. Grouped convolution can reduce the large computational cost of large kernel convolution, as shown in formula (4):

[0099] F i =Conv(k i ×k i G i (X) i (4);

[0100] Wherein: F i Denotes the multi-scale feature mapping generation function for the i-th branch; k i G represents the size of the i-th convolutional kernel; i X represents the number of groups in the i-th branch; i This represents the original feature map of the i-th branch;

[0101] Finally, the results of the four branches are concatenated along the channel dimension to obtain the feature map F, as shown in Equation (5):

[0102] F = Cat([F0,F1,...,F) S-1 ]) (5);

[0103] Where: F0, F1, ..., F S-1 Represent the multi-scale feature mapping generation function of the 0-(s-1) branch respectively; s represents the channel dimension; Cat([F0,F1,...,F...)] S-1 ]) indicates that the multi-scale feature mapping generation function of the 0-(s-1) branch is concatenated in the channel dimension.

[0104] The new feature map F is fed into the SE weight module to obtain the attention weights of feature maps at different scales. The relevant formula is shown in equation (6):

[0105] Z i =SEWeight(F i ),i=1,2...,S-1 (6);

[0106] Among them: Z i For attention weights, the SEWeight module is used to extract attention weights from input feature maps at different scales.

[0107] The improved YOLOv51 model, employing pyramid-compressed attention, fuses contextual information at different scales, resulting in better upward attention. In this invention, pyramid-compressed attention is placed in the first multi-layer cascaded module, replacing the first 3×3 convolutional layer. The new backbone structure is as follows: Figure 9 As shown, the last layer of the multi-layer cascaded module maintains the original number of channels.

[0108] In step 3, the SPPF module is replaced by the CSPSPP module, as detailed below:

[0109] The SPPF module primarily serially passes the input through multiple 5×5 MaxPooling layers, improving computational speed while enhancing the network's receptive field. However, this structure still has room for improvement. This invention utilizes the concepts of CSPNet and pyramid pooling to replace the original YOLOv51 SPPF module with a CSPSPP module structure.

[0110] The structure diagram of the YOLOv51 Neck module is as follows: Figure 10As shown, it basically adopts the idea of ​​PANet. YOLOv51 extracts three feature layers for object detection, which are located in the middle, lower middle, and bottom layers of the backbone network, respectively. When the input = (640, 640, 3), the sizes of the three feature layers are feat1 = (80, 80, 256), feat2 = (40, 40, 512), and feat3 = (20, 20, 1024), respectively. After obtaining the three effective feature layers, PANet is built using these three network layers. By fusing feature layers at different locations, the fine-grained information from the preceding image and the semantic information from the following image can be fully integrated, enhancing the expressive power of the feature layers and improving the detection results in the later stages.

[0111] The Neck module has been refactored, as detailed below:

[0112] The Neck module in YOLOv51 primarily adopts the ideas of PANet, and is built using the SPPF module, convolutional blocks, and C3 module. This invention further improves the SPPF module using the CSPSPP module and the pyramid concept.

[0113] We propose a CSPSPP module to replace the SPPF module, and further utilize the residual concept to reconstruct the Bacbone and Neck parts to enhance the network's feature learning ability.

[0114] To better integrate fine-grained and semantic information, this invention adjusts the structure of the SPPF module and the overall structure of the Neck module. The Neck module serially passes the input through multiple 5×5 MaxPpooling layers, improving computational speed while enhancing the network's receptive field. However, this structure still has room for improvement. This invention, leveraging the ideas of CSPNet and pyramid pooling, uses a CSPSPP module to replace the original YOLOv51 SPPF module, such as... Figure 12 As shown.

[0115] The CSPSPP module adopts a two-branch structure, as shown below. Figure 11 As shown, it includes a left branch and a right branch. The right branch is a pyramid pooling operation, which uses a 5×5, 9×9 and 13×13 max pooling layer and a no-processing operation layer for concatenation. Finally, it is concatenated with the information flow of the left branch in the channel dimension.

[0116] In addition, a large number of C3 modules in the Neck module will be used as... Figure 3 The multi-layer cascaded modules shown and as follows Figure 4 The downsampling module shown is replaced by a downsampling transition module, and the downsampling convolutional module Conv-BN-SiLU is replaced by a downsampling transition module. The new Neck module is as follows: Figure 13 As shown.

[0117] In step 4

[0118] Loss functions play a crucial role in object detection tasks. As a penalty measure, the loss function needs to be minimized during training to match the predicted object bounding boxes with the corresponding ground truth (GT) boxes. Therefore, loss functions have received extensive research in object detection in recent years. Existing localization loss functions all consider metrics of "mismatch" between two boxes, including center distance, overlap region, and aspect ratio.

[0119] In YOLOv51, the loss function mainly consists of three parts: classification loss, which uses BCELoss and only calculates the classification loss of positive samples; objectness loss, which also uses BCELoss, where obj refers to the CIoU between the target bounding box predicted by the network and the ground truth box; and localization loss, which uses CIoULLoss and only calculates the localization loss of positive samples. Equation (7) is the total loss function, where λ1, λ2, and λ3 are balance coefficients.

[0120] Loss=λ1L cls +λ2L obj +λ3L loc (7);

[0121] These localization loss methods all measure the distance between the bounding box's length, width, and center point, ignoring the vector angle between the predicted and ground truth bounding boxes. By using the angle distance, the predicted bounding box can be accelerated to drift towards the nearest X-axis or Y-axis, which can greatly help model training.

[0122] The loss function in YOLOv51 consists of three parts. To improve the regression of the predicted bounding box to small targets, this invention uses the loss function SIoULoss instead of the CIoU localization loss CIoULoss in YOLOv51. Specifically:

[0123] To address the issues of small target localization accuracy and background sample imbalance during training, a novel loss function, SIoU Loss, is introduced to help the detector focus on hard-to-classify samples during training. SIoU Loss is a loss function based on binary cross entropy. It introduces a dynamic scaling factor to dynamically reduce the weight of easily distinguishable samples during training, thereby quickly focusing the attention on those hard-to-classify samples. The design purpose of SIoU Loss is to solve the problem of sample imbalance, especially when faced with a large number of easily distinguishable samples. It effectively reduces the contribution of these samples to model training, allowing the model to focus more on those hard-to-classify samples, thus improving the model's accuracy on hard-to-classify samples.

[0124] The SIoULoss loss function, an angular loss, aims to minimize the number of distance-dependent variables. The improved YOLOv51 model needs to bring the predicted box to the nearest X or Y axis and continue approaching along the relevant axis. To get closer to the ground truth box B... GT The predicted bounding box B needs to minimize α or β first, and usually an angle less than π / 4 is chosen for convergence;

[0125] exist Figure 14 In this invention, angle α is chosen as the first convergence angle, and then β = π / 2 - α; the angle loss can be defined by equation (8), and equation (9) explains x, σ, and c. h The calculation method;

[0126]

[0127] Where Λ represents angle cost, x represents the ratio of the opposite side to the hypotenuse in a right triangle, and c h It represents the height of the smallest bounding rectangle between the ground truth bounding box and the predicted bounding box.

[0128] x = sin(α) is the ratio of the opposite side to the hypotenuse in a right triangle; σ represents the distance between the center point of the true bounding box and the center point of the predicted bounding box. and The coordinates of the center of the true bounding box; and These are the coordinates of the center of the prediction box.

[0129] When considering angle cost, distance cost needs to be redefined. Distance loss takes into account the minimum bounding rectangle between two boxes. Equation (10) describes the method for measuring distance loss, and Equation (11) explains ρ. x ρ y And the calculation method of γ,

[0130]

[0131] Where: △ represents distance cost, t represents direction, x represents the x-axis direction, y represents the y-axis direction, γ represents distance coefficient, and ρ represents angle coefficient. -γρt e represents the total rotation angle. -γρx e represents the angle of rotation about the x-axis. -γρy Indicates the angle of rotation about the y-axis; Represents the x-coordinates of the ground truth and predicted bounding boxes. Represents the y-coordinates of the actual and predicted bounding boxes.

[0132] c w and c h Let α be the width and height of the minimum bounding rectangle of the ground truth box and the predicted box; when α is small, the corner cost Λ will also decrease and the corresponding γ will increase, thus reducing the distance cost.

[0133] In addition, the loss function SIoULoss also considers the shape loss between the predicted box and the ground truth box, and the shape loss Ω is shown in Equation (12):

[0134]

[0135] Where: w and h are the width and height of the prediction box, respectively; h gt w gt These are the width and height of the prediction box, respectively; ω t ω represents the relative difference function; w ω represents the relative difference between the predicted bounding box width and the true bounding box width; h This represents the relative difference between the predicted bounding box height and the actual bounding box height. This represents the relative difference coefficient. This represents the relative difference coefficient between the predicted bounding box width and the actual bounding box width. This represents the relative difference coefficient between the predicted bounding box height and the actual bounding box height; max(w,w) gt ) represents the larger of the predicted bounding box width and the actual bounding box width; max(h,h) gt ) represents the larger of the predicted bounding box height and the actual bounding box height.

[0136] θ controls the degree of attention paid to shape loss. In order to avoid focusing too much on shape loss and reducing the movement of the prediction box, this invention directly uses θ = 4 as a constant.

[0137] The final loss function includes angle loss, distance loss, shape loss and traditional IoU loss, as shown in Equation (14), and Equation (15) describes how IoU is calculated.

[0138]

[0139] Where: L loc Let IoU represent the loss function, B represent the loss cost, and B represent the prediction box. GT The newly built detection model, which represents the ground truth bounding boxes, is uniformly referred to as Improved-YOLOv51 in this invention, based on the improvements made to each module of YOLOv51.

[0140] Example:

[0141] As the first single-stage detector in the era of deep learning, the YOLO algorithm directly regresses coordinates and categories after feature extraction, significantly improving detection speed compared to two-stage algorithms. However, its accuracy in locating small targets remains insufficient. The Backbone network in YOLOv51 borrows ideas from CSPNet, and the constructed CSP-DarkNet53 can fully integrate low-level feature information and high-level semantic information. However, its performance is not ideal in scenarios requiring ultra-high resolution to identify small targets. The Backbone part bears the heavy responsibility of feature extraction and is also the most computationally intensive part of the entire model, thus requiring feature enhancement and multi-scale feature learning. To address this problem, this invention proposes two modules to replace the original Backbone's C3 module and downsampling convolutional layers, and introduces pyramid compressed attention to learn information at different scales.

[0142] 1. Platform Construction and Evaluation Metrics:

[0143] (1) Experimental platform:

[0144] Training and inference in deep learning image recognition require extensive matrix operations. To better accomplish the project's objectives, all experiments in this invention were conducted on a server with two NVIDIA RTX 3090 graphics cards and an AMD EPYC 7763 CPU. The software used a Centons 7 operating system under Linux and a Python-based programming language. For ease of implementation, the mainstream PyTorch neural network computing framework was adopted.

[0145] (2) Evaluation indicators:

[0146] The evaluation metric used in this invention is primarily the average accuracy (AP), which can be further divided based on the IoU threshold between the predicted bounding box and the ground truth (GT) bounding box. This invention selects AP... [0.5] and AP [0.5:0.95] Two are used as accuracy evaluation indicators, among which AP [0.5:0.95] This mainly refers to the average sum of APs with IoU thresholds ranging from 0.5 to 0.95 in intervals of 0.05, i.e., equation (16):

[0147]

[0148] 2. Analysis of Aerial Image Inspection Results for Fittings:

[0149] The main purpose of this test is to verify the effectiveness of Improved-YOLOv51 on the hardware dataset. This invention trains both the original YOLOv51 and Improved-YOLOv51 on the hardware dataset under the same environment and validates them on the test set. The final results are shown in Table 1. From the AP (Average Performance) metric...[0.5] and AP [0.5:0.95] In terms of performance, Improved-YOLOv51 outperforms the YOLOv51 model by 4.9 and 4.7 points, respectively.

[0150] Table 1. Results of Fitting Inspection

[0151]

[0152] Based on the above hardware test results, the improved-YOLOv51 module will be subjected to an ablation experiment to verify the effectiveness of the proposed module.

[0153] Table 2 Hardware Testing - Improved - YOLOv51 Ablation Test

[0154]

[0155] As shown in Table 2, when the C3 module and downsampling layer in the original model were replaced with a multi-layer cascaded module and a downsampling transition module in the Backbone part, the overall accuracy improved by 1.3 points. When pyramid compressed attention was used to replace the first 3×3 convolution in the multi-layer cascaded module, the overall accuracy improved by another 1.6 points. This fully demonstrates that these improvements can help the Backbone part extract small target features more effectively. Introducing CSPSPP, the multi-layer cascaded module, and the downsampling transition layer into the Neck structure further improved the accuracy by 0.3 points and 0.8 points, respectively. Replacing the localization loss function with SIoU can improve the convergence speed and effect of the model's predicted bounding boxes, and further improve the accuracy by 0.9 points.

[0156] 3. Analysis of the results of wire clamp testing for hardware:

[0157] The two models were trained on the online dataset and the improvement results were tested on the validation set. The specific experimental results are shown in Table 3.

[0158] Table 3 Results of Fitting Inspection for Cable Clamps

[0159]

[0160] As shown in Table 3, in the task of inspecting wire clamps for hardware, Improved-YOLOv51 performed well in two metrics: AP. [0.5] and AP [0.5:0.95] The scores exceeded those of the YOLOv51 model by 4.3 and 3.2 points respectively. This fully verifies the effectiveness of the improved method proposed in this invention.

[0161] 4. Analysis of the results of the model cascade scheme:

[0162] The wire clamp detection scheme of this invention consists of two steps: First, based on the positional correlation between aerial photography, hardware, and wire clamp (the wire clamp is located at the junction of the hardware and the transmission line), the YOLOv51 detection model is used to detect the hardware position, and the predicted hardware image is cropped out. Then, the cropped hardware image is transmitted to an improved YOLOv51 detection model for wire clamp detection, resulting in the final wire clamp image. The core idea of ​​this scheme is to use a bottom-up detection method, gradually identifying the parts of interest from the whole to the details, in order to improve the accuracy of target detection and defect identification. The specific training and overall wire clamp detection implementation process is as follows: Figure 15 As shown.

[0163] The most common approach is to directly detect the target, such as... Figure 16 As shown in the figure, the ground truth (GT) of the cable clamp is directly marked on the aerial image, and then the predicted position of the cable clamp is obtained directly through training. This invention combines excellent detection algorithms from recent years to conduct a fair comparison between the two schemes, and the specific results are shown in Table 4.

[0164] Based on the data in Table 4, it can be seen that when using the direct detection scheme, Improved-YOLOv51 outperforms other direct detection scheme models, especially in AP. [0.5] In terms of metrics, it exceeds other models by more than 5 points, in AP [0.5:0.95] If the performance index exceeds 3.3, concatenating two Improved-YOLOv51 models can improve AP. [0.5] The accuracy reaches 90.3 in AP metrics. [0.5:0.95] The index reached 48.2 points, which proves the superiority of the detection method and improved model proposed in this paper.

[0165] Table 4. Detection results of wire clamps in the cascaded model versus the single model.

[0166]

Claims

1. A wire clamp detection method based on a wire clamp defect detection model using the YOLOv51 framework, characterized in that: First, based on the positional correlation between the aerial image, the hardware, and the clamp, the YOLOv51 detection model is used to detect the position of the hardware, and the predicted hardware image is cropped out. Then, the cropped hardware image is transmitted to the clamp defect detection model based on the YOLOv51 framework to detect the clamp and obtain the final clamp image. The method for establishing the wire clamp defect detection model based on the YOLOv51 framework includes the following steps: Step 1: Extract the Backbone features of YOLOv51; Step 2: Replace the C3 module and downsampling convolutional layer in the Backbone section with a multi-layer cascaded module and a downsampling transition module, and reconstruct the Backbone section by introducing pyramid compressed attention; Step 3: Propose the CSPSPP module to replace the SPPF module of YOLOv51; and further refactor the Backbone part and Neck module using the residual concept. Step 4: Replace the CIoU localization loss of YOLOv51 with the loss function SIoULoss; In step 2, a multi-layer cascaded module is first proposed. This module adds a residual connection after each convolution, employing six different paths. The shortest path contains only one convolutional layer, while the longest path contains five convolutional layers. Then, a downsampling transition module is proposed, whose left branch is a step size of... max pooling layer plus one Convolution, the right branch is a Convolution plus a convolution kernel of size Convolution with a stride of 2; Pyramid compressed attention is developed based on the SE module, which consists of two parts: compression and excitation. The compression module is used to encode global information and the excitation module is used to adaptively recalibrate channel relationships. Global average pooling (GAP) is used to generate channel statistics. This allows global spatial information to be embedded into a channel descriptor, and the specific calculation process is shown in equations (1) and (2): （1）； （2）； Where H, W, and C represent height, width, and number of input channels, respectively; This represents the horizontal and vertical coordinate information of the feature map; Indicates the horizontal coordinate; Represents the vertical coordinates; This represents the channel information generated during the first full-connection operation; Represents the ReLU operation, a non-linear activation function. and Each refers to one of the two full-join operations; It's the activation function, using the sigmoid function. The sigmoid function provides the weights for each channel. Then, a weighted operation is performed on the original feature map to obtain a new feature map. This process of generating attention weights is called the SE weight module. The implementation of pyramid-compressed attention includes the following steps: S2.1: First, multi-scale feature maps along the channel direction are obtained through compression cascade modules; S2.2: Secondly, by using the SE weighting module to extract the weight coefficients of the feature maps at different scales, the channel direction attention vector is obtained; S2.3: Third, by using Softmax to recalibrate the channel orientation attention vector, the recalibration weights of the multi-scale channels are obtained; S2.4: Fourth, the recalibrated weights and feature maps are multiplied element-wise to finally output a refined feature map that incorporates multi-scale information.

2. The wire clamp detection method based on the YOLOv51 framework-based wire clamp defect detection model according to claim 1, characterized in that: In step 1, firstly, The MaxPooLing layer input in the aerial image is downsampled to two convolutions. The MaxPooLing layer size is determined; then feature extraction is performed through multiple C3 modules; finally, downsampling and dimensionality upsampling are performed separately through convolutions with a stride of 2, and finally scaled to the desired size. The size of the data was used to extract features from the backbone portion of the aerial image.

3. The wire clamp detection method based on the YOLOv51 framework-based wire clamp defect detection model according to claim 1, characterized in that: In the compression cascade module, the feature map with channel dimension C is first subdivided into four parts, with each branch having a dimension of C / 4. Each branch, from top to bottom, employs... The grouped convolutions are used to obtain receptive fields at different scales and extract information at different scales. These four branches are operated in parallel, and different convolution kernel operations constitute a pyramid structure; as shown in formula (4): （4）； in: Let represent the multi-scale feature mapping generation function for the i-th branch; This represents the size of the i-th convolutional kernel; This represents the number of groups in the i-th branch; This represents the original feature map of the i-th branch; Finally, the results of the four branches are concatenated along the channel dimension to obtain the feature map F, as shown in equation (5): （5）； in: Let represent the multi-scale feature mapping generation function of the 0-(s-1) branch respectively; Indicates the channel dimension; This indicates that the multi-scale feature mapping generation function of the 0-(s-1) branch is concatenated along the channel dimension.

4. The wire clamp detection method based on the YOLOv51 framework-based wire clamp defect detection model according to claim 3, characterized in that: The new feature map F is fed into the SE weight module to obtain the attention weights of feature maps at different scales. The relevant formula is shown in equation (6): （6）； in: To emphasize the importance of weights, the SEWeight module is used to extract importance weights from input feature maps at different scales. The improved YOLOv51 model incorporates pyramid compressed attention, fusing contextual information at different scales to generate better upward attention; the pyramid compressed attention is placed in the first multi-layer cascaded module instead of the first one. Convolutional layers; the last layer of the new backbone section, a multi-layered cascaded module, retains the original number of channels.

5. The wire clamp detection method based on the YOLOv51 framework-based wire clamp defect detection model according to claim 1, characterized in that: In step 3, the Neck module is reconstructed, as follows: In YOLOv51, the Neck module is built using the SPPF module, convolutional blocks, and C3 module. The SPPF module is further improved using the concepts of CSPSPP module and pyramid, and the structure of the SPPF module and the overall structure of the Neck module are adjusted. The Neck module serially passes the input through multiple... The MaxPooling layer uses the CSPSPP module instead of the original YOLOv51 SPPF module. The CSPSPP module employs a two-branch structure, including a left branch and a right branch. The right branch performs pyramid pooling operations, each using one... The max pooling layer and a no-processing layer are concatenated, and finally concatenated with the information flow of the left branch in the channel dimension. In addition, a large number of C3 modules in the Neck module are replaced with multi-layer cascaded modules and downsampling modules, and the downsampling convolution module Conv-BN-SiLU is replaced with a downsampling transition module.

6. The wire clamp detection method based on the YOLOv51 framework-based wire clamp defect detection model according to claim 1, characterized in that: In step 4, the loss function SIoULoss is used instead of the CIoU localization loss CIoULoss of YOLOv51; specifically as follows: The SIoULoss loss function, to minimize the number of distance-related variables, shifts the predicted bounding box to the nearest X or Y axis and continues to approach along the relevant axis; to get closer to the ground truth bounding box... The predicted bounding box B needs to be minimized first. Choose less than Convergence is achieved from the angle of convergence; choose The angle is used as the first convergence angle, then ; Angle loss is defined by equation (8): (8); Equation (9) explains Calculation method: （9）； in, Indicates corner cost, This represents the ratio of the opposite side to the hypotenuse in a right triangle. This represents the height of the smallest bounding rectangle between the ground truth bounding box and the predicted bounding box. That is, the ratio of the opposite side to the hypotenuse in a right triangle; This represents the distance between the center point of the ground truth bounding box and the center point of the predicted bounding box; and The coordinates of the center of the true bounding box; The coordinates of the center of the prediction box; When considering angle cost, distance cost needs to be redefined. Distance loss takes into account the minimum bounding rectangle between two boxes; Equation (10) describes the method for measuring distance loss, and Equation (11) explains... The calculation method （10）； （11）； in: Indicates distance cost, Indicates direction, Indicates the x-axis direction. Indicates the y-axis direction. Represents the distance coefficient. Indicates the angle coefficient. Indicates the total rotation angle. This represents the angle of rotation about the x-axis. Indicates the angle of rotation about the y-axis; , Represents the x-coordinates of the ground truth and predicted bounding boxes. , Represents the y-coordinates of the ground truth and predicted bounding boxes; The width and height of the minimum bounding rectangle of the ground truth bounding box and the predicted bounding box; when When the size decreases, the corner cost It will also decrease accordingly. This will increase, thereby reducing the distance cost; In addition, the loss function SIoULoss also considers the shape loss between the predicted bounding box and the ground truth bounding box. As shown in equation (12): （12）； （13）； in: These are the width and height of the predicted bounding box, respectively; These are the width and height of the predicted bounding box, respectively; Represents the relative difference function; This represents the relative difference between the predicted bounding box width and the actual bounding box width. This represents the relative difference between the predicted bounding box height and the actual bounding box height. This represents the relative difference coefficient. This represents the relative difference coefficient between the predicted bounding box width and the actual bounding box width. This represents the relative difference coefficient between the predicted bounding box height and the actual bounding box height. This represents the larger of the predicted bounding box width and the actual bounding box width. This represents the larger of the predicted bounding box height and the actual bounding box height. Control the degree of attention paid to shape loss; The final loss function includes angle loss, distance loss, shape loss and IoU loss; as shown in formula (14), formula (15) describes how IoU is calculated; （14）； （15）； in: Represents the loss function. Indicates the cost of loss. Indicates the prediction box. Represents a real bounding box.

Images