Infrared Image Insulator Rotation Detection Method Based on IoU Prediction

Abstract

This invention discloses an infrared image insulator rotation detection method based on IoU prediction, comprising: labeling insulators in infrared images and constructing a dataset; performing data augmentation on the dataset; designing an insulator rotation detector, adding IoU and rotation angle score prediction branches; modifying the training strategy of the insulator rotation detector and designing loss functions for the IoU and rotation angle score prediction branches; training the insulator rotation detector using the insulator infrared image dataset, and obtaining the trained detector by minimizing the joint loss; detecting images using the insulator rotation detector, designing a θ-IoU-NMS algorithm, and evaluating the quality of the rotated bounding boxes through comprehensive scores; using the θ-IoU-NMS algorithm to suppress the predicted bounding boxes generated by the detector, and selecting more accurately located rotated bounding boxes; and outputting the insulator detection results. This invention can significantly improve the localization accuracy of insulators in infrared images and reduce missed detections and false detections.

Description

Technical Field

[0001] This invention relates to the technical field of computer vision and automatic inspection of power equipment, and in particular to an infrared image insulator rotation detection method based on IoU prediction. Background Technology

[0002] Insulators play a vital role in power transmission lines, providing both support and insulation, and are crucial for ensuring the safety of the power system. Due to their long-term exposure to harsh outdoor environments, they are susceptible to corrosion and overheating defects. If these overheating defects are not detected in time, they can easily lead to further insulator burnout, seriously affecting the safe operation of the power system. Therefore, regular inspections of transmission line insulators are an important task for the power sector.

[0003] In recent years, with the continuous development of drone technology and deep learning, the power sector has begun to use drones equipped with infrared devices to capture infrared images of insulators and combine this with deep learning-based target detection methods to build intelligent inspection solutions for automatic insulator detection, thereby reducing the time and manpower costs of power inspections. However, insulator overheating detection requires precise positioning of the insulator's core rod. Currently used horizontal detectors have low positioning accuracy when detecting tilted objects like insulators with large aspect ratios, and are prone to missed detections, making them unsuitable for insulator overheating defect detection tasks. Although some general-purpose rotation detectors can solve these problems to some extent by generating rotating detection boxes, the changing drone aerial angle results in a large number of closely spaced insulator strings in the image. Intra-class coupling and mutual interference between features can easily lead to prediction deviations in rotating detection boxes, or even missed or false detections. Current rotation detectors use classification scores as the criterion for selecting the optimal prediction box; however, the prediction box with the highest classification score is not necessarily the most accurately positioned prediction box, and there is no necessary correlation between the two. This screening method leads to deviations in the rotation angle prediction of the general rotary detector when faced with a large number of closely spaced insulators, misdetecting two closely spaced insulator strings as the same insulator, and causing false detections in the background between two insulator strings. This seriously affects the positioning accuracy of the insulator core rod, thereby affecting the accuracy of the insulator overheating defect diagnosis.

[0004] In summary, although some deep learning-based insulator detection methods and general rotation detectors exist, their localization accuracy still needs improvement. The traditional method of using classification scores as the basis for determining the optimal bounding box still suffers from localization errors, missed detections, and false detections due to intra-class coupling and feature interference when dealing with a large number of closely spaced insulators. Therefore, a novel infrared image insulator rotation detection method based on IoU prediction is urgently needed to address these issues. Summary of the Invention

[0005] The purpose of this invention is to address the problems of positioning deviation, missed detections, and false detections caused by intra-class coupling and feature interference when traditional rotating detectors use classification scores as the basis for determining the optimal prediction box, especially when dealing with a large number of closely spaced insulators. This invention proposes an infrared image-based insulator rotation detection method based on IoU prediction, which can effectively improve the positioning accuracy of insulators and reduce missed and false detections of closely spaced insulators. This allows for more precise inspection of insulator overheating defects when used in conjunction with UAVs.

[0006] To achieve the above objectives, the technical solution provided by this invention is: an infrared image insulator rotation detection method based on IoU prediction, comprising the following steps:

[0007] 1) Select UAV aerial infrared images, use the rotating box annotation tool to annotate the smallest circumscribed rotating rectangle of the insulator in the image, called the rotating box, convert the coordinates of the four vertices of the rotating box, write them into a txt file, and construct the DOTA format insulator infrared image dataset.

[0008] 2) Perform data augmentation on the insulator infrared image dataset to obtain the augmented insulator infrared image dataset;

[0009] 3) Construct an insulator rotation detector. This detector uses ResNet50 as the backbone extraction network and FPN as the feature fusion network. Based on the RetinaNet detection head, the number of output channels of the last convolution in the regression branch is modified to 5A, where A is the number of anchors generated for each pixel. In addition, an IoU and rotation angle score prediction branch is added. This branch shares the first four 3*3 convolutions with the regression branch. Finally, a 3*3 convolution with 2A output channels is used to predict the IoU and rotation angle scores of the rotating box respectively. After the 3*3 convolution, a Sigmoid module is added to limit the IoU and rotation angle scores.

[0010] 4) Design a training strategy for the insulator rotation detector, modify the loss function during training, use Focal Loss for classification loss and Rotated IoU Loss for regression loss; design loss functions for IoU and rotation angle score prediction branches, using the sum of the absolute differences between the predicted IoU value and the actual rotation IoU between the predicted box and the ground truth box as the IoU prediction loss; use the sum of the absolute differences between the rotation angle score and the actual predicted deviation of the rotation angle as the rotation angle score loss;

[0011] 5) Input the insulator infrared image dataset into the insulator rotation detector for training. By minimizing the joint loss composed of classification, regression, IoU prediction and rotation angle score loss, train the weights of the insulator rotation detector, save the weights with the best performance, and obtain the trained insulator rotation detector.

[0012] 6) Using the trained insulator rotation detector to predict images, the non-maximum suppression (NMS) algorithm is improved to obtain the θ-IoU-NMS algorithm. This algorithm changes the criterion for determining the optimal rotation box in non-maximum suppression from the original classification score to a comprehensive score after weighted summation of IoU prediction, rotation angle score, and classification score. The comprehensive score is used to better evaluate the quality of the rotation box and retain the rotation box with more accurate positioning.

[0013] 7) The θ-IoU-NMS algorithm is used to suppress the rotating frames detected by the insulator rotation detector, remove the low-quality rotating frames that are repeatedly detected for the same insulator, and obtain the final rotating frames with more accurate positioning. The insulator detection results in the infrared image are then output.

[0014] Further, in step 1), the smallest circumscribed rotating rectangle of the insulator in the image is labeled using the roLabelImg annotation tool. This is called the rotating frame. The coordinates of the center point, width, height, and rotation angle of the rotating frame are obtained. These coordinates are then converted into the coordinates of the four vertices of the rotating frame and written to a txt file to construct a DOTA format insulator infrared image dataset.

[0015] Furthermore, in step 2), data augmentation employs random rotation, random scaling, random brightness adjustment, random removal, and Mosaic enhancement. Since infrared images contain temperature information, the temperature information matrix needs to be transformed in the same way when performing image transformation to obtain the data-enhanced insulator infrared image dataset.

[0016] Furthermore, in step 3), ResNet50 is used as the backbone extraction network of the detector to extract features from the infrared image; FPN is used as the feature fusion network to further fuse the features extracted by the backbone extraction network, thereby improving the detector's multi-scale detection capability; the detection head adopts a structure similar to the RetinaNet detection head, divided into classification and regression branches. Each branch uses four 3*3 convolutions to further fuse features. The last 3*3 convolution of the regression branch is modified to have 5A output channels, which are used to predict the five positional parameters x, y, w, h, and θ of the bounding box, where A represents each image. The number of anchors generated by the pixel is defined as follows: x represents the x-coordinate of the center point of the rotated bounding box, y represents the y-coordinate of the center point of the rotated bounding box, w represents the width of the rotated bounding box, h represents the height of the rotated bounding box, and θ represents the rotation angle of the rotated bounding box. An IoU and rotation angle score prediction branch is added, consisting of an IoU prediction branch and a rotation angle score prediction branch, sharing the weights of the first four 3x3 convolutions with the regression branch. Finally, a 3x3 convolution with 2A output channels is added to predict the IoU and rotation angle score of the rotated bounding box. A Sigmoid module is added after the 3x3 convolution to limit the output of the 3x3 convolution, and its calculation formula is as follows:

[0017]

[0018] In the formula, t represents the input value, and S(t) represents the output value after Sigmoid processing;

[0019] By using the Sigmoid module to restrict the output of the 3*3 convolution to the range of [0,1], the final IoU of the rotated box and the rotation angle score prediction results are obtained.

[0020] Furthermore, in step 4), Focal Loss is used for classification loss, and Rotated IoU Loss is used for regression loss. The formula for calculating Rotated IoU Loss is as follows:

[0021]

[0022] In the formula, Rotated IoU represents the rotation IoU between the predicted bounding box and the ground truth bounding box, L RIoU This represents Rotated IoULoss, where m indicates that the overlapping region is divided into m small triangles using the lines connecting the vertices of the overlapping region of the two rotated frames, k represents the current small triangle, and Area... tri Area represents the area of ​​the small triangle. p Area represents the area of ​​the prediction box. g Represents the area of ​​the actual bounding box;

[0023] The loss function for the IoU and rotation angle score prediction branches is designed. The formula for calculating the IoU prediction loss is as follows:

[0024] L IoU-predict =∑|P IoU -RotatedIoU| (3)

[0025] Where, L IoU-predict P represents the predicted loss based on IoU. IoU This represents the IoU value predicted by the IoU prediction branch;

[0026] Using the sum of the absolute differences between the IoU value predicted by the IoU prediction branch and the actual calculated rotation IoU between the predicted bounding box and the ground truth bounding box as the IoU prediction loss, the gap between the IoU prediction value and the actual IoU value can be reduced by minimizing the loss function, thereby improving the accuracy of IoU prediction.

[0027] The formula for calculating the rotation angle score loss is as follows:

[0028]

[0029] In the formula, Score θ The score represents the rotation angle, θ. p θ represents the rotation angle of the prediction box. g L represents the rotation angle of the true bounding box. θ-predict P represents the score loss due to rotation angle. Scoreθ This represents the rotation angle score predicted by the rotation angle score prediction branch;

[0030] Using rotation angle score to predict branch prediction rotation angle score P Scoreθ The rotation angle score is calculated by comparing the rotation angles of the predicted bounding box and the ground truth bounding box. θ The sum of the absolute differences is used as the rotation angle score loss. By minimizing the loss function, the difference between the predicted rotation angle score and the actual rotation angle score can be reduced, thereby improving the prediction accuracy of the rotation angle score.

[0031] Furthermore, in step 5), the insulator infrared image dataset is fed into the insulator rotation detector for training. The joint loss function formula for training is as follows:

[0032] L train =L cls +L reg +L IoU-predict +L θ-predict (5)

[0033] Where, L train L represents the joint loss of training. cls L represents the classification loss.reg Indicates regression loss;

[0034] The joint loss, consisting of classification, regression, IoU prediction, and rotation angle score loss, is used as the training loss. By minimizing the joint loss during training, the weights of each part of the detector are adjusted, and the weights with the highest detection accuracy on the validation set are stored to obtain the trained insulator rotation detector.

[0035] Furthermore, in step 6), the Non-Maximum Suppression (NMS) algorithm is improved to obtain the θ-IoU-NMS algorithm. This algorithm modifies the NMS algorithm from using classification score as the criterion for determining the optimal rotation box to using a comprehensive score obtained by weighted summation of IoU prediction, rotation angle score, and classification score. The calculation formula is as follows:

[0036]

[0037] In the formula, Score c Score represents the overall score. cls Indicates the category score;

[0038] Using the comprehensive score as the criterion for determining the optimal bounding box allows for a comprehensive consideration of the bounding box's location accuracy (IoU), rotation angle prediction accuracy, and classification score. cls By retaining the rotating frame with higher positioning accuracy and more accurate rotation angle prediction, the insulator positioning accuracy of the detector is significantly improved.

[0039] To address the issue of false detections caused by feature interference in the background area between two closely spaced insulator strings, a method is used to predict the IoU (Intersection over Union) value P of the bounding boxes in the same image. IoU Perform clustering and remove P IoU Outliers that are lower than other predicted boxes are removed, thus eliminating false positives.

[0040] Furthermore, in step 7), the θ-IoU-NMS algorithm is used to suppress the rotating frames detected by the detector, retaining the rotating frames with high positioning accuracy and rotation angle prediction accuracy, removing low-quality rotating frames that are repeatedly detected on the same insulator, and outputting the detection results of the insulator in the final infrared image.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] 1. This invention addresses the characteristic of numerous closely spaced insulators in images captured by drones by designing an insulator rotation detection method based on IoU prediction. Compared to the commonly used NMS algorithm based on classification scores, this method can better filter rotating boxes by relying on the predicted IoU value when facing intra-class coupling caused by closely spaced insulators, thereby avoiding missed detections caused by detecting two closely spaced insulators as one.

[0043] 2. This invention comprehensively considers the IoU of the rotating frame, the prediction accuracy of the rotation angle, and the classification score. Compared with commonly used methods, when facing inclined insulators that are close to each other, it can retain the rotating frame with the most accurate positioning accuracy and rotation angle prediction, and avoid the deviation of the positioning and rotation angle prediction of the rotating frame due to mutual interference of features, thereby improving the positioning accuracy of the insulator core rod.

[0044] 3. This invention uses the predicted IoU value to determine whether the rotating frame is falsely detected. Compared with the currently commonly used methods, it can effectively avoid the problem of false detection caused by feature interference in the background position between two closely spaced insulator strings. Attached Figure Description

[0045] Figure 1 This is a flowchart of the method of the present invention.

[0046] Figure 2 The figure shows a comparison of the detection results of a universal rotating detector and the insulator rotating detector of the present invention in one embodiment; (a) in the figure shows the detection result of the universal rotating detector, and (b) shows the detection result of the insulator rotating detector of the present invention.

[0047] Figure 3 The figure shows a comparison of the detection results of a universal rotating detector and the insulator rotating detector of the present invention in one embodiment; (a) in the figure shows the detection result of the universal rotating detector, and (b) shows the detection result of the insulator rotating detector of the present invention. Detailed Implementation

[0048] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0049] like Figure 1 As shown, this embodiment provides a method for detecting the rotation of an infrared insulator based on IoU prediction, which includes the following steps:

[0050] 1) Organize the UAV aerial infrared images and select those containing insulators. Use the roLabelImg annotation tool to annotate the smallest bounding rectangle of the insulators in the images, called the rotation box. Obtain the five positional parameters of the rotation box (x, y, w, h, θ), where x represents the x-coordinate of the center point of the rotation box, y represents the y-coordinate of the center point of the rotation box, w represents the width of the rotation box, h represents the height of the rotation box, and θ represents the rotation angle of the rotation box. Calculate the coordinates of the four vertices of the rotation box (x1, y1, x2, y2, x3, y3, x4, y4) and write them to a txt file to construct a DOTA format insulator infrared image dataset.

[0051] 2) Data augmentation is performed on the labeled insulator infrared image dataset. Augmentation methods include random rotation, random scaling, random brightness adjustment, random removal, and Mosaic enhancement. This improves the detector's accuracy for insulators of different sizes and rotation angles, and enhances its generalization ability in detection scenarios with occlusion and complex backgrounds. Since infrared images contain temperature information, the temperature information matrix of the infrared image needs to be transformed in the same way during rotation and scaling operations to ensure that the transformed temperature information matrix maintains the correspondence with the pixels of the infrared image.

[0052] 3) Construct an insulator rotation detector, which consists of three parts: a backbone extraction network, a feature fusion network, and a detection head. ResNet50 is used as the backbone extraction network to extract insulator-related features from infrared images. FPN is used as the feature fusion network to further fuse the features extracted by the backbone extraction network, fusing low-level localization information with high-level semantic information to improve the detector's ability to detect insulators at multiple scales. The detection head adopts a structure similar to the RetinaNet detection head, decoupling the classification and regression branches. The classification branch first further fuses features through four 3x3 convolutions to improve the detector's fitting ability, and then predicts the classification score through a 3x3 convolution with A output channels, where A represents the generation of A anchors for each pixel on the feature map, used to predict the insulator. The regression branch first further fuses features through four 3x3 convolutions, and then modifies the output channel number of its last 3x3 convolution from 4A to 5A to predict the five positional parameters (x, y, w, h, θ) of the rotation box. Building upon this foundation, a new branch for IoU and rotation angle score prediction is added. This branch consists of an IoU prediction branch and a rotation angle score prediction branch, sharing the weight parameters of the first four convolutions with the regression branch. Since IoU and rotation angle score prediction are closely related to the localization of the rotation box, this method of sharing convolution weights not only effectively reduces the number of detector parameters but also helps the IoU and rotation angle score prediction branches converge, improving the training speed of the detector. Finally, a 3x3 convolution with 2A output channels is added to this branch to predict the IoU and rotation angle score of the rotation box. Because the IoU and rotation angle score of the rotation box range between [0,1], a Sigmoid module needs to be added after the 3x3 convolution to limit the output of the 3x3 convolution, thereby accelerating the convergence speed. The calculation formula is as follows:

[0053]

[0054] In the formula, t represents the input value, and S(t) represents the output value after Sigmoid processing;

[0055] The output of the 3*3 convolution is passed through the Sigmoid module to obtain the IoU of the rotated bounding box in the range [0,1] and the prediction results of the rotation angle score.

[0056] 4) Design a training strategy for the insulator rotation detector and modify the loss function during training. Since the number of insulators in the drone aerial images is relatively small, most of the anchors generated by the detector are negative samples consisting only of background pixels, leading to an imbalance of positive and negative samples. Therefore, Focal Loss is used for classification loss to address the imbalance caused by the limited number of insulators. Rotated IoU Loss is used for regression loss. Compared to IoU calculation for horizontal bounding boxes, Rotated IoU calculation for rotated bounding boxes requires dividing the overlapping region into multiple small triangles. The calculation formula is as follows:

[0057]

[0058] In the formula, Rotated IoU represents the rotation IoU between the predicted bounding box and the ground truth bounding box, L RIoU This represents Rotated IoULoss, where m indicates that the overlapping region is divided into m small triangles using the lines connecting the vertices of the overlapping region of the two rotated frames, k represents the current small triangle, and Area... tri Area represents the area of ​​the small triangle. p Area represents the area of ​​the prediction box. g Represents the area of ​​the actual bounding box;

[0059] The loss function for the IoU and rotation angle score prediction branches is designed as follows: Since the loss function of the regression branch has already calculated the rotation IoU between the predicted box and the ground truth box, the IoU prediction branch uses the sum of the absolute differences between the predicted IoU value and the actual calculated rotation IoU between the predicted box and the ground truth box as the loss function. The calculation formula is as follows:

[0060] L IoU-predict =∑|P IoU -RotatedIoU| (3)

[0061] Where, L IoU-predict P represents the predicted loss based on IoU. IoU This represents the IoU value predicted by the IoU prediction branch;

[0062] During training, minimize L IoU-predict This can reduce the gap between the IoU value predicted by the IoU prediction branch and the rotation IoU between the actual calculated predicted box and the ground truth box, thus enabling the finally trained detector to predict the IoU value between the generated predicted box and the ground truth box more accurately.

[0063] The rotation angle score prediction branch uses the sum of the absolute differences between the predicted rotation angle scores and the actual calculated rotation angle scores between the predicted bounding box and the ground truth bounding box as the loss function. The calculation formula is as follows:

[0064]

[0065] In the formula, Score θ The score represents the rotation angle, θ. p θ represents the rotation angle of the prediction box. g L represents the rotation angle of the true bounding box. θ-predict P represents the score loss due to rotation angle. Scoreθ This represents the rotation angle score predicted by the rotation angle score prediction branch;

[0066] During training, minimize L θ-predict This can reduce the gap between the predicted rotation angle score and the actual calculated rotation angle score, thereby improving the prediction accuracy of the rotation angle score.

[0067] 5) The insulator infrared image dataset is fed into the insulator rotation detector for training. The training uses a joint loss function, the calculation formula of which is as follows:

[0068] L train =L cls +L reg +L IoU-predict +L θ-predict (5)

[0069] In the formula L train L represents the joint loss of training. cls L represents the classification loss. reg Indicates regression loss;

[0070] By minimizing the joint loss L train The weights of each convolutional and pooling layer in the detector can be adjusted to make the final predicted classification score, the position of the rotated box, the predicted IoU value of the rotated box, and the predicted rotation angle score continuously approach the true values. The weights with the highest detection accuracy on the validation set are stored to obtain the trained insulator rotation detector.

[0071] 6) Using the trained insulator rotation detector to predict images, the Non-Maximum Suppression (NMS) algorithm is improved to obtain the θ-IoU-NMS algorithm. Commonly used NMS algorithms use classification scores as the criterion for determining the optimal bounding box. However, the highest classification score for a bounding box does not necessarily indicate the most accurate localization; there is no direct correlation between the two. The IoU value of the bounding box represents the localization difference between it and the ground truth bounding box, effectively reflecting the localization accuracy of the bounding box. Therefore, the criterion for determining the optimal bounding box is modified to a weighted sum of IoU prediction, rotation angle score, and classification score. The calculation formula is as follows:

[0072]

[0073] In the formula, Score c Score represents the overall score. cls Indicates the category score;

[0074] Using a comprehensive score as the criterion for determining the optimal rotation box, this method comprehensively considers the positioning accuracy, rotation angle prediction accuracy, and classification score of the rotation box. It retains rotation boxes with higher positioning accuracy and more precise rotation angle prediction, thus avoiding problems such as rotation angle prediction deviations, missed detections, and low positioning accuracy caused by close insulator distances, intra-class coupling, and feature interference. Figure 2 As shown, the insulator rotation detector of the present invention can take into account the IoU prediction value P of the rotating frame when detecting two strings of tilted insulators that are close to each other. IoU Compared to general rotating detectors that only consider classification scores, cls The test results show that the rotating frame positioning of the present invention is more accurate, which is beneficial for mandrel positioning and insulator overheating defect diagnosis. Figure 3 As shown, when detecting two closely spaced insulator strings, a conventional rotating detector easily mistakes them for the same string, leading to missed detections. The insulator rotating detector of this invention can predict the IoU value P using the rotating frame. IoU A more precise rotating frame is used to filter out samples, thus avoiding missed detections.

[0075] To address the issue of false detections at the background location between two closely spaced insulator strings due to feature interference, the IoU prediction value of the rotated bounding box is used. By clustering the IoU prediction values ​​of the predicted bounding boxes in the same image, outliers with significantly lower IoU prediction values ​​than other predicted bounding boxes are removed, thereby avoiding false detections at the background location between two closely spaced insulator strings.

[0076] 7) The θ-IoU-NMS algorithm is used to filter the rotating frames predicted by the detector, removing low-quality rotating frames with low positioning accuracy, deviations in rotation angle prediction, and false detections, retaining rotating frames with more accurate positioning, and outputting the insulator detection results in the infrared image.

[0077] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A method for detecting insulator rotation based on infrared image prediction using IoU, characterized in that, Includes the following steps: 1) Select UAV aerial infrared images, use the rotating box annotation tool to annotate the smallest circumscribed rotating rectangle of the insulator in the image, called the rotating box, convert the coordinates of the four vertices of the rotating box, write them into a txt file, and construct the DOTA format insulator infrared image dataset. 2) Perform data augmentation on the insulator infrared image dataset to obtain the augmented insulator infrared image dataset; 3) Construct an insulator rotation detector. This detector uses ResNet50 as the backbone extraction network and FPN as the feature fusion network. Based on the RetinaNet detection head, the number of output channels of the last convolution in the regression branch is modified to 5A, where A is the number of anchors generated for each pixel. In addition, an IoU and rotation angle score prediction branch is added. This branch shares the first four 3*3 convolutions with the regression branch. Finally, a 3*3 convolution with 2A output channels is used to predict the IoU and rotation angle scores of the rotating box respectively. After the 3*3 convolution, a Sigmoid module is added to limit the IoU and rotation angle scores. 4) Design a training strategy for the insulator rotation detector, modify the loss function during training, use FocalLoss for classification loss and Rotated IoU Loss for regression loss; design loss functions for IoU and rotation angle score prediction branches, using the sum of the absolute differences between the predicted IoU value and the actual rotation IoU between the predicted box and the ground truth box as the IoU prediction loss; use the sum of the absolute differences between the rotation angle score and the actual predicted deviation of the rotation angle as the rotation angle score loss; 5) Input the insulator infrared image dataset into the insulator rotation detector for training. By minimizing the joint loss composed of classification, regression, IoU prediction and rotation angle score loss, train the weights of the insulator rotation detector, save the weights with the best performance, and obtain the trained insulator rotation detector. 6) Using the trained insulator rotation detector to predict images, the non-maximum suppression (NMS) algorithm is improved to obtain the θ-IoU-NMS algorithm. This algorithm changes the criterion for determining the optimal rotation box in non-maximum suppression from the original classification score to a comprehensive score after weighted summation of IoU prediction, rotation angle score, and classification score. The comprehensive score is used to better evaluate the quality of the rotation box and retain the rotation box with more accurate positioning. 7) The θ-IoU-NMS algorithm is used to suppress the rotating frames detected by the insulator rotation detector, remove the low-quality rotating frames that are repeatedly detected for the same insulator, and obtain the final rotating frames with more accurate positioning. The insulator detection results in the infrared image are then output.

2. The infrared image insulator rotation detection method based on IoU prediction according to claim 1, characterized in that, In step 1), the smallest circumscribed rotating rectangle of the insulator in the image is labeled using the roLabelImg annotation tool. This is called the rotating frame. The coordinates of the center point, width, height, and rotation angle of the rotating frame are obtained. These coordinates are then converted into the coordinates of the four vertices of the rotating frame and written to a txt file to construct a DOTA format insulator infrared image dataset.

3. The infrared image insulator rotation detection method based on IoU prediction according to claim 2, characterized in that, In step 2), data augmentation employs random rotation, random scaling, random brightness adjustment, random removal, and Mosaic enhancement. Since infrared images contain temperature information, the temperature information matrix needs to be transformed in the same way when performing image transformation to obtain the data-enhanced insulator infrared image dataset.

4. The infrared image insulator rotation detection method based on IoU prediction according to claim 3, characterized in that, In step 3), ResNet50 is used as the backbone extraction network of the detector to extract features from the infrared image; FPN is used as the feature fusion network to re-fuse the features extracted by the backbone extraction network, improving the detector's multi-scale detection capability. The detection head adopts a structure similar to the RetinaNet detection head, divided into classification and regression branches. Each branch uses four 3*3 convolutions to further fuse features. The last 3*3 convolution in the regression branch is modified to have 5A output channels, used to predict the five positional parameters x, y, w, h, and θ of the rotation box, where A represents the number of anchors generated per pixel, and x represents the center position of the rotation box. The x-coordinate of the center point is given by y, the y-coordinate of the center point of the rotated bounding box is given by w, the width of the rotated bounding box is given by h, and the rotation angle of the rotated bounding box is given by θ. An IoU and rotation angle score prediction branch is added, consisting of an IoU prediction branch and a rotation angle score prediction branch, sharing the weights of the first four 3x3 convolutions with the regression branch. Finally, a 3x3 convolution with 2A output channels is added to predict the IoU and rotation angle score of the rotated bounding box. A Sigmoid module is added after the 3x3 convolution to limit the output of the 3x3 convolution, and its calculation formula is as follows: In the formula, t represents the input value, and S(t) represents the output value after Sigmoid processing; By using the Sigmoid module to restrict the output of the 3*3 convolution to the range of [0,1], the final IoU of the rotated box and the rotation angle score prediction results are obtained.

5. The infrared image insulator rotation detection method based on IoU prediction according to claim 4, characterized in that, In step 4), Focal Loss is used for classification loss, and Rotated IoU Loss is used for regression loss. The formula for calculating Rotated IoU Loss is as follows: In the formula, Rotated IoU represents the rotation IoU between the predicted bounding box and the ground truth bounding box, L RIoU This represents Rotated IoU Loss, where m indicates that the overlapping region is divided into m small triangles using the lines connecting the vertices of the overlapping region of the two rotated boxes, k represents the current small triangle, and Area... tri Area represents the area of ​​the small triangle. p Area represents the area of ​​the prediction box. g Represents the area of ​​the actual bounding box; The loss function for the IoU and rotation angle score prediction branches is designed. The formula for calculating the IoU prediction loss is as follows: L IoU-predict =∑|P IoU -Rotated IoU| (3) In the formula, L IoU-predict P represents the predicted loss based on IoU. IoU This represents the IoU value predicted by the IoU prediction branch; Using the sum of the absolute differences between the IoU value predicted by the IoU prediction branch and the actual calculated rotation IoU between the predicted bounding box and the ground truth bounding box as the IoU prediction loss, the gap between the IoU prediction value and the actual IoU value can be reduced by minimizing the loss function, thereby improving the accuracy of IoU prediction. The formula for calculating the rotation angle score loss is as follows: In the formula, Score θ The score represents the rotation angle, θ. p θ represents the rotation angle of the prediction box. g L represents the rotation angle of the true bounding box. θ-predict P represents the score loss due to rotation angle. Scoreθ This represents the rotation angle score predicted by the rotation angle score prediction branch; Using rotation angle score to predict branch prediction rotation angle score P Scoreθ The rotation angle score is calculated by comparing the rotation angles of the predicted bounding box and the ground truth bounding box. θ The sum of the absolute differences is used as the rotation angle score loss. By minimizing the loss function, the difference between the predicted rotation angle score and the actual rotation angle score can be reduced, thereby improving the prediction accuracy of the rotation angle score.

6. The infrared image insulator rotation detection method based on IoU prediction according to claim 5, characterized in that, In step 5), the insulator infrared image dataset is fed into the insulator rotation detector for training. The joint loss function formula for training is as follows: L train L cls +L reg +L IoU-predict +L θ-predict (5) In the formula, L train L represents the joint loss of training. cls L represents the classification loss. reg Indicates regression loss; The joint loss, consisting of classification, regression, IoU prediction, and rotation angle score loss, is used as the training loss. By minimizing the joint loss during training, the weights of each part of the detector are adjusted, and the weights with the highest detection accuracy on the validation set are stored to obtain the trained insulator rotation detector.

7. The infrared image insulator rotation detection method based on IoU prediction according to claim 6, characterized in that, In step 6), the Non-Maximum Suppression (NMS) algorithm is improved to obtain the θ-IoU-NMS algorithm. This algorithm modifies the NMS algorithm, which used classification score as the criterion for determining the optimal rotation box, to use a comprehensive score obtained by weighted summation of IoU prediction, rotation angle score, and classification score. The calculation formula is as follows: In the formula, Score c Score represents the overall score. cls Indicates the category score; Using the comprehensive score as the criterion for determining the optimal bounding box allows for a comprehensive consideration of the bounding box's location accuracy (IoU), rotation angle prediction accuracy, and classification score. cls By retaining the rotating frame with higher positioning accuracy and more accurate rotation angle prediction, the insulator positioning accuracy of the detector is significantly improved. To address the issue of false detections caused by feature interference in the background area between two closely spaced insulator strings, a method is used to predict the IoU (Intersection over Union) value P of the bounding boxes in the same image. IoU Perform clustering and remove P IoU Outliers that are lower than other predicted boxes are removed, thus eliminating false positives.

8. The infrared image insulator rotation detection method based on IoU prediction according to claim 7, characterized in that, In step 7), the θ-IoU-NMS algorithm is used to suppress the rotating frames detected by the detector, retaining the rotating frames with high positioning accuracy and rotation angle prediction accuracy, removing low-quality rotating frames that are repeatedly detected on the same insulator, and outputting the detection results of the insulator in the final infrared image.

Images