Insulator Fault Detection Method and Equipment Based on Deep Learning and Adaptive Image Processing

Abstract

The application discloses an insulator fault detection method and device based on deep learning and adaptive image processing, first constructs an insulator dataset, trains a DetectoRS target detection network and a Bi SeNet semantic segmentation network on a training set; inputs a test set image into the DetectoRS network to generate an insulator detection frame; extracts each insulator according to the detection frame and inputs the extraction area into the Bi SeNet network to generate an insulator mask; carries out post-processing on the image in a result image of semantic segmentation to obtain an outer contour of the insulator; calculates a minimum circumscribed rotating rectangle of the insulator outer contour and rotates the image to be horizontal according to a rotating angle thereof; constructs a quantitative curve of an insulator shape structure according to the horizontal insulator image; finally, smooths the curve, constructs an adaptive threshold required for fault judgment, judges an existing state and a position of the insulator fault and effectively improves the precision and the generality of fault detection.

Description

Technical Field

[0001] This invention belongs to the technical field of machine vision and power line inspection, and mainly relates to an insulator fault detection method and equipment based on deep learning and adaptive image processing. Background Technology

[0002] Transmission lines are a crucial component of the power grid, widely distributed and situated in complex environments. The operational status of transmission line equipment directly impacts the reliability of the power supply. Insulators, as vital components of transmission lines, are installed between conductors at different potentials or between conductors and grounding structures. They play a vital role in supporting the conductors and providing electrical insulation, bearing significant mechanical tension and extremely high voltage. Over long-term operation, insulator failures can occur, causing instability in the operation of high-voltage transmission lines, and in severe cases, even leading to large-scale power outages. Therefore, to ensure the normal operation of transmission lines, regular safety inspections of the insulators are necessary to eliminate potential faults.

[0003] In recent years, my country's power industry has widely adopted drone technology for power line inspection. Drones can quickly collect large amounts of insulator data through high-definition cameras, helping to identify problems and potential hazards in power equipment and improve the reliability and safety of the power system. Therefore, researching how to achieve efficient and accurate insulator fault detection using drone images is of significant practical importance.

[0004] Currently, methods for insulator fault detection in images can be mainly divided into two types: The first method uses a single target detection network to directly and simultaneously detect the insulator string and the fault on it. Commonly used detection networks include the YOLO series, SSD series, and RCNN series. The advantage of this method is its simple deployment and strong versatility. However, since insulator faults are usually small, and drones cannot get too close to the transmission line for shooting, the fault generally occupies a small proportion in the image, resulting in limited detection accuracy. The second method first uses a target detection network to locate the insulator, then crops out the corresponding area, and then combines it with classical image processing algorithms to complete the fault detection. The advantage of this method is that locating the insulator first allows the algorithm to focus more on the fault area of ​​interest, which helps improve detection accuracy. However, since the second step of fault detection relies entirely on classical image processing algorithms, the versatility of detection is poor, and the algorithm may fail in complex and diverse scenarios. In summary, in practical applications of drone power grid inspection, neither of the two commonly used methods can ideally complete the task of insulator fault detection. Summary of the Invention

[0005] This invention addresses the limitations of existing insulator fault detection methods by providing a method and device based on deep learning and adaptive image processing. First, insulator images are acquired to construct an insulator dataset. The DetectoRS object detection network and the BiSeNet semantic segmentation network are trained on the training set. Test set images are input into the DetectoRS network to generate insulator detection boxes. Each insulator is extracted based on the detection boxes, and the extracted regions are input into the BiSeNet network to generate insulator masks. The resulting image from semantic segmentation is post-processed to obtain the insulator's outer contour. The minimum bounding rectangle of the insulator's outer contour is calculated, and the image is rotated horizontally based on its rotation angle. A quantization curve of the insulator's shape structure is constructed based on the horizontal insulator image. Finally, the curve is smoothed, and an adaptive threshold is constructed to determine the presence and location of the insulator fault. This invention improves upon the method of locating the insulator before detecting the fault, effectively enhancing the accuracy and versatility of the fault detection algorithm without significantly increasing computational load.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is: an insulator fault detection method based on deep learning and adaptive image processing, comprising the following steps:

[0007] S1. Dataset Construction: Collect insulator image samples, divide them into training and test sets, and annotate the images to construct an insulator dataset;

[0008] S2. Network Model Construction: The DetectoRS object detection network and the BiSeNet semantic segmentation network are constructed. The DetectoRS object detection network is a convolutional neural network-based object detection method that combines a recursive feature pyramid (RFP) and a switchable dilated convolution (SAC) module. The RFP module enhances the network's ability to capture fine-grained and contextual information by recursively fusing multi-scale features. The SAC module improves the network's ability to perceive multi-scale objects by dynamically switching the dilated convolution kernel size. The DetectoRS object detection network has good detection accuracy and robustness, and is suitable for detecting insulators in complex scenes. The BiSeNet semantic segmentation network is a lightweight semantic segmentation method based on convolutional neural networks, mainly including two branches: a spatial path and a contextual path. The spatial path preserves the spatial dimensions of the input image and encodes spatial information, while the contextual path utilizes a lightweight downsampling model and global average pooling to provide a large receptive field. The SeNet semantic segmentation network has good inference accuracy and speed, and is suitable for quickly segmenting insulators in complex scenes. The training set divided in step S1 is input into the network model for training to obtain the trained DetectoRS object detection network and Bi SeNet semantic segmentation network model.

[0009] S3. Insulator Category and Location Information Acquisition: The test set images divided in step S1 are input into the DetectoRS target detection network trained in step S2. During detection inference, firstly, the image passes through the network's feature extraction stage. This stage extracts multi-level features of the image through convolutional layers. During feature extraction, the switchable dilated convolution (SAC) module dynamically switches between dilated convolution kernels of different scales to enhance the network's ability to perceive insulators of different sizes. Then, the image after feature extraction is input into the recursive feature pyramid (RFP) module. This module further enhances the capture of fine-grained and contextual information by recursively fusing feature maps of different scales. Finally, the network generates candidate boxes based on feature maps of different scales and classifies each detected target category based on the position and feature information of the candidate boxes. Simultaneously, the network predicts the position of the target through a regression task, thereby generating a detection box for each insulator and realizing the detection of insulators in the image.

[0010] S4. Insulator Mask Information Acquisition: Based on the insulator detection boxes obtained in step S3, each insulator is extracted from the entire image, and the extracted regions are input into the BiSeNet semantic segmentation network. During segmentation inference, firstly, the extracted region images undergo convolution, batch normalization, and ReLU activation processing in the spatial path branch to preserve the original spatial dimensions and encode spatial information. Simultaneously, the extracted region images obtain a large receptive field in the context path branch through a lightweight downsampling model and global average pooling. Then, pixel-level classification is performed on the fused feature map of the spatial path branch and context path branch results to output the insulator mask image, thereby achieving the segmentation of insulators in the extracted regions.

[0011] S5. Image post-processing: Post-process the insulator image obtained in step S4 to determine the outer contour of the insulator. The post-processing includes, but is not limited to, image grayscale conversion, image binarization, morphological closing operation, edge detection, and contour finding.

[0012] S6. Generation of circumscribed rotating rectangle and affine transformation: Calculate the minimum circumscribed rotating rectangle of the insulator's outer contour, and apply an affine transformation to the insulator image according to its rotation angle to rotate it to the horizontal direction;

[0013] S7. Insulator structure quantification: Divide the horizontal insulator image obtained in step S6 into upper and lower parts. In each part of the image, use the horizontal coordinate of the image as the abscissa and the sum of pixels in each column of the image as the ordinate to draw the "column pixel and structure curve" of the image to quantify the shape and structure of the insulator.

[0014] S8. Adaptive distance threshold construction: The "column pixel and structure curve" obtained in step S7 is smoothed by using the sliding window averaging method. Each peak value in the curve is found, and the horizontal distance between each pair of consecutive peak values ​​is calculated in order from left to right to construct the adaptive distance threshold.

[0015] S9. Fault Judgment: If the horizontal distance between a pair of consecutive wave peaks is greater than the adaptive distance threshold constructed by the previous pair of consecutive wave peaks, then there is a cap loss fault between the pair of wave peaks, and the current insulator is a faulty insulator, thus realizing fault detection.

[0016] As an improvement of the present invention, in step S1 data annotation, the annotated image labels include at least the detection box labels for object detection and the mask labels for semantic segmentation.

[0017] As an improvement of the present invention, in step S2, the source code of the models reproduced by MMDetection and MMSegmentation is used to construct the DetectoRS object detection and BiSeNet semantic segmentation network models.

[0018] As another improvement of the present invention, in step S3, the test set image is input into the trained DetectoRS object detection network to generate an insulator detection box, specifically as follows:

[0019] bbox = [x, y, w, h, cls]

[0020] Here, bbox represents the entire detection box, (x, y) represents the coordinates of the top-left corner of the detection box, w and h are the width and height of the detection box, respectively, and cls represents the insulator category. By generating the detection boxes, the category of each insulator in the image can be determined and its location can be located. Then, when the extracted region is input into the trained BiSeNet semantic segmentation network for inference testing, an insulator mask will be generated. The segmentation result is reflected as a pixel-level classification map that can distinguish insulators from the background.

[0021] As another improvement of the present invention, in step S5, the image is binarized using the Otsu thresholding method, and in the obtained binary image, image edge filling with a border_size = 5 is performed. Then, morphological closing operation is used to fill small holes in the image. Finally, the Canny edge detection algorithm is used to obtain the basic outline of the insulator, and the findContours function in OpenCV is used to find the clear outer outline of the insulator.

[0022] As another improvement of the present invention, the rotating rectangle in step S6 is specifically:

[0023] rotated_box = [x c ,y c [width,height,θ]

[0024] Where rotated_box represents the entire rotated rectangle, (x c y c ) represents the coordinates of the center point of the rotating rectangle, width and height are the width and height of the rotating rectangle, respectively, and θ is the rotation angle of the rotating rectangle;

[0025] The angle θ is corrected as follows:

[0026]

[0027] Where θ' is the corrected rotation angle;

[0028] The affine transformation matrix M is constructed as follows:

[0029]

[0030] Where M represents the constructed affine transformation matrix, θ needs to be converted to radians, and s is the scaling factor;

[0031] The dimensions of the rotated image are adjusted as follows:

[0032] w new =|w ori ·cosθ|+|h ori ·sinθ|

[0033] h new =|w ori ·sinθ|+|h ori ·cosθ|

[0034] Among them, w new and h new w represents the width and height of the adjusted image, respectively. ori and h ori These represent the width and height of the original image, respectively.

[0035] The adjusted affine transformation matrix M' is as follows:

[0036]

[0037] Where M is the affine transformation matrix, M' is the adjusted affine transformation matrix as a whole, and w new and h new M'(0,2) and M'(1,2) represent the width and height of the adjusted image, respectively. M'(0,2) and M'(1,2) represent the elements at the corresponding index positions in the M' matrix, and M(0,2) and M(1,2) are the elements at the corresponding index positions in the M matrix.

[0038] The adjusted affine transformation matrix M' and the adjusted image size w new h new The image of the insulator is rotated to a horizontal position by inputting into the OpenCV warpAffine function to perform an affine transformation.

[0039] As another improvement of the present invention, the sliding window averaging method in step S8 is specifically as follows:

[0040]

[0041] in, Let y(n), y(n-1), and y(n-(N-1)) represent the sum of pixels in the nth, (n-1), and (n-(N-1))th columns of the image in the sliding window, respectively, and N be the length of the sliding window.

[0042] The peak is identified by finding local maxima of the signal, specifically as follows:

[0043] y(i-1)<y(i)> y(i+1)

[0044] Where y(i-1), y(i), and y(i+1) represent the sum of pixels in the (i-1), i, and (i+1)th consecutive columns of the image, respectively, and y(i) is the peak value found, denoted as yi. peak (i); The peak value is selected as follows:

[0045] y valid_peak (j)=y peak (i), if y peak (i)>α·y max

[0046] Among them, y valid_peak (j) represents the j-th effective peak value selected, y max α is the sum of the maximum column pixels of the image, and α is the peak adjustment coefficient.

[0047] As another improvement of the present invention, the adaptive distance threshold T in step S8 i for:

[0048] T i =β·d i,i+1 =β·(x(i+1)-x(i))

[0049] Among them, T i This represents the adaptive distance threshold constructed from the i-th and i+1-th valid peaks, where β is the horizontal distance adjustment coefficient, β>1, and d i,i+1 Let x(i) be the horizontal distance between the i-th and i+1-th valid peaks, and let x(i) and x(i+1) represent the horizontal coordinates of the image corresponding to the i-th and i+1-th valid peaks, respectively.

[0050] As a further improvement of the present invention, the fault detection in step S9 specifically includes:

[0051]

[0052] Among them, defect i This indicates the fault presence status between the i-th and (i+1)-th valid peaks, where 1 indicates a fault and 0 indicates normal operation. i-1 The adaptive distance threshold is constructed from the (i-1)th and i-th valid peaks.

[0053] To achieve the above objectives, the present invention also adopts the following technical solution: a computer device, comprising:

[0054] Memory, used to store instructions;

[0055] A processor is configured to execute the instructions, causing the computer device to perform the operational steps of any of the above-described insulator fault detection methods based on deep learning and adaptive image processing.

[0056] Compared with the prior art, the present invention has the following advantages:

[0057] (1) Before performing fault detection in the extracted insulator region, the method proposed in this invention proposes to use a lightweight semantic segmentation network BiSeNet to further separate the insulator and the background. Compared with the use of classical image segmentation methods, the use of lightweight deep learning methods can significantly improve the accuracy of insulator segmentation and adaptability to complex background environments without significantly increasing the computational burden.

[0058] (2) In the post-processing of data, the present invention can effectively model the shape and structure of insulators by performing operations such as grayscale conversion, binarization, closing operation, edge detection and contour search on semantic segmentation images, as well as affine transformation of images and drawing the "image horizontal coordinate - column pixel sum" curve, which helps to carry out subsequent fault identification.

[0059] (3) The adaptive distance threshold finally constructed in the method of the present invention can cope with insulator images of different sizes compared with the constant threshold, and has better versatility. Attached Figure Description

[0060] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0061] Figure 2 This is a schematic diagram of the result after target detection in step S3 of Embodiment 1 of the present invention;

[0062] Figure 3 This is a schematic diagram of the semantic segmentation result after step S4 in Embodiment 1 of the present invention;

[0063] Figure 4 This is a schematic diagram of the horizontal insulator image after image post-processing and affine transformation in step S6 of Embodiment 1 of the present invention;

[0064] Figure 5 This is a schematic diagram of the column pixels and structural curves of the insulator image in Embodiment 1 of the present invention;

[0065] Figure 6 This is a schematic diagram of the fault detection results in Embodiment 1 of the present invention. Detailed Implementation

[0066] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0067] Example 1

[0068] A fault detection method for insulators based on deep learning and adaptive image processing, such as Figure 1 As shown, the specific steps are as follows:

[0069] Step S1: Collect insulator image samples in the field using a drone, divide the dataset into training and testing sets, and annotate the images to construct an insulator dataset. The labeled images include bounding box labels for object detection and mask labels for semantic segmentation.

[0070] Step S2: Construct the DetectoRS object detection and BiSeNet semantic segmentation network models. The DetectoRS object detection network is a convolutional neural network-based object detection method that combines a recursive feature pyramid (RFP) and a switchable dilated convolution (SAC) module. The RFP module enhances the network's ability to capture fine-grained and contextual information by recursively fusing multi-scale features. The SAC module improves the network's ability to perceive multi-scale objects by dynamically switching the size of the dilated convolution kernel. The DetectoRS object detection network has good detection accuracy and robustness, and is suitable for detecting insulators in complex scenes. The BiSeNet semantic segmentation network is a lightweight semantic segmentation method based on convolutional neural networks, mainly including two branches: a spatial path and a context path. The spatial path can preserve the spatial size of the input image and encode spatial information, while the context path can provide a large receptive field by utilizing a lightweight downsampling model and global average pooling. The BiSeNet semantic segmentation network has good inference accuracy and speed, and is suitable for quickly segmenting insulators in complex scenes. The network is trained using training set data.

[0071] When constructing the DetectoRS object detection and BiSeNet semantic segmentation network models, the model source code was reproduced using MMDetection and MMSegmentation, respectively.

[0072] Step S3: Input the test set images into the DetectoRS object detection network to generate insulator detection boxes and obtain the category and location information of the insulators. Input the test set images divided in Step S1 into the DetectoRS object detection network trained in Step S2. During detection inference, firstly, the image passes through the network's feature extraction stage. This stage extracts multi-level features of the image through convolutional layers. During feature extraction, the Switchable Dilated Convolution (SAC) module dynamically switches between dilated convolution kernels of different scales to enhance the network's ability to perceive insulators of different sizes. Then, the image after feature extraction is input into the Recursive Feature Pyramid (RFP) module. This module further enhances the capture of fine-grained and contextual information by recursively fusing feature maps of different scales. Finally, the network generates candidate boxes based on feature maps of different scales and classifies each detected target category based on the position and feature information of the candidate boxes. Simultaneously, the network predicts the target position through a regression task, thereby generating detection boxes for each insulator and realizing the detection of insulators in the image.

[0073] Images are read from the test set and fed into the trained DetectoRS object detection network for inference testing. The object detection network generates a bounding box for each string of insulators, containing its category and location information, as shown in the following expression:

[0074] bbox = [x, y, w, h, cls]

[0075] Where bbox represents the entire detection box, (x, y) represents the coordinates of the top left corner of the detection box, w and h are the width and height of the detection box, respectively, and cls represents the insulator category.

[0076] The generated detection bounding boxes can be used to determine the category and location of each insulator in the image. This embodiment uses an actual photographed image of an insulator, and an example of the target detection results is shown below. Figure 2 As shown.

[0077] Step S4: Extract each insulator from the entire image based on the detection bounding boxes, narrowing the region of interest, and inputting the extracted region into the BiSeNet semantic segmentation network to obtain insulator mask information, achieving separation of insulators from the background. Based on the insulator detection bounding boxes obtained in Step S3, extract each insulator from the entire image, and input the extracted region into the BiSeNet semantic segmentation network. During segmentation inference, firstly, the extracted region image undergoes convolution, batch normalization, and ReLU activation processing in the spatial path branch to preserve the original spatial dimensions and encode spatial information. Simultaneously, the extracted region image obtains a large receptive field in the context path branch through a lightweight downsampling model and global average pooling. Then, pixel-level classification is performed on the fused feature map of the spatial path branch and context path branch results, thereby outputting the insulator mask image and achieving segmentation of insulators in the extracted region.

[0078] The result of semantic segmentation is a pixel-level classification image that can distinguish insulators from the background. In this embodiment, the image used, after semantic segmentation in step S4, is as follows: Figure 3 As shown.

[0079] Step S5: Post-process the semantically segmented insulator image by performing image grayscale conversion, image binarization, morphological closing operation, edge detection, and contour finding algorithms to obtain the outer contour of the insulator. First, the pixel-level classification image generated in step S4 is converted into a grayscale image and binarized using the Otsu thresholding method. Then, in the obtained binary image, image edge filling with a border_size = 5 is performed to preserve the edge information of the insulator at the image boundary. Morphological closing operation is used to fill small holes in the image. Finally, the Canny edge detection algorithm is used to obtain the basic contour of the insulator, and then the findContours function in OpenCV is used to find the clear outer contour of the insulator.

[0080] Step S6: Calculate the minimum circumscribed rotation rectangle of the insulator's outer contour, and apply an affine transformation to the insulator image according to its rotation angle to rotate it to the horizontal direction.

[0081] Based on the insulator's outer contour found in step S5, the minimum bounding rectangle of the insulator is obtained using the `minAreaRect` function in OpenCV. The expression for the bounding rectangle is as follows:

[0082] rotated_box = [x c ,y c [width,height,θ]

[0083] In the formula, rotated_box represents the entire rotated rectangle, (xc y c ) represents the coordinates of the center point of the rotating rectangle, width and height are the width and height of the rotating rectangle, respectively, and θ is the rotation angle of the rotating rectangle;

[0084] The angle θ is corrected to meet the requirements of the affine transformation as follows:

[0085]

[0086] Where θ' is the corrected rotation angle. The affine transformation matrix M is thus constructed as follows:

[0087]

[0088] Where M represents the constructed affine transformation matrix, the unit of θ needs to be converted to radians, and s is the scaling factor, which is set to 1 here to indicate no scaling.

[0089] To prevent edge pixels from exceeding the boundary and causing information loss after rotation, the size of the rotated image is adjusted as follows:

[0090] w new =|w ori ·cosθ|+|h ori ·sinθ|

[0091] h new =|w ori ·sinθ|+|h ori ·cosθ|

[0092] In the formula, w new and h new w represents the width and height of the adjusted image, respectively. ori and h ori Let represent the width and height of the original image, respectively. Then, further adjust some elements of the affine transformation matrix M to fit the new image as follows:

[0093]

[0094] In the formula, M is the affine transformation matrix, M' is the adjusted affine transformation matrix as a whole, and w new and h new M'(0,2) and M'(1,2) represent the width and height of the adjusted image, respectively. M'(0,2) and M'(1,2) represent the elements at the corresponding index positions in the M' matrix, and M(0,2) and M(1,2) are the elements at the corresponding index positions in the M matrix.

[0095] After obtaining the new affine transformation matrix M' and the adjusted image size w new h newThen, these parameters are input into the OpenCV `warpAffine` function to perform an affine transformation, thereby rotating the insulator to a horizontal position. An example of an insulator image rotated to a horizontal position in this embodiment is shown below. Figure 4 As shown.

[0096] Step S7: Divide the horizontal insulator image into upper and lower parts. In each part of the image, plot the "column pixel and structure curve" with the horizontal coordinate of the image as the x-axis and the sum of pixels in each column of the image as the y-axis, thereby quantifying the shape and structure of the insulator.

[0097] Examples of column pixels and structure curves in the image in this embodiment are as follows: Figure 5 As shown in the figure, the dashed line represents the quantization structure of the upper half of the image, and the solid line represents the quantization structure of the lower half of the image.

[0098] Step S8: Smooth the column pixel and structure curves, find each peak in the curve, calculate the horizontal distance between each pair of consecutive peaks in order from left to right, and use the horizontal distance to construct an adaptive distance threshold.

[0099] First, a sliding window averaging method is used to smooth the column pixels and structure curves, eliminating noise interference. The corresponding sliding window averaging calculation method is as follows:

[0100]

[0101] In the formula, The sum of pixels in the nth column of the image is the result after smoothing. y(n), y(n-1), and y(n-(N-1)) are the sums of pixels in the nth, (n-1), and (n-(N-1))th columns of the image in the sliding window, respectively, and N is the length of the sliding window. When smoothing, the movement step of the sliding window is set to 1.

[0102] For the smoothed curve, the peak is identified by finding local maxima of the signal, as shown in the following expression:

[0103] y(i-1)<y(i)> y(i+1)

[0104] Where y(i-1), y(i), and y(i+1) represent the sum of pixels in the (i-1), i, and (i+1)th consecutive columns of the image, respectively, and y(i) is the peak value found. This peak value is also denoted as yi. peak (i); In order for each wave peak to represent the top of each insulator cap, the wave peak values ​​are selected as follows:

[0105] y valid_peak (j)=y peak (i), if y peak (i)>α·ymax

[0106] In the formula, y valid_peak (j) represents the j-th effective peak value selected, y max Let be the maximum column pixel sum in the image, i.e., the maximum signal value in the curve, and α be the peak adjustment coefficient, with a value range of [0,1]. After confirming each valid peak, the horizontal distance between consecutive peak pairs is calculated sequentially from left to right, constructing an adaptive distance threshold T related to the position of the peak pair. i for:

[0107] T i =β·d i,i+1 =β·(x(i+1)-x(i))

[0108] Among them, T i This represents the adaptive distance threshold constructed from the i-th and i+1-th valid peaks, where β is the horizontal distance adjustment coefficient, β>1, and d i,i+1 Let x(i) be the horizontal distance between the i-th and i+1-th valid peaks, and let x(i) and x(i+1) represent the horizontal coordinates of the image corresponding to the i-th and i+1-th valid peaks, respectively.

[0109] Step S9: Use the constructed adaptive distance threshold to determine the fault. The fault detection criteria are as follows:

[0110]

[0111] In the formula, defect i This indicates the fault presence status between the i-th and (i+1)-th valid peaks, where 1 indicates a fault and 0 indicates normal operation. i-1 The adaptive distance threshold is constructed from the (i-1)th and i-th valid peaks.

[0112] In other words, if the horizontal distance between a pair of consecutive wave peaks is greater than the adaptive distance threshold constructed from the previous pair of consecutive wave peaks, then a cap-drop fault exists between the pair of wave peaks, and the current insulator is a faulty insulator, thus achieving fault detection. Through the above fault detection criteria, it can be determined whether a cap-drop fault exists in the currently processed insulator, and the location of the fault can be located. After judgment, the insulator fault detection example in this embodiment is as follows: Figure 6 As shown, Figure 6 The rectangle in the image clearly shows that a cap is missing, thus completing the detection of the fault location.

[0113] In summary, the method presented in this case improves upon the traditional approach of locating insulators first and then detecting faults. By using the lightweight semantic segmentation network BiSeNet to further separate the insulators from the background, and through image post-processing such as grayscale conversion, binarization, closing operation, edge detection, and contour finding of the semantically segmented image, as well as affine transformation of the image and plotting the "image horizontal coordinate - column pixel sum" curve, the shape and structure of the insulator are effectively modeled. Finally, an adaptive threshold required for fault judgment is constructed, which, unlike a constant threshold, more efficiently and accurately determines the existence and location of insulator faults. It can handle insulator images of different sizes and has better versatility.

[0114] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A method for insulator fault detection based on deep learning and adaptive image processing, characterized in that, Comprise the following steps: S1, data set construction: collect insulator image samples, divide training set and test set, and label the images, and construct an insulator data set; S2, network model construction: construct a DetectoRS target detection network and a BiSeNet semantic segmentation network model, the DetectoRS target detection network is used for detecting insulators; the BiSeNet semantic segmentation network is used for quickly segmenting insulators; the training set divided in step S1 is input into the network model for training, and the trained DetectoRS target detection network and BiSeNet semantic segmentation network model are obtained; S3, insulator class and position information acquisition: input the test set image divided in step S1 into the trained DetectoRS target detection network in step S2, after network feature extraction and feature fusion, the network generates a candidate box according to the feature map of different scales, and classifies according to the position and feature information of the candidate box to determine the class of each detected target; at the same time, the network predicts the position of the target through the regression task to generate a detection box for each insulator, realizing the detection of insulators in the image; S4, insulator mask information acquisition: according to the insulator detection box obtained in step S3, each insulator is extracted from the whole image, and the extracted area is input into the BiSeNet semantic segmentation network, and an insulator mask image is output, realizing the segmentation of the insulator in the extracted area; S5, image post-processing: post-processing is performed on the insulator image obtained in step S4 to determine the outer contour of the insulator, the post-processing includes but is not limited to image graying, image binarization, morphological closing operation, edge detection and contour searching; S6, generating and affine transformation of the minimum circumscribed rotating rectangle: calculating the minimum circumscribed rotating rectangle of the insulator outer contour, and applying affine transformation to the insulator image according to the rotating angle to rotate it to the horizontal direction; S7, insulator structure quantization: dividing the horizontal insulator image obtained in step S6 into two parts, and drawing a "column pixel sum structure curve" in each part of the image, taking the horizontal coordinate of the image as the abscissa and the pixel sum of each column of the image as the ordinate, to quantitatively represent the shape structure of the insulator; S8, adaptive distance threshold construction: smoothing the "column pixel sum structure curve" obtained in step S7 by using the sliding window average method, finding each peak value in the curve, calculating the horizontal distance between each pair of continuous peaks in order from left to right, and constructing an adaptive distance threshold; S9, fault judgment: if the horizontal distance between a pair of continuous peaks is greater than the adaptive distance threshold constructed by the previous pair of continuous peaks, there is a cap loss fault between the pair of peaks, and the current insulator is a fault insulator, realizing fault detection.

2. The deep learning and adaptive image processing based insulator fault detection method of claim 1, wherein: In the image labeling in step S1, the labeled image label at least includes a detection box label for target detection and a mask label for semantic segmentation. 3.The insulator fault detection method based on deep learning and adaptive image processing of claim 1, wherein: In the step S2, the model source code reproduced by the MMDetection and the MMSegmentation is used to construct a DetectoRS target detection network and a BiSeNet semantic segmentation network model, The DetectoRS target detection network at least includes a recursive feature pyramid and a switchable atrous convolution module; The recursive feature pyramid module is used to enhance the network's ability to capture fine-grained and contextual information; the switchable atrous convolution module improves the network's perception ability of multi-scale objects by dynamically switching the size of the atrous convolution kernel. The BiSeNet semantic segmentation network includes two branches of spatial path and context path; the spatial path is used to retain the spatial size of the input image and encode spatial information; the context path uses a lightweight down-sampling model and a global average pooling to provide a large receptive field.

4. The deep learning and adaptive image processing based insulator fault detection method of claim 1, wherein: In the step S3, the trained DetectoRS target detection network is input with test set images to generate insulator detection boxes, specifically: ； Wherein, bbox represents the whole of each detection box, (x, y) represents the left upper corner coordinates of the detection box, w and h are the width and height of the detection box respectively, and cls represents the insulator category.

5. The deep learning and adaptive image processing based insulator fault detection method of claim 1, wherein: In the step S5, the Otsu threshold method is used to binarize the image, and in the obtained binary image, image edge filling with a border size of 5 is implemented, then morphological closing operation is used to fill small holes in the image, finally Canny edge detection algorithm is used to obtain the basic outline of the insulator, and findContours function in OpenCV is used to find the clear outer contour of the insulator.

6. The deep learning and adaptive image processing based insulator fault detection method of claim 1, wherein: The rotating rectangle in the step S6 is specifically: ； Wherein, rotated_box represents the whole of the rotated rectangle, (x c , y c ) represents the center point coordinates of the rotated rectangle, width, height are the width and height of the rotated rectangle respectively, is the rotation angle of the rotated rectangle; Adjusted affine transformation matrix Specifically: ； where M is an affine transformation matrix, is the adjusted affine transformation matrix as a whole, w new and h new represent the width and height of the adjusted image, respectively, and represent the element at the corresponding index position in the matrix, and the element at the corresponding index position in the M matrix; The adjusted affine transformation matrix and the size w of the adjusted image new , h new Implement the affine transformation in the warpAffine function of OpenCV, and rotate the insulator image to the horizontal direction.

7. The deep learning and adaptive image processing based insulator fault detection method of claim 1, wherein: The method of sliding window average in the step S8 is specifically: ； wherein, denotes the nth column of pixels of the image and the smoothed result, , and are the sums of the nth, (n-1)th and (n-(N-1))th columns of pixels in the sliding window, respectively, and N is the length of the sliding window. The method of finding local maximum value of signal is used to confirm the wave peak, specifically: ； wherein, , and respectively denote the pixels of the (i-1)th, ith and (i+1)th consecutive columns in the image and is the found peak value, let the peak value be ; the peak value is filtered as follows: ； wherein, represents the jth valid peak value screened out, is the maximum column pixel sum of the image, is a peak adjustment coefficient.

8. The deep learning and adaptive image processing based insulator fault detection method of claim 1, wherein: The adaptive distance threshold in the step S8 is: ； wherein, denotes an adaptive distance threshold built from the i-th and i+1-th significant peaks, is a horizontal distance adjustment coefficient, , is the horizontal distance between the i-th and i+1-th significant peaks, and denote the image horizontal coordinates corresponding to the i-th and i+1-th significant peaks, respectively. 9.The insulator fault detection method based on deep learning and adaptive image processing of claim 1, wherein: The fault detection in the step S9 is specifically: ； wherein represents the fault presence state between the i-th and i+1-th effective peaks, 1 indicates a fault, 0 indicates normal, is an adaptive distance threshold value constructed from the i-1-th and i-th effective peaks.

10. A computer device, comprising: including, a memory for storing instructions; a processor for executing the instructions, so that the computer device performs the operation steps of the insulator fault detection method based on deep learning and adaptive image processing according to any one of claims 1-9.

Images