Insulator fault detection method and device based on deep learning and adaptive image processing

Abstract

The invention discloses an insulator fault detection method and device based on deep learning and adaptive image processing, and the method comprises the steps: firstly constructing an insulator data set, and training a DetectoRS target detection network and a Bi SeNet semantic segmentation network on a training set; inputting the test set image into a DetectoRS network to generate an insulator detection frame; extracting each insulator according to the detection frame, and inputting an extraction region into a Bi SeNet network to generate an insulator mask; post-processing the image in the semantic segmentation result image to obtain the outer contour of the insulator; calculating the minimum external rotation rectangle of the outer contour of the insulator, and rotating the image to be horizontal according to the rotation angle of the minimum external rotation rectangle; constructing a quantization curve of the shape structure of the insulator according to the horizontal insulator image; and finally, the curve is smoothed, a self-adaptive threshold value required by fault judgment is constructed, the existence state and the position of the insulator fault are judged, and the accuracy and the universality of fault detection are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision and power inspection, and mainly relates to an insulator fault detection method and device based on deep learning and adaptive image processing. Background Art

[0002] Transmission lines are an important part of the power grid. They are widely distributed and have a complex surrounding environment. The operating status of transmission line equipment is directly related to the power supply reliability of the power grid. In transmission lines, insulators, as important components, are installed between conductors of different potentials or between conductors and grounding components. They play an important role in supporting line conductors and electrical insulation isolation, and are subject to large mechanical tension and extremely high voltage. In long-term operation, insulators may fail, causing unstable operation of high-voltage transmission lines, and even causing large-scale power outages in severe cases. Therefore, in order to ensure the normal operation of transmission lines, it is necessary to conduct regular safety inspections on the status of insulators to eliminate potential fault hazards.

[0003] In recent years, my country's power industry has widely adopted drone technology for power inspections. Drones can quickly collect a large amount of insulator data through high-definition cameras, help discover problems and hidden dangers in power equipment, and improve the reliability and safety of the power system. Therefore, it is of great practical significance to study how to achieve efficient and accurate insulator fault detection through drone images.

[0004] At present, the methods for realizing insulator fault detection in images can be mainly divided into two types: the first method is to use a single target detection network to directly detect the insulator string and the fault on it at the same time. Commonly used detection networks include YOLO series, SSD series, RCNN series, etc. The advantages of this method are simple deployment and strong versatility. However, since the insulator fault is usually small and the drone cannot get too close to the transmission line for shooting. Therefore, the fault generally accounts for a small proportion in the image, resulting in limited detection accuracy of this method. The second method is to first use the target detection network to locate the insulator position, then crop the corresponding area, and then combine the classic image processing algorithm to complete the fault detection. The advantage of this method is that locating the insulator first allows the algorithm to pay more attention to the fault area of ​​interest, which helps to improve the accuracy of detection. However, since the fault detection in the second step is completely dependent on the classic image processing algorithm, the versatility of the detection is poor, and the algorithm may fail in complex and diverse scenarios. In summary, in the actual application of drone power grid inspection, neither of the two commonly used methods can ideally complete the task of insulator fault detection. Summary of the invention

[0005] The present invention aims at the limitations of the existing insulator fault detection methods, and provides an insulator fault detection method and device based on deep learning and adaptive image processing. First, the insulator image is collected to construct an insulator data set; the DetectoRS target detection network and the BiSeNet semantic segmentation network are trained on the training set; the test set image is input into the DetectoRS network to generate an insulator detection frame; each insulator is extracted according to the detection frame, and the extracted area is input into the BiSeNet network to generate an insulator mask; the image is post-processed in the result image of the semantic segmentation to obtain the outer contour of the insulator; the minimum circumscribed rotation rectangle of the outer contour of the insulator is calculated, and the image is rotated to the horizontal according to its rotation angle; a quantization curve of the insulator shape structure is constructed according to the horizontal insulator image; finally, the curve is smoothed, and the adaptive threshold required for fault judgment is constructed to judge the existence state and location of the insulator fault. The method of the present invention is improved on the basis of the method of first locating the insulator and then detecting the fault, and effectively improves the accuracy and versatility of the fault detection algorithm without significantly increasing the amount of calculation.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: an insulator fault detection method based on deep learning and adaptive image processing, comprising the following steps:

[0007] S1. Dataset construction: Collect insulator image samples, divide them into training set and test set, annotate the images, and construct the insulator dataset;

[0008] S2. Network model construction: DetectoRS target detection network and BiSeNet semantic segmentation network model are constructed. The DetectoRS target detection network is a target detection method based on convolutional neural network, which combines recursive feature pyramid (RFP) and switchable hole convolution (SAC) modules; the RFP module enhances the network's ability to capture fine-grained and contextual information by recursively fusing multi-scale features; the SAC module improves the network's perception of multi-scale objects by dynamically switching the size of the hole convolution kernel; the DetectoRS target detection network has good detection accuracy and robustness, and is suitable for detecting insulators in complex scenes; the BiSeNet semantic segmentation network is a lightweight semantic segmentation method based on convolutional neural network, which mainly includes two branches: spatial path and context path; the spatial path can retain the spatial size of the input image and encode spatial information, and the context path can use a lightweight down-sampling model and global average pooling to provide a large receptive field; BiSeNet semantic segmentation network is a lightweight semantic segmentation method based on convolutional neural network, which mainly includes two branches: spatial path and context path; the spatial path can retain the spatial size of the input image and encode spatial information, and the context path can use a lightweight down-sampling model and global average pooling to provide a large receptive field; The SeNet semantic segmentation network has good inference accuracy and speed, and is suitable for fast segmentation of insulators in complex scenes. The training set divided in step S1 is input into the network model for training to obtain the trained DetectoRS target detection network and Bi SeNet semantic segmentation network model.

[0009] S3, insulator category and location information acquisition: the test set images divided in step S1 are input into the DetectoRS target detection network trained in step S2; when performing detection reasoning, first, the image passes through the feature extraction stage of the network, which extracts multi-level features of the image through the convolution layer. During the feature extraction process, the switchable atrous convolution (SAC) module dynamically switches between atrous convolution kernels of different scales to enhance the network's perception of insulators of different sizes; then, the image after feature extraction is input into the recursive feature pyramid (RFP) module, which further enhances the capture of fine-grained and contextual information by recursively fusing feature maps of different scales; finally, the network generates candidate boxes based on feature maps of different scales, and classifies them according to the positions of the candidate boxes and their feature information to determine the category of each detected target; at the same time, the network predicts the position of the target through the regression task, thereby generating a detection box for each insulator to detect the insulator in the image;

[0010] S4, insulator mask information acquisition: according to the insulator detection frame obtained in step S3, each insulator is extracted from the entire image, and the extracted area is input into the Bi SeNet semantic segmentation network; when performing segmentation reasoning, first, the extracted regional image is processed by convolution, batch normalization and ReLU activation of the spatial path branch to retain the original spatial size and encode spatial information; at the same time, the extracted regional image is subjected to a lightweight down-sampling model and global average pooling in the context path branch to obtain a large receptive field; then, pixel-level classification is performed on the fused feature map of the spatial path branch and the context path branch results, thereby outputting the insulator mask image to achieve the segmentation of the insulator in the extracted area;

[0011] S5, image post-processing: performing post-processing on the insulator image obtained in step S4 to determine the outer contour of the insulator, wherein the post-processing includes but is not limited to image graying, image binarization, morphological closing operation, edge detection and contour finding;

[0012] S6. Generate circumscribed rotated rectangle and affine transformation: Calculate the minimum circumscribed rotated rectangle of the outer contour of the insulator, and apply affine transformation to the insulator image according to its rotation angle to rotate it to the horizontal direction;

[0013] S7, quantification of insulator structure: the horizontal insulator image obtained in step S6 is divided into two parts, the upper and lower parts, and in each part of the image, the horizontal coordinate of the image is used as the abscissa and the sum of pixels in each column of the image is used as the ordinate to draw a "column pixel sum structure curve" of the image, so as to quantitatively represent the shape structure of the insulator;

[0014] S8, constructing an adaptive distance threshold: smoothing the "column pixel and structure curve" obtained in step S7 by using a sliding window average method, finding each peak value in the curve, calculating the horizontal distance between each continuous peak pair from left to right, and constructing an adaptive distance threshold;

[0015] S9, fault judgment: If the horizontal distance between a pair of continuous peaks is greater than the adaptive distance threshold constructed by the previous pair of continuous peaks, there is a cap loss fault between the pair of peaks, and the current insulator is a faulty insulator, thus achieving fault detection.

[0016] As an improvement of the present invention, in the data labeling step S1, the labeled image labels at least include detection box labels for target detection and mask labels for semantic segmentation.

[0017] As an improvement of the present invention, in step S2, the model source code reproduced by MMDetection and MMSegmentation is used to construct the DetectoRS target detection and BiSeNet semantic segmentation network models.

[0018] As another improvement of the present invention, in step S3, the test set image is input into the trained DetectoRS target detection network to generate an insulator detection frame, specifically:

[0019] bbox=[x,y,w,h,cls]

[0020] Among them, bbox represents each detection box as a whole, (x, y) represents the coordinates of the upper left corner of the detection box, w and h are the width and height of the detection box respectively, and cls represents the insulator category; through the generated detection box, the category of each insulator in the image can be determined and its location can be located, and then the extracted area is input into the trained BiSeNet semantic segmentation network for inference testing, and an insulator mask will be generated. The segmentation result is reflected in a pixel-level classification map that can distinguish insulators from the background.

[0021] As another improvement of the present invention, in step S5, the image is binarized using the Otsu threshold method, and in the obtained binary image, image edge filling with a filling length of border_size=5 is implemented, and then a morphological closing operation is used to fill small holes in the image, and finally the Canny edge detection algorithm is used to obtain the basic contour of the insulator, and then the findContours function in OpenCV is used to find the clear outer contour of the insulator.

[0022] As another improvement of the present invention, the rotating rectangle in step S6 is specifically:

[0023] rotated_box=[x c ,y c ,width,height,θ]

[0024] Among them, rotated_box represents the entirety of the rotated rectangle, (x c ,y c ) represents the coordinates of the center point of the rotated rectangle, width and height are the width and height of the rotated rectangle respectively, and θ is the rotation angle of the rotated rectangle;

[0025] The correction for angle θ is as follows:

[0026]

[0027] Where θ' is the rotation angle after correction;

[0028] Construct the affine transformation matrix M as follows:

[0029]

[0030] Where M represents the constructed affine transformation matrix, the unit of θ needs to be converted to radians, and s is the scaling factor;

[0031] Adjust the size of the rotated image as follows:

[0032] w new =|w ori ·cosθ|+|h ori ·sinθ|

[0033] h new =|w ori ·sinθ|+|h ori ·cosθ|

[0034] Among them, w new and h new Respectively represent the width and height of the adjusted image, w ori and h ori Respectively represent the width and height of the original image;

[0035] The adjusted affine transformation matrix M' is specifically:

[0036]

[0037] Among them, M is the affine transformation matrix, M' is the adjusted affine transformation matrix as a whole, and w new and h new Respectively represent the width and height of the adjusted image, M'(0,2) and M'(1,2) represent the elements at the corresponding index positions in the M' matrix, and M(0,2) and M(1,2) are the elements at the corresponding index positions in the M matrix;

[0038] The adjusted affine transformation matrix M' and the adjusted image size w new ,h new Input into the warpAffi ne function of OpenCV to implement affine transformation and rotate the insulator image to the horizontal direction.

[0039] As another improvement of the present invention, the sliding window averaging method in step S8 is specifically:

[0040]

[0041] in, represents the smoothed result of the nth column of the image, y(n), y(n-1) and y(n-(N-1)) are the pixel sums of the nth, n-1th and n-(N-1th) columns of the image in the sliding window respectively, and N is the length of the sliding window;

[0042] The peak is confirmed by finding the local maximum value of the signal, specifically:

[0043] y(i-1)<y(i)> y(i+1)

[0044] Among them, y(i-1), y(i) and y(i+1) represent the sum of the pixels in the i-1th, ith and i+1th columns of the image respectively, and y(i) is the peak value found, which is recorded as y peak (i) The peak value is screened as follows:

[0045] y valid_peak (j) = y peak (i), if y peak (i)>α·y max

[0046] Among them, y valid_peak (j) represents the jth effective peak value selected, y max is the maximum column pixel sum of the image, and α is the peak adjustment coefficient.

[0047] As another improvement of the present invention, the adaptive distance threshold T in step S8 i for:

[0048] T i =β·d i,i+1 =β·(x(i+1)-x(i))

[0049] Among them, T i represents the adaptive distance threshold constructed by the i-th and i+1-th effective peaks, β is the horizontal distance adjustment coefficient, β>1, d i,i+1 is the horizontal distance between the i-th and i+1-th effective peaks, and x(i) and x(i+1) represent the image horizontal coordinates corresponding to the i-th and i+1-th effective peaks, respectively.

[0050] As a further improvement of the present invention, the fault detection in step S9 is specifically as follows:

[0051]

[0052] Among them, defect i Indicates the fault status between the i-th and i+1-th effective peaks, 1 indicates a fault, 0 indicates normal, T i-1 is the adaptive distance threshold constructed by the i-1th and i-th valid peaks.

[0053] In order to achieve the above object, the present invention also adopts a technical solution: a computer device, comprising:

[0054] A memory for storing instructions;

[0055] A processor is used to execute the instructions so that the computer device performs the operating steps of any of the above-mentioned insulator fault detection methods based on deep learning and adaptive image processing.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] (1) Before performing fault detection in the extracted insulator area, the method proposed in the present invention proposes to use a lightweight semantic segmentation network BiSeNet to further separate the insulator and the background. Compared with the use of classical image segmentation methods, the use of lightweight deep learning methods can greatly improve the accuracy of insulator segmentation and adaptability to complex background environments without significantly increasing the computational burden.

[0058] (2) In the process of post-processing the data by the method of the present invention, the shape structure of the insulator can be effectively modeled through grayscale, binarization, closing operation, edge detection and contour search of the semantic segmentation image, as well as affine transformation of the image and drawing of the "image horizontal coordinate-column pixel sum" curve, which is helpful for subsequent fault identification.

[0059] (3) Compared with the constant threshold, the adaptive distance threshold finally constructed in the method of the present invention can cope with insulator images of different sizes and has better versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of the steps of the method of the present invention;

[0061] Figure 2 This is a schematic diagram of the result after target detection in step S3 of Example 1 of the present invention;

[0062] Figure 3 This is a schematic diagram of the result after semantic segmentation in step S4 in Example 1 of the present invention;

[0063] Figure 4 This is a schematic diagram of a horizontal insulator image after image post-processing and affine transformation in step S6 of embodiment 1 of the present invention;

[0064] Figure 5 Schematic diagram of insulator image column pixels and structure curves in Example 1 of the present invention;

[0065] Figure 6 This is a schematic diagram of the fault detection results of Example 1 of the present invention. DETAILED DESCRIPTION

[0066] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0067] Example 1

[0068] An insulator fault detection method based on deep learning and adaptive image processing, such as Figure 1 As shown, the specific steps are as follows:

[0069] Step S1: Use drones to collect insulator image samples on site, divide them into training sets and test sets, and annotate the images to construct an insulator dataset. The annotated image labels include detection box labels for target detection and mask labels for semantic segmentation.

[0070] Step S2: construct DetectoRS target detection and Bi SeNet semantic segmentation network models. The DetectoRS target detection network is a target detection method based on convolutional neural networks, which combines recursive feature pyramid (RFP) and switchable atrous convolution (SAC) modules; the RFP module enhances the network's ability to capture fine-grained and contextual information by recursively fusing multi-scale features; the SAC module improves the network's perception of multi-scale objects by dynamically switching the size of the atrous convolution kernel; the DetectoRS target detection network has good detection accuracy and robustness, and is suitable for detecting insulators in complex scenes; the Bi SeNet semantic segmentation network is a lightweight semantic segmentation method based on convolutional neural networks, mainly including two branches: a spatial path and a context path; the spatial path can retain the spatial size of the input image and encode spatial information, and the context path can use a lightweight down-sampling model and global average pooling to provide a large receptive field; the Bi SeNet semantic segmentation network has good inference accuracy and inference speed, and is suitable for quickly segmenting insulators in complex scenes; the network is trained using training set data.

[0071] When building the DetectorRS target detection and BiSeNet semantic segmentation network models, the model source codes reproduced by MMDetection and MMSegmentation are used respectively.

[0072] Step S3: Input the test set image into the DetectoRS target detection network, generate the insulator detection frame, and obtain the category and position information of the insulator. Input the test set image divided in step S1 into the DetectoRS target detection network trained in step S2; when performing detection reasoning, first, the image passes through the feature extraction stage of the network, which extracts the multi-level features of the image through the convolution layer. During the feature extraction process, the switchable atrous convolution (SAC) module dynamically switches between atrous convolution kernels of different scales to enhance the network's perception of insulators of different sizes; then, the image after feature extraction is input into the recursive feature pyramid (RFP) module, which further enhances the capture of fine-grained and contextual information by recursively fusing feature maps of different scales; finally, the network generates candidate frames based on feature maps of different scales, and classifies them according to the position of the candidate frames and their feature information to determine the category of each detected target; at the same time, the network predicts the position of the target through the regression task, thereby generating a detection frame for each insulator to detect the insulator in the image;

[0073] Read images from the test set and input them into the trained DetectoRS object detection network for inference testing. The object detection network generates a detection box for each string of insulators, including its category and location information, as shown below:

[0074] bbox=[x,y,w,h,cls]

[0075] Among them, bbox represents each detection box as a whole, (x, y) represents the coordinates of the upper left corner of the detection box, w, h are the width and height of the detection box respectively, and cls represents the insulator category.

[0076] Through the generated detection frame, the category of each insulator in the image can be determined and its location can be located. This embodiment uses an actual insulator image, and an example of its target detection result is as follows: Figure 2 shown.

[0077] Step S4: Extract each insulator from the whole image according to the detection frame, narrow the scope of the region of interest, and input the extracted region into the Bi SeNet semantic segmentation network to obtain the insulator mask information and realize the separation of the insulator and the background. According to the insulator detection frame obtained in step S3, extract each insulator from the whole image, and input the extracted region into the Bi SeNet semantic segmentation network; when performing segmentation reasoning, first, the extracted regional image is processed by convolution, batch normalization and ReLU activation of the spatial path branch to retain the original spatial size and encode spatial information; at the same time, the extracted regional image is processed by lightweight down-sampling model and global average pooling in the context path branch to obtain a large receptive field; then, pixel-level classification is performed on the fused feature map of the spatial path branch and the context path branch results, thereby outputting the insulator mask image to realize the segmentation of the insulator in the extracted region.

[0078] The result of semantic segmentation is reflected in the pixel-level classification map that can distinguish the insulator from the background. The image cited in this embodiment is semantically segmented in step S4, such as Figure 3 shown.

[0079] Step S5: Post-process the semantically segmented insulator image by implementing algorithms such as image grayscale, image binarization, morphological closing operation, edge detection and contour search to obtain the outer contour of the insulator. First, the pixel-level classification image generated in step S4 is converted into a grayscale image and binarized using the Otsu threshold method. Then, in the obtained binary image, the image edge filling with a filling length of border_size=5 is implemented to protect the edge information of the insulator at the image boundary, and the morphological closing operation is used to fill the small holes in the image. Finally, the Canny edge detection algorithm is used to obtain the basic contour of the insulator, and then the findContours function in OpenCV is used to find the clear outer contour of the insulator.

[0080] Step S6: Calculate the minimum circumscribed rotation rectangle of the insulator outer contour, and apply an affine transformation to the insulator image according to its rotation angle to rotate it to the horizontal direction.

[0081] According to the outer contour of the insulator found in step S5, the minAreaRect function in OpenCV is used to obtain the minimum circumscribed rotated rectangle of the insulator. The expression of the rotated rectangle is as follows:

[0082] rotated_box=[x c ,y c ,width,height,θ]

[0083] In the formula, rotated_box represents the entire rotated rectangle, (xc ,y c ) represents the coordinates of the center point of the rotated rectangle, width and height are the width and height of the rotated rectangle respectively, and θ is the rotation angle of the rotated rectangle;

[0084] The angle θ is corrected to meet the requirements of the affine transformation as follows:

[0085]

[0086] Among them, θ' is the rotation angle after correction. The affine transformation matrix M is constructed as follows:

[0087]

[0088] Among them, M represents the constructed affine transformation matrix, the unit of θ needs to be converted to radians, and s is the scaling factor, which is 1 here, indicating no scaling.

[0089] In order to prevent the edge pixels of the rotated image from exceeding the boundary and causing information loss, the size of the rotated image is adjusted as follows:

[0090] w new =|w ori ·cosθ|+|h ori ·sinθ|

[0091] h new =|w ori ·sinθ|+|h ori ·cosθ|

[0092] In the formula, w new and h new Respectively represent the width and height of the adjusted image, w ori and h ori Represent the width and height of the original image respectively. At this point, further adjust some elements of the affine transformation matrix M to adapt to the new image as follows:

[0093]

[0094] Where M is the affine transformation matrix, M' is the adjusted affine transformation matrix as a whole, and w new and h new Respectively represent the width and height of the adjusted image, M'(0,2) and M'(1,2) represent the elements at the corresponding index positions in the M' matrix, and M(0,2) and M(1,2) are the elements at the corresponding index positions in the M matrix;

[0095] After obtaining the new affine transformation matrix M' and the adjusted image size w new ,h newAfter that, these parameters are input into the warpAffi ne function of OpenCV to implement affine transformation, thereby rotating the insulator to the horizontal direction. An example of an insulator image rotated to the horizontal direction in this embodiment is as follows: Figure 4 shown.

[0096] Step S7: Divide the horizontal insulator image into two parts, upper and lower parts. In each part of the image, use the horizontal coordinate of the image as the abscissa and the sum of pixels in each column of the image as the ordinate to draw a "column pixel and structure curve" of the image, so as to quantitatively represent the shape structure of the insulator.

[0097] An example of a column pixel and a structure curve of an image in this embodiment is as follows: Figure 5 As shown, the dotted line in the figure represents the quantization structure of the upper half of the image, and the solid line represents the quantization structure of the lower half of the image.

[0098] Step S8: Smooth the column pixels and the structure curve, find each peak value in the curve, calculate the horizontal distance between each continuous peak pair from left to right, and use the horizontal distance to construct an adaptive distance threshold.

[0099] First, the column pixels and structure curves are smoothed by sliding window averaging to eliminate noise interference. The corresponding sliding window average calculation method is as follows:

[0100]

[0101] In the formula, represents the result of smoothing the pixels of the nth column of the image. y(n), y(n-1) and y(n-(N-1)) are the pixel sums of the nth, n-1th and n-(N-1)th columns of the image in the sliding window respectively, and N is the length of the sliding window. When smoothing, the moving step size of the sliding window is set to 1.

[0102] For the smoothed curve, the peak is confirmed by finding the local maximum value of the signal. The expression is as follows:

[0103] y(i-1)<y(i)> y(i+1)

[0104] Where y(i-1), y(i) and y(i+1) represent the sum of the pixels in the i-1th, i-th and i+1th columns of the image respectively, and y(i) is the peak value found, which is also denoted by y peak (i) In order to make each wave peak represent the top of each insulator cap, the wave peak value is screened as follows:

[0105] y valid_peak (j) = y peak (i), if y peak (i)>α·ymax

[0106] In the formula, y valid_peak (j) represents the jth effective peak value selected, y max is the maximum column pixel sum in the image, that is, the maximum signal value in the curve, and α is the peak adjustment coefficient, which ranges from [0,1]. After confirming each valid peak, the horizontal distance between consecutive peak pairs is calculated from left to right, and an adaptive distance threshold T related to the position of the peak pair is constructed. i for:

[0107] T i =β·d i,i+1 =β·(x(i+1)-x(i))

[0108] Among them, T i represents the adaptive distance threshold constructed by the i-th and i+1-th effective peaks, β is the horizontal distance adjustment coefficient, β>1, d i,i+1 is the horizontal distance between the i-th and i+1-th effective peaks, and x(i) and x(i+1) represent the image horizontal coordinates corresponding to the i-th and i+1-th effective peaks, respectively.

[0109] Step S9: Use the constructed adaptive distance threshold to perform fault judgment. The fault detection criteria are as follows:

[0110]

[0111] In the formula, defect i Indicates the fault status between the i-th and i+1-th effective peaks, 1 indicates a fault, 0 indicates normal, T i-1 is the adaptive distance threshold constructed by the i-1th and i-th valid peaks.

[0112] That is, if the horizontal distance between a pair of continuous wave peaks is greater than the adaptive distance threshold constructed by the previous pair of continuous wave peaks, there is a cap loss fault between the pair of wave peaks, and the current insulator is a faulty insulator, thus achieving fault detection. By using the above fault detection criteria, it can be determined whether there is a cap loss fault in the insulator currently being processed, and the location of the fault can be located. After the determination, the detection example of the insulator fault in this embodiment is as follows: Figure 6 As shown, Figure 6 It can be clearly seen from the rectangular box in that there is a cap loss here, completing the detection of the fault location.

[0113] In summary, the method in this case improves on the traditional method of first locating the insulator and then detecting the fault. The insulator and the background are further separated by using the lightweight semantic segmentation network Bi SeNet. The shape structure of the insulator is effectively modeled through image post-processing such as grayscale, binarization, closing operation, edge detection and contour search of the semantic segmentation image, as well as affine transformation of the image and drawing of the "image horizontal coordinate-column pixel sum" curve. Finally, the adaptive threshold required for fault judgment is constructed, which is different from the constant threshold. It can more efficiently and accurately judge the existence status and location of the insulator fault, and can cope with insulator images of different sizes, with better versatility.

[0114] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. Insulator fault detection method based on deep learning and adaptive image processing, characterized in that: The steps include: S1. Dataset construction: Collect insulator image samples, divide them into training set and test set, annotate the images, and construct the insulator dataset; S2. Network model construction: construct a DetectoRS target detection network and a BiSeNet semantic segmentation network model, wherein the DetectoRS target detection network is used to detect insulators; the BiSeNet semantic segmentation network is used to quickly segment insulators; the training set divided in step S1 is input into the network model for training, and the trained DetectoRS target detection network and BiSeNet semantic segmentation network model are obtained; S3, obtaining the category and position information of insulators: input the test set images divided in step S1 into the DetectoRS target detection network trained in step S2. After network feature extraction and feature fusion, the network generates candidate boxes according to feature maps of different scales, and classifies them according to the positions and feature information of the candidate boxes to determine the category of each detected target. At the same time, the network predicts the position of the target through regression tasks, generates a detection box for each insulator, and realizes the detection of insulators in the image. S4, obtaining insulator mask information: according to the insulator detection frame obtained in step S3, each insulator is extracted from the entire image, and the extracted area is input into the BiSeNet semantic segmentation network, and the insulator mask image is output to achieve the segmentation of the insulator in the extracted area; S5, image post-processing: performing post-processing on the insulator image obtained in step S4 to determine the outer contour of the insulator, wherein the post-processing includes but is not limited to image graying, image binarization, morphological closing operation, edge detection and contour finding; S6. Generate circumscribed rotated rectangle and affine transformation: Calculate the minimum circumscribed rotated rectangle of the outer contour of the insulator, and apply affine transformation to the insulator image according to its rotation angle to rotate it to the horizontal direction; S7, quantification of insulator structure: the horizontal insulator image obtained in step S6 is divided into two parts, upper and lower parts, and in each part of the image, the horizontal coordinate of the image is used as the abscissa and the sum of pixels in each column of the image is used as the ordinate to draw a "column pixel sum structure curve" of the image, so as to quantitatively represent the shape structure of the insulator; S8, adaptive distance threshold construction: Use the sliding window average method to smooth the "column pixel and structure curve" obtained in step S7, find each peak value in the curve, calculate the horizontal distance between each continuous peak pair from left to right, and construct an adaptive distance threshold; S9, fault judgment: If the horizontal distance between a pair of continuous peaks is greater than the adaptive distance threshold constructed by the previous pair of continuous peaks, there is a cap loss fault between the pair of peaks, and the current insulator is a faulty insulator, thus achieving fault detection.

2. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: In the data labeling step S1, the labeled image labels at least include detection box labels for target detection and mask labels for semantic segmentation.

3. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: In step S2, the model source code reproduced by MMDetection and MMSegmentation is used to build the DetectoRS target detection network and the BiSeNet semantic segmentation network model. The DetectoRS target detection network includes at least a recursive feature pyramid (RFP) and a switchable atrous convolution (SAC) module; the RFP module is used to enhance the network's ability to capture fine-grained and contextual information; the SAC module improves the network's perception of multi-scale objects by dynamically switching the size of the atrous convolution kernel; The BiSeNet semantic segmentation network includes two branches: a spatial path and a context path; the spatial path is used to retain the spatial size of the input image and encode spatial information; the context path uses a lightweight down-sampling model and global average pooling to provide a large receptive field.

4. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: In step S3, the test set image is input into the trained DetectoRS target detection network to generate an insulator detection frame, specifically: bbox = [x, y, w, h, cls] Among them, bbox represents each detection box as a whole, (x, y) represents the coordinates of the upper left corner of the detection box, w, h are the width and height of the detection box respectively, and cls represents the insulator category.

5. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: In step S5, the image is binarized using the Otsu threshold method, and in the obtained binary image, image edge filling with a filling length of border_size=5 is implemented, and then a morphological closing operation is used to fill small holes in the image, and finally the Canny edge detection algorithm is used to obtain the basic contour of the insulator, and then the findContours function in OpenCV is used to find the clear outer contour of the insulator.

6. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: The rotated rectangle in step S6 is specifically: rotated_box=[x c ,y c ,width,height,θ] Among them, rotated_box represents the entirety of the rotated rectangle, (x c ,y c ) represents the coordinates of the center point of the rotated rectangle, width and height are the width and height of the rotated rectangle respectively, and θ is the rotation angle of the rotated rectangle; The angle θ is corrected as follows: Where θ' is the rotation angle after correction; Construct the affine transformation matrix M as follows: Where M represents the constructed affine transformation matrix, the unit of θ needs to be converted to radians, and s is the scaling factor; Adjust the size of the rotated image as follows: w new =|w ori ·cosθ|+|h ori ·sinθ| h new =|w ori ·sinθ|+|h ori ·cosθ| Among them, w new and h new Respectively represent the width and height of the adjusted image, w ori and h ori Respectively represent the width and height of the original image; The adjusted affine transformation matrix M' is specifically: Among them, M is the affine transformation matrix, M' is the adjusted affine transformation matrix as a whole, and w new and h new Respectively represent the width and height of the adjusted image, M'(0,2) and M'(1,2) represent the elements at the corresponding index positions in the M' matrix, and M(0,2) and M(1,2) are the elements at the corresponding index positions in the M matrix; The adjusted affine transformation matrix M' and the adjusted image size w new ,h new Input into the warpAffine function of OpenCV to implement affine transformation and rotate the insulator image to the horizontal direction.

7. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: The sliding window averaging method in step S8 is specifically as follows: in, represents the smoothed result of the nth column of the image, y(n), y(n-1) and y(n-(N-1)) are the pixel sums of the nth, n-1th and n-(N-1th) columns of the image in the sliding window respectively, and N is the length of the sliding window; The peak is confirmed by finding the local maximum value of the signal, specifically: y(i-1)<y(i)> y(i+1) Among them, y(i-1), y(i) and y(i+1) represent the sum of the pixels in the i-1th, ith and i+1th columns of the image respectively, and y(i) is the peak value found, which is recorded as y peak (i) The peak value is screened as follows: y valid_peak (j)=y peak (i),if y peak (i)>α·y max Among them, y valid_peak (j) represents the jth effective peak value selected, y max is the maximum column pixel sum of the image, and α is the peak adjustment coefficient.

8. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: In step S8, the adaptive distance threshold T i for: T i =β·d i,i+1 =β·(x(i+1)-x(i)) Among them, T i represents the adaptive distance threshold constructed by the i-th and i+1-th effective peaks, β is the horizontal distance adjustment coefficient, β>1, d i,i+1 is the horizontal distance between the i-th and i+1-th effective peaks, and x(i) and x(i+1) represent the image horizontal coordinates corresponding to the i-th and i+1-th effective peaks, respectively.

9. The insulator fault detection method based on deep learning and adaptive image processing according to claim 1, characterized in that: The fault detection in step S9 is specifically as follows: Among them, defect i Indicates the fault status between the i-th and i+1-th effective peaks, 1 indicates a fault, 0 indicates normal, T i-1 is the adaptive distance threshold constructed by the i-1th and i-th valid peaks.

10. A computer device, characterized in that: include, A memory for storing instructions; A processor is used to execute the instructions so that the computer device performs the operating steps of any of the above-mentioned insulator fault detection methods based on deep learning and adaptive image processing.

Images