A Fault Detection Method for Electrical Equipment Based on Infrared Images

Abstract

The present invention relates to a method for detecting electrical equipment faults based on infrared images. Compared with traditional fault detection based on image recognition technology, in this application, the first-dimensional infrared image and the second-dimensional infrared image of the electrical equipment are obtained, and the infrared images in different dimensions are used to correct each other, basically eliminating the interference hot spots in cases of dislocation, overlap, etc., reducing interference and improving the accuracy of image recognition. For fault testing, the watershed algorithm is used to segment the equipment, and finally the abnormal area and fault type of the faulty equipment are extracted. Among them, an opening operation and a closing operation are added to the watershed algorithm; the opening operation is used to remove small points and burrs of effective hot spots, and the closing operation is used to fill some small holes and connect the contours of the electrical equipment in the original image to avoid missing the contours of the extracted electrical equipment, so that the accuracy of the finally obtained image recognition is improved, and a fault recognition rate of more than 99% can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of electrical equipment fault detection, and particularly to a method for detecting electrical equipment faults based on infrared images. Background Art

[0002] Currently, in the fault detection of electrical equipment, infrared thermal imaging detection has the advantages of non-contact and non-power-off detection, and has been widely used in various electrical units. With the continuous development of the Chinese power grid, the pressure for infrared inspection of electrical equipment is increasing, and a large amount of infrared thermal image data has been collected by devices such as infrared inspection robots and fixed infrared thermal imagers. Merely relying on methods such as setting ordinary temperature thresholds or manual analysis and comparison is difficult to meet the detection of batch infrared thermal image data, and may cause false detections and missed detections, which not only increases the detection cost but also affects the real-time performance of detection. The time for electrical equipment to have a thermal fault is short. If the type of faulty equipment is not identified in time and the area showing abnormalities is not located, it will bring greater economic and human resource consumption. Using conventional image feature analysis methods, it is difficult to extract and identify the features of electrical equipment and the features of abnormal heating areas. Combining deep learning and computer vision algorithms with traditional image processing algorithms to automatically analyze and detect infrared data of electrical equipment and abnormal heating can significantly improve the efficiency of infrared image detection. Integrating this algorithm into the main equipment in the substation control room, by detecting the infrared thermal images of electrical equipment collected in real time, an alarm is given in a timely manner when an abnormal situation occurs, and the subsequently detected abnormal images are added to the dataset for continuous training to achieve higher detection accuracy and achieve more accurate intelligent inspection. For the electrical equipment infrared fault recognition method with dual supervision signals, firstly, the Slic superpixel segmentation method is used to merge similar pixel blocks; secondly, the brightness information in the improved HSV space is used to judge the abnormal temperature area of the equipment, and then the connected area where the abnormal temperature area is located and the corresponding equipment are segmented; finally, feature extraction is performed on the electrical equipment infrared fault image based on the GoogLeNet convolutional neural network, and then two supervision signals, namely softmax loss and center loss, are used to supervise the training of the extracted features.

[0003] The disadvantages of the prior art are as follows: The electrical equipment infrared fault recognition method with dual supervision signals cannot exclude the influence of interference points; the algorithm complexity is relatively high; the recognition accuracy is relatively low. Currently, for the elimination of interference points, mainly analyze whether the heating point is within the equipment contour. As long as it is within the equipment contour, it is regarded as a valid heating point, which will undoubtedly reduce the detection accuracy. Summary of the Invention

[0004] The object of the present invention is to overcome the deficiencies of the prior art and provide a method for detecting electrical equipment faults based on infrared images. First, the infrared image is preliminarily segmented by the OTSU algorithm to extract the contour of the electrical equipment. Secondly, the RGB of the original infrared image is converted into the HSV color space to extract the information of the heat source points. Then, the Resnet-34 network and the method of transfer learning are used to locate the heat source points. Finally, in the test stage, an improved watershed algorithm is used to segment the equipment, and finally the abnormal area and fault type of the faulty equipment are extracted, which can minimize interference and improve the detection accuracy.

[0005] The object of the present invention is achieved by the following technical solutions:

[0006] A method for detecting electrical equipment faults based on infrared images, comprising:

[0007] Step 1): Obtain the first-dimensional infrared image and the second-dimensional infrared image of the electrical equipment, and the infrared image planes corresponding to the first dimension and the second dimension are perpendicular to each other;

[0008] Step 2): Preliminarily segment the infrared image by the OTSU algorithm to extract the infrared image A of the electrical equipment contour in the first dimension and the infrared image B of the electrical equipment contour in the second dimension;

[0009] Step 3): Establish a space rectangular coordinate system (X, Y, Z) based on the infrared image A and the infrared image B. Among them, the coordinates of the area covered by the infrared image A are (Y1 - Y2, Z1 - Z2), and the coordinates of the area covered by the infrared image B are (Y1 - Y2, X1 - X2);

[0010] Step 4): Convert the RGB of the infrared image A and the infrared image B into the HSV color space to extract the information of the heat source points;

[0011] Step 5): Use the Resnet-34 network and the method of transfer learning to locate the heat source points; among them, the coordinates of the heat source points in the infrared image A are (Yn, Zn), and the coordinates of the heat source points in the infrared image B are (Ym, Xm), where n and m are the numbers of heat source points in the infrared image A and the infrared image B respectively;

[0012] Step 6): Interference point investigation, including:

[0013] When using the infrared image A as the recognition model, exclude and the points, and the Yn after exclusion is denoted as Yn';

[0014] Based on the coordinates (Ym, Xm) of the heat source points located in the infrared image B, exclude the points, and the finally obtained points are the effective heat source points;

[0015] Or

[0016] When using the infrared image B as the recognition model, exclude and the points. After exclusion, the Ym is denoted as Ym'.

[0017] Based on the coordinates (Yn, Zn) of the heat-generating points located in the infrared image A, exclude the points. The finally obtained points are the effective heat-generating points.

[0018] Step 7): Fault testing. Use the watershed algorithm to segment the device, and finally extract the abnormal area and fault type of the faulty device. Among them, a top-hat transformation and a closing operation are added to the watershed algorithm;

[0019] The top-hat transformation is used to remove small dots and burrs of the effective heat-generating points, and the closing operation is used to fill some small holes and connect the contours of the electrical devices in the original image to avoid missing the contours of the extracted electrical devices.

[0020] Compared with the traditional fault detection based on image recognition technology, in this application, the infrared images in different dimensions are mutually corrected, and basically the interference heat-generating points in the cases of dislocation, overlap, etc. can be excluded, reducing interference and improving the accuracy of image recognition. At the same time, the watershed algorithm is introduced to segment the device, and a top-hat transformation and a closing operation are added, further improving the fineness of image recognition. That is to say, in this application, a correction is performed from the "macro" and "micro" perspectives, so that the accuracy of the finally obtained image recognition is improved, and a fault recognition rate of more than 99% can be achieved.

[0021] Optionally, the first dimension and the second dimension are the vertical dimension and the horizontal dimension respectively. Preferably, the infrared images are extracted at the angles of the vertical dimension and the horizontal dimension. In this combination, the installation of the device is simpler, and in terms of operation, its implementation is more reliable.

[0022] Optionally, step 1) further includes an infrared image enhancement process, and the first-dimension infrared image and the second-dimension infrared image are processed as follows respectively:

[0023] 1-1): Translate the image in a certain way on the image plane;

[0024] 1-2): Flip the image along the horizontal or vertical direction;

[0025] 1-3): Rotate the original image by 45 degrees, 90 degrees, 180 degrees, and 270 degrees respectively, and each original image will obtain 4 rotated images;

[0026] 1-4): Enhance the image contrast, and histogram equalization can also be used;

[0027] 1 - 5): Increase the brightness of the entire image.

[0028] In this application, image enhancement technology is used to expand the recognition object. It can be imagined that in traditional fault recognition, areas with relatively small heat spots or areas with insignificant heat generation may be ignored. In this application, image enhancement technology is used to magnify these details, thereby solving the problem of missed fault reports.

[0029] Optionally, the extraction of the electrical equipment contour in step 2) includes:

[0030] 2 - 1): Construct an image gradient image;

[0031] 2 - 2): Generate n initial water - filling regions through rules, prior knowledge, or local gradient minima;

[0032] 2 - 3): Add water to the water - filling regions. When two water - filling regions are about to merge, record the boundary at this time;

[0033] 2 - 4): The algorithm ends when the image edge is completely divided into n independent regions;

[0034] Erosion: Erode A with structure B. An origin needs to be defined in B. When the origin of B is translated to the pixel (x, y) of image A, if B is completely contained in the overlapping region of image A at (x, y), then assign the pixel (x, y) of the output image to 1; otherwise, assign it to 0. Here, A represents the image to be processed, and B is used to process A, and B is called a structuring element.

[0035] Dilation: Dilate A with structure B. Translate the origin of the structuring element B to the position of the image pixel (x, y). If the intersection of B and A at the image pixel (x, y) is not empty, then assign the pixel (x, y) of the output image to 1; otherwise, assign it to 0.

[0036] Optionally, the rule means that pixels that are spatially adjacent and have similar gray values are divided into one water - filling region.

[0037] Optionally, step 4) is specifically:

[0038] 4 - 1): Conversion from RGB to HSV:

[0039] Let (r, g, b) be the red, green, and blue coordinates of a color respectively, and their values are real numbers between 0 and 1. Find the values of (h, s, v) in the HSV space, calculated as:

[0040]

[0041] Let max be equivalent to the maximum of r, g, and b, and let min be equal to the minimum of these values;

[0042] H = H * 60

[0043] If H < 0, H = H + 360

[0044] Where h ∈ [0, 360) is the hue angle of the angle;

[0045] V = max(R, G, B)

[0046] S = (max - min) / max

[0047] Where s, v ∈ [0, 1] are saturation and brightness;

[0048] 4 - 2): Extract the target area from the image using the HSV components, and extract the image of the yellow part close to white.

[0049] Optionally, the HSV components are set as:

[0050] h max = 180, h min = 0, s max = 100, s min = 0, v max = 255, v min = 221.

[0051] Optionally, step 5) is:

[0052] 5 - 1): Use pytorch to build a ResNet network and train it based on the method of transfer learning. By obtaining the pre - trained ResNet weight file, then putting the self - built dataset into the model for continued training, finally getting the training result, and then putting it into the test set for fault prediction;

[0053] 5 - 2): After the infrared image dataset training is completed, add the watershed algorithm and the HSV color space temperature discrimination method in the prediction link, combine the deep learning method recognition into this link, input an infrared image, and the final output result includes the fault type, and draw the device contour and locate the position of the heat source in the output image.

[0054] Optionally, the ResNet weight file is a network layer with 34 layers.

[0055] The beneficial effects of the present invention are:

[0056] 1) Add an opening operation and a closing operation to the watershed algorithm. The opening operation will remove isolated small dots and burrs in the original image. There are some interfering small dots and interfering points with abnormal brightness in the background of the original image. These interfering points are removed through the opening operation, which is convenient for improving the accuracy of abnormally heated points. The closing operation can fill some small holes and connect the contours of electrical equipment in the original image to avoid missing the extracted contours of electrical equipment.

[0057] 2) Perform data augmentation on the acquired infrared images, increasing the amount of the dataset and also improving the fault recognition accuracy and robustness to a certain extent.

[0058] 3) Use the Resnet algorithm in the training stage. After the training is completed, add the watershed algorithm and the HSV color space in the testing stage to extract temperature abnormal points, which can not only obtain the fault type, but also segment the faulty equipment, identify the fault type, and locate the abnormally heated points.

[0059] 4) Use infrared images of different dimensions and mutually correct the heated points in different dimensions to eliminate the interference caused by the coincidence of heated points and improve the accuracy of fault determination.

[0060] 5) Convert the RGB color space of the original image to the HSV color space, extract the temperature abnormal area of the fault point, use the watershed algorithm to segment the target equipment, avoiding the influence of external interfering points on fault detection; use the improved Resnet algorithm to train the processed infrared images, and finally jointly use the Resnet algorithm and the watershed algorithm to detect faults in electrical equipment and locate the heated points of electrical equipment. This effectively reduces the model complexity and shortens the training time and parameter calculation. Description of the Drawings

[0061] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail with reference to the accompanying drawings, where:

[0062] Figure 1 is the flow chart of the present invention;

[0063] Figure 2 is the schematic diagram of an embodiment. Detailed Embodiments

[0064] The technical solutions of the present invention will be further described in detail below with reference to specific embodiments, but the protection scope of the present invention is not limited to the following.

[0065] A method for detecting faults in electrical equipment based on infrared images includes:

[0066] Step 1): Obtain the first-dimension infrared image and the second-dimension infrared image of the electrical equipment, where the infrared image planes corresponding to the first dimension and the second dimension are perpendicular to each other;

[0067] Step 2): Perform preliminary segmentation on the infrared images through the OTSU algorithm, and extract the infrared image A of the electrical equipment contour in the first dimension and the infrared image B of the electrical equipment contour in the second dimension;

[0068] Step 3): Establish a rectangular coordinate system (X, Y, Z) based on the infrared image A and the infrared image B. Among them, the coordinates of the area covered by the infrared image A are (Y1 - Y2, Z1 - Z2), and the coordinates of the area covered by the infrared image B are (Y1 - Y2, X1 - X2). Take the infrared image A and the infrared image B as a plane respectively, so that the infrared image A and the infrared image B are perpendicular to each other, thereby forming a three-dimensional coordinate system.

[0069] Step 4): Convert the RGB of the infrared image A and the infrared image B to the HSV color space, and extract the information of the heat source points;

[0070] Step 5): Use the Resnet-34 network and the method of transfer learning to locate the heat source points; among them, the coordinates of the heat source points in the infrared image A are (Yn, Zn), and the coordinates of the heat source points in the infrared image B are (Ym, Xm), where n and m are the numbers of the heat source points in the infrared image A and the infrared image B respectively;

[0071] Step 6): Interference point investigation, including:

[0072] When taking the infrared image A as the recognition model, exclude and the points, and the Yn after exclusion is denoted as Yn';

[0073] Based on the coordinates (Ym, Xm) of the heat source points located in the infrared image B, exclude the points, and the finally obtained points are the effective heat source points;

[0074] Or,

[0075] When taking the infrared image B as the recognition model, exclude and the points, and the Ym after exclusion is denoted as Ym';

[0076] Based on the coordinates (Yn, Zn) of the heat source points located in the infrared image A, exclude the points, and the finally obtained points are the effective heat source points;

[0077] Step 7): Fault testing. The device is segmented using the watershed algorithm, and finally the abnormal area and fault type of the faulty device are extracted. Among them, an opening operation and a closing operation are added to the watershed algorithm.

[0078] The opening operation is used to remove small dots and burrs of effective heat sources, and the closing operation is used to fill some small holes and connect the contours of electrical devices in the original image to avoid missing the contours of the extracted electrical devices.

[0079] Optionally, in some embodiments, the first dimension and the second dimension are the vertical dimension and the horizontal dimension respectively. Preferably, the infrared image is extracted at an angle of the vertical dimension and the horizontal dimension. In this combination, the installation of the device is simpler, and in terms of operation, its implementation is more reliable. The infrared image is extracted by using an ultra-high-definition thermal imaging camera, so that the obtained original infrared image itself has a high accuracy, which can improve the accuracy of recognition.

[0080] Optionally, in some embodiments, step 1) further includes an infrared image enhancement process, and the following processes are respectively performed on the first-dimension infrared image and the second-dimension infrared image:

[0081] 1-1): Translate the image in a certain way on the image plane.

[0082] 1-2): Flip the image along the horizontal or vertical direction.

[0083] 1-3): Rotate the original image by 45 degrees, 90 degrees, 180 degrees, and 270 degrees respectively, and each original image will obtain 4 rotated images.

[0084] 1-4): Enhance the image contrast, and histogram equalization can also be used.

[0085] 1-5): Increase the brightness of the entire image.

[0086] In this application, using image enhancement technology, the recognition object is expanded. It can be imagined that in traditional fault recognition, areas with relatively small heat sources or areas with insignificant heat generation may be ignored. In this application, image enhancement technology is used to magnify these details, thereby solving the problem of missed fault reports.

[0087] Step 2): Infrared image fault point and area segmentation. Use an improved watershed algorithm to mark the contour of the faulty device. In step 2), for the augmented data set obtained in step 1), an improved watershed algorithm is used to mark the contour of the infrared image. For the infrared image, there are many interference factors in the background. A closing operation and an opening operation are added to the watershed algorithm to eliminate external interference points of the device without losing detailed feature information.

[0088] Optionally, in some embodiments, the extraction of the electrical device contour in step 2) includes:

[0089] 2-1): Construct an image gradient image;

[0090] 2-2): Generate n initial water injection regions through rules, prior knowledge, or local gradient minima;

[0091] 2-3): Add water to the water injection regions. When two water injection regions are about to merge, record the boundary at this time;

[0092] 2-4): The algorithm ends when the image edge is completely divided into n independent regions;

[0093] Erosion: Erode A with structure B. An origin needs to be defined in B. When the origin of B is translated to the pixel (x, y) of image A, if B is completely contained in the overlapping region of image A at (x, y), then assign the pixel (x, y) of the output image to 1; otherwise, assign it to 0. Here, A represents the image to be processed, and B is used to process A, and B is called a structuring element.

[0094] Dilation: Dilate A with structure B. Translate the origin of the structuring element B to the position of the image pixel (x, y). If the intersection of B and A at the image pixel (x, y) is not empty, then assign the pixel (x, y) of the output image to 1; otherwise, assign it to 0.

[0095] Optionally, in some embodiments, the rule means that pixels that are spatially adjacent and have similar gray values are divided into one water injection region.

[0096] Step 4): Extract the heat source in the HSV color space. Convert the RGB color space of the original image to the HSV color space to facilitate the intuitive extraction of the color and brightness information of the infrared image. Optionally, in some embodiments, step 4) is specifically:

[0097] 4-1): Conversion from RGB to HSV:

[0098] Let (r, g, b) be the red, green, and blue coordinates of a color respectively, and their values are real numbers between 0 and 1. Find the (h, s, v) values in the HSV space, calculated as:

[0099]

[0100] Let max be equivalent to the maximum of r, g, and b, and let min be equal to the minimum of these values;

[0101] H = H * 60

[0102] if H < 0, H = H + 360

[0103] where h ∈ [0, 360) is the hue angle of the color;

[0104] V = max(R, G, B)

[0105] S = (max - min) / max

[0106] where s, v ∈ [0, 1] are the saturation and brightness;

[0107] 4 - 2): Extract the target area from the image using the HSV components, and extract the image of the yellow part close to white.

[0108] Optionally, the HSV components are set as:

[0109] h max = 180, h min = 0, s max = 100, s min = 0, v max = 255, v min = 221.

[0110] 4 - 2) HSV is proposed to digitalize the image, which is quite different from the way the human eye understands the image. The image is converted from the RGB color space to the HSV color space to better perceive the image color, and the HSV components are used to extract the region of interest from the image. To extract the image of the yellow part close to white, by referring to the HSV color look - up table, the three component parameters of HSV are set as shown in Table 1:

[0111] Table 1 HSV Color Look - up Table

[0112]

[0113] Step 5): ResNet is combined with the improved watershed algorithm to identify faults in the infrared images of electrical equipment. When using ResNet combined with the improved watershed algorithm to identify faults in infrared images, optionally, in some embodiments, Step 5) is as follows:

[0114] 5 - 1): Use pytorch to build a ResNet network and train it based on the method of transfer learning. By obtaining the pre - trained ResNet weight file, then putting the self - built dataset into the model for further training, finally obtaining the training result, and then putting it into the test set for fault prediction;

[0115] 5-2): After the infrared image dataset training is completed, the watershed algorithm and the HSV color space temperature discrimination method are added in the prediction link. The deep learning method recognition is combined into this link. An infrared image is input, and the final output result includes the fault type. The device contour and the position of the heat source are drawn in the output image.

[0116] Optionally, the ResNet weight file is a 34-layer network layer.

[0117] The RGB color space of the original image is converted to the HSV color space, the abnormal temperature area of the fault point is extracted, and the watershed algorithm is used to segment the target device, avoiding the influence of external interference points of the device on fault detection; the improved Resnet algorithm is used to train the processed infrared image, and finally the Resnet algorithm and the watershed algorithm are combined to detect the faults of electrical equipment and locate the heat sources of electrical equipment. The model complexity is effectively reduced, and the training time and parameter calculation are reduced.

[0118] The above are only the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be within the scope of the concept described herein, through the above teachings or the technology or knowledge in related fields. Any changes and modifications made by those skilled in the art without departing from the spirit and scope of the present invention shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for detecting electrical equipment faults based on infrared images, characterized in that, It includes the following steps: 1): Obtain the first - dimension infrared image and the second - dimension infrared image of the electrical equipment, where the infrared image planes corresponding to the first dimension and the second dimension are perpendicular to each other; 2): Perform preliminary segmentation on the infrared images through the OTSU algorithm, and extract the infrared image A of the electrical equipment contour in the first dimension and the infrared image B of the electrical equipment contour in the second dimension; 3): Establish a spatial rectangular coordinate system (X, Y, Z) based on the infrared image A and the infrared image B. Among them, the coordinates of the area covered by the infrared image A are (Y1 - Y2, Z1 - Z2), and the coordinates of the area covered by the infrared image B are (Y1 - Y2, X1 - X2); 4): Convert the RGB of the infrared image A and the infrared image B to the HSV color space and extract the information of the heat - generating points; 5): Use the Resnet - 34 network and the method of transfer learning to locate the heat - generating points; among them, the coordinates of the heat - generating points in the infrared image A are (Yn, Zn), and the coordinates of the heat - generating points in the infrared image B are (Ym, Xm), where n and m are the numbers of heat - generating points in the infrared image A and the infrared image B respectively; 6): Troubleshooting of interference points, including: When using the infrared image A as the recognition model, exclude and points. Denote the Yn after exclusion as Yn'. Based on the coordinates (Ym, Xm) of the heat source points located in the infrared image B, exclude Yn’ the points in the set {Ym}, and the finally obtained points are the effective heat source points; Or, When using the infrared image B as the recognition model, exclude and the points. After exclusion, the Ym is denoted as Ym'. Based on the coordinates (Yn, Zn) of the heat source points located in the infrared image A, exclude the points in the set {Yn}, and the finally obtained points are the effective heat source points; 7): Fault testing, use the watershed algorithm to segment the equipment, and finally extract the abnormal area and fault type of the faulty equipment, where an opening operation and a closing operation are added to the watershed algorithm; The opening operation is used to remove small dots and burrs of effective heat - generating points, and the closing operation is used to fill some small holes and connect the contours of the electrical equipment in the original image to avoid missing the contours of the extracted electrical equipment; Step 1) also includes the following processing for the first - dimension infrared image and the second - dimension infrared image respectively: 1 - 1): Translate the image in a certain way on the image plane; 1 - 2): Flip the image along the horizontal or vertical direction; 1 - 3): Rotate the original image by 45 degrees, 90 degrees, 180 degrees, and 270 degrees respectively, and each original image will obtain 4 rotated images; 1 - 4): Enhance the image contrast; 1 - 5): Increase the brightness of the whole image; The extraction of the electrical equipment contour in step 2) includes: 2 - 1): Construct an image gradient image; 2 - 2): Generate n initial water - filling areas through rules, prior knowledge or local gradient minima; 2 - 3): Add water to the water - filling areas. When two water - filling areas are about to merge, record the boundary at this time; 2 - 4): The algorithm ends when the image edge is completely segmented into N independent areas; Step 5) is: 5 - 1): Use pytorch to build a ResNet network and train it based on the method of transfer learning. By obtaining the pre - trained ResNet weight file, then put the self - built data set into the model for further training, finally get the training result, and then put it into the test set for fault prediction; 5 - 2): After the infrared image data set training is completed, add the watershed algorithm and the HSV color space temperature discrimination method in the prediction link, combine the deep - learning method recognition into this link, input an infrared image, and the final output result includes the fault type, and draw the equipment contour and the position of the located heat - generating points in the output image.

2. The method for detecting electrical equipment faults based on infrared images according to claim 1, characterized in that, The first dimension and the second dimension are the vertical dimension and the horizontal dimension respectively.

3. The method for detecting electrical equipment faults based on infrared images according to claim 2, characterized in that, The rule means that pixels that are spatially adjacent and have similar gray values are divided into a water filling region.

Images