Insulator fault identification method and device based on image fusion

Abstract

The invention provides an insulator fault identification method and device based on image fusion. The method comprises the following steps: acquiring an infrared image and a visible light image of an insulator; respectively preprocessing the infrared image and the visible light image to obtain a preprocessed infrared image and a preprocessed visible light image; performing fusion processing according to the preprocessed infrared image and the preprocessed visible light image to obtain a fused image; and performing fault identification on the insulator according to the fused image to obtain a fault identification result. According to the method, the infrared image and the visible light image of the insulator are acquired and are preprocessed respectively to obtain the preprocessed infrared image and the preprocessed visible light image, then the preprocessed infrared image and the preprocessed visible light image are fused to obtain the fused image, and finally fault identification is performed on the insulator according to the fused image. Therefore, the comprehensiveness of the image information is improved, and the accuracy of insulator fault identification is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of insulator fault identification, and also to an insulator fault identification method and device based on image fusion. Background Art

[0002] Transmission line inspection is crucial for ensuring the safe and reliable operation of power grids, and insulator inspection is a key component of this task. To ensure the safe and reliable operation of transmission lines, regular inspections of insulators along these lines are essential to identify faulty insulators and promptly repair them. Therefore, insulator fault identification technology is a key aspect of power system operation and maintenance. Promptly detecting and accurately identifying insulator faults is crucial to ensuring safe system operation. Fault identification of insulators on overhead transmission lines is often performed manually. With the development of new technologies and new developments, drones are now being used to identify insulator faults on overhead transmission lines, making power inspections more automated and intelligent. However, existing methods that use a single infrared or visible light image source for insulator fault identification suffer from limitations such as incomplete information, limited data volume, difficulty interpreting infrared images, limited lighting conditions, and difficulty detecting specific faults. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an insulator fault identification method and device based on image fusion, so as to improve the accuracy of insulator fault identification.

[0004] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0005] A first aspect of the present invention provides an insulator fault identification method based on image fusion, comprising:

[0006] Acquire infrared and visible light images of insulators;

[0007] Preprocessing the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image;

[0008] Performing fusion processing on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image;

[0009] Fault identification is performed on the insulator according to the fused image to obtain a fault identification result.

[0010] Optionally, preprocessing the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image includes:

[0011] performing filtering processing on the infrared image and the visible light image respectively to obtain a filtered infrared image and a filtered visible light image;

[0012] performing enhancement processing on the filtered infrared image and the filtered visible light image respectively to obtain an enhanced infrared image and an enhanced visible light image;

[0013] The enhanced infrared image and the enhanced visible light image are respectively registered to obtain a preprocessed infrared image and a preprocessed visible light image.

[0014] Optionally, filtering the infrared image and the visible light image respectively to obtain a filtered infrared image and a filtered visible light image includes:

[0015] Obtain a preset window; the size of the preset window is N×N;

[0016] Performing filtering according to the preset window, the infrared image, the visible light image, and g(x,y)=median{f(x+i,y+j)}, (i,j)∈N to obtain a filtered infrared image and a filtered visible light image;

[0017] Where g(x,y) is the pixel value of the filtered image at position (x,y), where the filtered image is a filtered infrared image or a filtered visible light image; f(x+i,y+j) is the pixel value in the window N centered at (x,y) in the original image, where the original image is an infrared image or a visible light image; N is the preset window, which is an odd number; and median is the median of all pixel values in the window after sorting.

[0018] Optionally, performing enhancement processing on the filtered infrared image and the filtered visible light image respectively to obtain an enhanced infrared image and an enhanced visible light image includes:

[0019] performing grayscale normalization processing on the filtered infrared image and the filtered visible light image respectively to obtain a grayscale infrared image and a grayscale visible light image;

[0020] According to the grayscale infrared image, the grayscale visible light image and Obtain infrared gray level probability and visible light gray level probability;

[0021] According to the infrared gray level probability, the visible light gray level probability and Obtain infrared cumulative probability and visible light cumulative probability;

[0022] According to the infrared cumulative probability, the visible light cumulative probability and q k =round[(L-1)·Sk ], get the new infrared gray level and the new visible light gray level;

[0023] Obtaining an enhanced infrared image and an enhanced visible light image according to the new infrared grayscale and the new visible light grayscale;

[0024] Among them, n k is the number of pixels at each gray level in the original image, k = 0, 1, ..., L-1, L is the total number of gray levels, M and O are the width and height of the original image, f(i, j) is the gray value of the pixel at the i-th row and j-th column in the original image, δ(a, b) is the exponential function, p k is the probability that the gray level is the gray level of the pixel in the original image, where the original image is a grayscale infrared image or a grayscale visible light image; S k is the infrared cumulative probability or the visible light cumulative probability; q k is the new infrared grayscale or the new visible light grayscale, round is rounded to the nearest integer; (L-1) is the value of S k Normalized to the range [0, L-1].

[0025] Optionally, performing registration processing on the enhanced infrared image and the enhanced visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image includes:

[0026] Extracting feature points from the enhanced infrared image and the enhanced visible light image to obtain feature points;

[0027] Matching the feature points to obtain a corresponding relationship between the enhanced infrared image and the enhanced visible light image;

[0028] A preprocessed infrared image and a preprocessed visible light image are obtained according to the enhanced infrared image, the enhanced visible light image and the corresponding relationship.

[0029] Optionally, performing fusion processing on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image includes:

[0030] According to the pre-processed infrared image and I 01 =G11(I1)+G12(I1)+G13(I1), I′ 01 =G14(I 01 )+G14(I 01 )×I 01 、f1=G15(I′ 01 )+G15(I′ 01 )×I 01 +G15(I′ 01 )×I′ 01 , get the infrared image output features;

[0031] According to the preprocessed visible light image and I 02 =G21(I2)+G22(I2)+G23(I2), I′ 02 =G24(I 02 )+G24(I 02 )×I 02 、f2=G25(I′ 02 )+G25(I′ 02 )×I 02 +G25(I′ 02 )×I′ 02 , obtain the visible light image output features;

[0032] According to the infrared image output feature, the visible light image output feature and f3=G7(G6(f c ))+G7(G6(f c ))×f1+G7(G6(f c ))×f2, to obtain enhanced fusion features;

[0033] Performing dimensionality reduction processing on the enhanced fusion features to obtain a fused image;

[0034] Among them, f c =Concat(f1,f2);

[0035] Among them, G11 and G21 are 1×1 convolution kernels, G12 and G22 are 3×3 convolution kernels, G13 and G23 are 5×5 convolution kernels, the input dimension is 1 and the output dimension is 64; G14 is a 5×5 convolution kernel, G24 is a 7×7 convolution kernel, the input dimension and output dimension are 64; G15 and G25 are both 3×3 convolution kernels, the input dimension and output dimension are 64; G6 and G7 are both 1×1 convolution kernels, the input dimension is 128 and the output dimension is 64; I 01 , I 02 are the multi-scale features of pre-processed infrared images and the multi-scale features of pre-processed visible light images, I′ 01 , I′ 02 are the deep features of the preprocessed infrared image and the deep features of the preprocessed visible light image, respectively. f1 and f2 are the output features of the infrared image and the output features of the visible light image, respectively. c is the fusion feature, and f3 is the enhanced fusion feature.

[0036] Optionally, performing fault identification on the insulator according to the fused image to obtain a fault identification result includes:

[0037] performing fault identification on the insulator according to the fused image to obtain an initial identification result;

[0038] Screening is performed based on the initial identification result and preset screening conditions to obtain a fault identification result.

[0039] A second aspect of the present invention provides an insulator fault identification device based on image fusion, comprising:

[0040] an acquisition module, used for acquiring infrared images and visible light images of insulators;

[0041] The processing module is used to preprocess the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image; perform fusion processing on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image; and perform fault identification on the insulator based on the fused image to obtain a fault identification result.

[0042] According to a third aspect of the present invention, a computing device is provided, comprising: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to the first aspect is executed.

[0043] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the method described in the first aspect.

[0044] The above solution of the present invention includes at least the following beneficial effects:

[0045] The above-mentioned scheme of the present invention obtains the infrared image and visible light image of the insulator, and preprocesses them respectively to obtain a preprocessed infrared image and a preprocessed visible light image, then fuses the preprocessed infrared image and the preprocessed visible light image to obtain a fused image, and finally performs fault identification on the insulator based on the fused image to obtain a fault identification result, thereby improving the comprehensiveness of the image information and being conducive to improving the accuracy of insulator fault identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of an insulator fault identification method based on image fusion in an embodiment of the present invention;

[0047] Figure 2 It is a structural diagram of an insulator fault identification device based on image fusion in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0049] like Figure 1 As shown, an embodiment of the present invention proposes an insulator fault identification method based on image fusion, comprising the following steps:

[0050] Step 101, acquiring an infrared image and a visible light image of an insulator;

[0051] Step 102, preprocessing the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image;

[0052] Step 103, performing fusion processing on the pre-processed infrared image and the pre-processed visible light image to obtain a fused image;

[0053] Step 104 : performing fault identification on the insulator according to the fused image to obtain a fault identification result.

[0054] The insulator fault identification method based on image fusion of an embodiment of the present invention obtains an infrared image and a visible light image of the insulator, and preprocesses them respectively to obtain a preprocessed infrared image and a preprocessed visible light image, then fuses the preprocessed infrared image and the preprocessed visible light image to obtain a fused image, and finally performs fault identification on the insulator based on the fused image to obtain a fault identification result, which improves the comprehensiveness of image information and is conducive to improving the accuracy of insulator fault identification.

[0055] In an optional embodiment of the present invention, the infrared image of the insulator obtained in step 101 is acquired by an infrared camera; and the visible light image is acquired by an ordinary camera or a digital camera.

[0056] In an optional embodiment of the present invention, step 102 includes:

[0057] Step 1021: Filter the infrared image and the visible light image to obtain a filtered infrared image and a filtered visible light image respectively.

[0058] Step 1022: performing enhancement processing on the filtered infrared image and the filtered visible light image to obtain an enhanced infrared image and an enhanced visible light image respectively;

[0059] Step 1023 : performing registration processing on the enhanced infrared image and the enhanced visible light image respectively to obtain a pre-processed infrared image and a pre-processed visible light image.

[0060] Specifically, in this embodiment, by filtering the infrared image and the visible light image separately, the noise in the image can be reduced, the image can be made clearer, and the effect of subsequent processing can be improved; through enhancement processing, the details and visualization effects in the image can be enhanced, and through registration processing, the deviation in the correspondence between the feature points or feature areas of the infrared image and the visible light image caused by the spatial differences between the infrared acquisition device and the visible light acquisition device can be eliminated, thereby improving the subsequent fusion quality and accuracy, and thus improving the accuracy of fault identification.

[0061] In an optional embodiment of the present invention, step 1021 includes:

[0062] Step 10211: Obtain a preset window; the size of the preset window is N×N;

[0063] Step 10212 , performing filtering processing based on the preset window, the infrared image, the visible light image, and g(x,y)=median{f(x+i,y+j)}, (i,j)∈N, to obtain a filtered infrared image and a filtered visible light image;

[0064] Where g(x,y) is the pixel value of the filtered image at position (x,y), where the filtered image is a filtered infrared image or a filtered visible light image; f(x+i,y+j) is the pixel value in the window N centered at (x,y) in the original image, where the original image is an infrared image or a visible light image; N is the preset window, which is an odd number; and median is the median of all pixel values in the window after sorting.

[0065] Specifically, in this embodiment, by filtering the image, noise in the image is reduced, making the image clearer, which is beneficial to improving the effect of subsequent processing.

[0066] In an optional embodiment of the present invention, step 1022 includes:

[0067] Step 10221: performing grayscale normalization processing on the filtered infrared image and the filtered visible light image, respectively, to obtain a grayscale infrared image and a grayscale visible light image;

[0068] Specifically, since the grayscale value distribution of insulator images taken under different lighting conditions may vary significantly (such as 0-100 or 150-255), after normalization, the pixel values of all images are mapped to a unified range (such as [0,1] or [-1,1]), eliminating the influence of external factors such as lighting and equipment parameters on the image, which is conducive to improving the efficiency and accuracy of subsequent processing. Here, you can use The grayscale normalization processing of the filtered infrared image and the filtered visible light image is performed respectively in the manner of , where H3 is the grayscale infrared image or the grayscale visible light image, H is the original pixel value of the grayscale infrared image or the original pixel value of the grayscale visible light image, and H min is the original pixel value of the grayscale infrared image or the minimum grayscale value of the grayscale visible light image, H max is the original pixel value of the grayscale infrared image or the maximum grayscale value of the grayscale visible light image, and (x′, y′) is the target range, such as [0, 1] or [-1, 1].

[0069] Step 10222: Based on the grayscale infrared image, the grayscale visible light image and Obtain infrared gray level probability and visible light gray level probability;

[0070] Step 10223, based on the infrared gray level probability, the visible light gray level probability and Obtain infrared cumulative probability and visible light cumulative probability;

[0071] Step 10224: Based on the infrared cumulative probability, the visible light cumulative probability and q k =round[(L-1)·S k ], get the new infrared gray level and the new visible light gray level;

[0072] Step 10225: Obtain an enhanced infrared image and an enhanced visible light image according to the new infrared grayscale and the new visible light grayscale.

[0073] Among them, n k is the number of pixels at each gray level in the original image, k = 0, 1, ..., L-1, L is the total number of gray levels, M and O are the width and height of the original image, f(i, j) is the gray value of the pixel at the i-th row and j-th column in the original image, δ(a, b) is the exponential function, p k is the probability that the gray level is the gray level of the pixel in the original image, where the original image is a grayscale infrared image or a grayscale visible light image; S k is the infrared cumulative probability or the visible light cumulative probability; q k is the new infrared grayscale or the new visible light grayscale, round is rounded to the nearest integer; (L-1) is the value of Sk Normalized to the range [0, L-1].

[0074] Specifically, if the original image is an 8-bit grayscale image, L is 256.

[0075] In an optional embodiment of the present invention, step 1023 includes:

[0076] Step 10231: extract feature points from the enhanced infrared image and the enhanced visible light image to obtain feature points.

[0077] Specifically, feature point extraction can be performed in the following manner: a multi-scale Gaussian pyramid is established for the enhanced infrared image and the enhanced visible light image respectively, and each layer of the image is generated by blurring the Gaussian kernels of different scales to simulate the visual effect from near to far when the human eye observes an object, and the Gaussian blurred images of adjacent scales are subtracted to generate a Gaussian difference pyramid for detecting extreme points; in the Gaussian difference pyramid, each pixel point is compared with 26 neighboring points of the same scale and adjacent scales, and if it is an extreme point, it is retained. By fitting a three-dimensional quadratic function, the position error of the extreme point is eliminated, the key point positioning accuracy is improved, and noise interference is reduced; the gradient amplitude and direction are calculated in the neighborhood of the key point to generate a 36-dimensional direction histogram, and the direction of the histogram peak is selected as the main direction. If there are multiple peaks, multiple key points, i.e., feature points, are generated.

[0078] Step 10232: Match the feature points to obtain a correspondence between the enhanced infrared image and the enhanced visible light image.

[0079] Specifically, you can Calculate the distance between the feature points in the enhanced infrared image and the feature points in the enhanced visible light image. If the distance is less than the preset distance, the two feature points are matched feature points. Where d(e,f) is the distance between the feature point e in the enhanced infrared image and the feature point f in the enhanced visible light image, v ei is the i-th descriptor of feature point e in the enhanced infrared image, v fi is the i-th descriptor of feature point f in the enhanced visible light image, and c is the dimension of the descriptor. Through the matched feature points, a projective transformation is used to determine the geometric transformation relationship, i.e., the correspondence relationship, between the enhanced infrared image and the enhanced visible light image.

[0080] Step 10233: Obtain a preprocessed infrared image and a preprocessed visible light image according to the enhanced infrared image, the enhanced visible light image, and the corresponding relationship.

[0081] Specifically, the preprocessed infrared image includes an enhanced infrared image with feature points corresponding to the enhanced visible light image, and the preprocessed visible light image includes an enhanced visible light image with feature points corresponding to the enhanced infrared image. By registering the infrared and visible light images, the images are aligned, feature matching is established, geometric consistency is maintained, and information is supplemented and enhanced, thereby improving the quality and accuracy of the fused image and, in turn, enhancing subsequent fault identification.

[0082] In an optional embodiment of the present invention, step 103 includes:

[0083] Step 1031, based on the pre-processed infrared image and I 01 =G11(I1)+G12(I1)+G13(I1), I′ 01 =G14(I 01 )+G14(I 01 )×I 01 、f1=G15(I′ 01 )+G15(I′ 01 )×I 01 +G15(I′ 01 )×I′ 01 , get the infrared image output features;

[0084] Step 1032: Based on the pre-processed visible light image and I 02 =G21(I2)+G22(I2)+G23(I2), I′ 02 =G24(I 02 )+G24(I 02 )×I 02 、f2=G25(I′ 02 )+G25(I′ 02 )×I 02 +G25(I′ 02 )×I′ 02 , obtain the visible light image output features;

[0085] Step 1033: Based on the infrared image output feature, the visible light image output feature, and f3=G7(G6(f c ))+G7(G6(f c ))×f1+G7(G6(f c ))×f2, to obtain enhanced fusion features;

[0086] Step 1034: performing dimensionality reduction processing on the enhanced fusion features to obtain a fused image;

[0087] Among them, f c =Concat(f1,f2);

[0088] Among them, G11 and G21 are 1×1 convolution kernels, G12 and G22 are 3×3 convolution kernels, G13 and G23 are 5×5 convolution kernels, the input dimension is 1 and the output dimension is 64; G14 is a 5×5 convolution kernel, G24 is a 7×7 convolution kernel, the input dimension and output dimension are 64; G15 and G25 are both 3×3 convolution kernels, the input dimension and output dimension are 64; G6 and G7 are both 1×1 convolution kernels, the input dimension is 128 and the output dimension is 64; I 01 , I 02 are the multi-scale features of pre-processed infrared images and the multi-scale features of pre-processed visible light images, I′ 01 , I′ 02 are the deep features of the preprocessed infrared image and the deep features of the preprocessed visible light image, respectively. f1 and f2 are the output features of the infrared image and the output features of the visible light image, respectively. c is the fusion feature, and f3 is the enhanced fusion feature.

[0089] Specifically, in step 1034, the enhanced fusion feature can be input into the 1×1 convolution kernel G9 to reduce the 64-dimensional feature to a 1-dimensional image feature, thereby obtaining a fused image.

[0090] In an optional embodiment of the present invention, step 104 includes:

[0091] Step 1041 , performing fault identification on the insulator according to the fused image to obtain an initial identification result;

[0092] Specifically, according to the fusion image and F out =C2f(F in )=Conv2d(Concat(Split(Bottleneck(F in )))),F SPPF =Concat(MaxPool(F out ,k1),MaxPool(F out , k2), ...) to extract features and obtain the extracted features; Among them, F in is the fused image, F out is the initial feature extracted, Bottleneck is the bottleneck block, Split is the feature segmentation operation, Concat is the feature concatenation operation, and Conv2d is the convolution layer; F SPPF is the extracted feature, k1, k2, ... are pooling kernels of different scales. Through pooling operations of different scales, the initial features are spliced together to improve the detection ability of targets of different sizes. MaxPool is the maximum pooling operation.

[0093] According to the extracted features and FFPN,i =Conv2d(Upsample(F FPN,i+1 )+F Backbone,i ), F PAN,i =Conv2d(Downsample(F PAN,i-1 )+F FPN,i ) to perform feature fusion and obtain the fused features; where F FPN,i is the FPN feature map of the i-th layer in the extracted features, F Backbone,i is the Backbone feature map of the i-th layer in the extracted features, Upsample is the upsampling operation, F FPN,i+1 is the i+1th layer FPN feature map in the extracted features, Downsample is the downsampling operation, Conv2d is the convolution layer, F PAN,i-1 is the feature map of the lower level (shallower layer) in the PAN path, F PAN,i The fused features.

[0094] According to the fused features and Class pred =σ(Conv2d(F PAN,i ))、Box pred =σ(Conv2d(F PAN,i ))、Conf pred =σ(Conv2d obj (F PAN,i )) Get the bounding box coordinates, fault category and confidence. The fault recognition result includes the bounding box coordinates, fault category and confidence. pred is the fault category, Box pred is the bounding box coordinate, Conv2d is the convolution layer, F PAN,i is the fused feature, Conf pred is the confidence level, ranging from [0,1], Conv2d obj is a convolution branch of size 1×1.

[0095] Step 1042 , performing screening based on the initial identification result and preset screening conditions to obtain a fault identification result.

[0096] Specifically, the bounding box coordinates and fault category corresponding to the highest confidence level or confidence level greater than a preset value are output as the fault identification result. The bounding box coordinates indicate the location of the fault, and the fault category indicates the type of fault, such as crack, heat, or discharge.

[0097] A specific example of the insulator fault identification method based on image fusion according to an embodiment of the present invention includes:

[0098] Step 111, acquiring an infrared image and a visible light image of the insulator;

[0099] The infrared image of the insulator is acquired by an infrared camera, and the visible light image of the insulator is acquired by an ordinary camera or a digital camera.

[0100] Step 112, image preprocessing;

[0101] The infrared image and the visible light image are filtered and enhanced respectively, and finally the infrared image and the visible light image are registered to find the feature point with the smallest distance value in the two images to obtain the preprocessed infrared image and the preprocessed visible light image.

[0102] Step 113: image fusion processing;

[0103] According to the preprocessed infrared image, the preprocessed visible light image and related formulas, the infrared image output features and the visible light image output features are obtained, and the preprocessed infrared image and the preprocessed visible light image are feature fused according to the two features to obtain a fused image.

[0104] Step 114: perform fault identification on the insulator.

[0105] The insulator is identified for faults based on the fused image, generating an initial identification result consisting of bounding box coordinates, fault category, and confidence level. The bounding box coordinates and fault category corresponding to the highest confidence level, or a confidence level greater than a preset value, are then output as the fault identification result. The bounding box coordinates indicate the location of the fault, while the fault category indicates the type of fault, such as crack, thermal, or discharge defects.

[0106] The insulator fault identification method based on image fusion in the embodiment of the present invention can improve the reliability and safety of insulator detection in the power system, reduce operation and maintenance costs, promote the construction of smart grids and improve environmental protection levels.

[0107] like Figure 2 As shown, an embodiment of the present invention provides an insulator fault identification device 200 based on image fusion, comprising:

[0108] An acquisition module 201 is used to acquire an infrared image and a visible light image of an insulator;

[0109] The processing module 202 is used to preprocess the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image; perform fusion processing based on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image; and perform fault identification on the insulator based on the fused image to obtain a fault identification result.

[0110] Optionally, preprocessing the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image includes:

[0111] performing filtering processing on the infrared image and the visible light image respectively to obtain a filtered infrared image and a filtered visible light image;

[0112] performing enhancement processing on the filtered infrared image and the filtered visible light image respectively to obtain an enhanced infrared image and an enhanced visible light image;

[0113] The enhanced infrared image and the enhanced visible light image are respectively registered to obtain a preprocessed infrared image and a preprocessed visible light image.

[0114] Optionally, filtering the infrared image and the visible light image respectively to obtain a filtered infrared image and a filtered visible light image includes:

[0115] Obtain a preset window; the size of the preset window is N×N;

[0116] Performing filtering according to the preset window, the infrared image, the visible light image, and g(x,y)=median{f(x+i,y+j)}, (i,j)∈N to obtain a filtered infrared image and a filtered visible light image;

[0117] Where g(x,y) is the pixel value of the filtered image at position (x,y), where the filtered image is a filtered infrared image or a filtered visible light image; f(x+i,y+j) is the pixel value in the window N centered at (x,y) in the original image, where the original image is an infrared image or a visible light image; N is the preset window, which is an odd number; and median is the median of all pixel values in the window after sorting.

[0118] Optionally, performing enhancement processing on the filtered infrared image and the filtered visible light image respectively to obtain an enhanced infrared image and an enhanced visible light image includes:

[0119] performing grayscale normalization processing on the filtered infrared image and the filtered visible light image respectively to obtain a grayscale infrared image and a grayscale visible light image;

[0120] According to the grayscale infrared image, the grayscale visible light image and Obtain infrared gray level probability and visible light gray level probability;

[0121] According to the infrared gray level probability, the visible light gray level probability and Obtain infrared cumulative probability and visible light cumulative probability;

[0122] According to the infrared cumulative probability, the visible light cumulative probability and q k =round[(L-1)·S k ], get the new infrared gray level and the new visible light gray level;

[0123] Obtaining an enhanced infrared image and an enhanced visible light image according to the new infrared grayscale and the new visible light grayscale;

[0124] Among them, n k is the number of pixels at each gray level in the original image, k = 0, 1, ..., L-1, L is the total number of gray levels, M and O are the width and height of the original image, f(i, j) is the gray value of the pixel at the i-th row and j-th column in the original image, δ(a, b) is the exponential function, p k is the probability that the gray level is the gray level of the pixel in the original image, where the original image is a grayscale infrared image or a grayscale visible light image; S k is the infrared cumulative probability or the visible light cumulative probability; q k is the new infrared grayscale or the new visible light grayscale, round is rounded to the nearest integer; (L-1) is the value of S k Normalized to the range [0, L-1].

[0125] Optionally, performing registration processing on the enhanced infrared image and the enhanced visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image includes:

[0126] Extracting feature points from the enhanced infrared image and the enhanced visible light image to obtain feature points;

[0127] Matching the feature points to obtain a corresponding relationship between the enhanced infrared image and the enhanced visible light image;

[0128] A preprocessed infrared image and a preprocessed visible light image are obtained according to the enhanced infrared image, the enhanced visible light image and the corresponding relationship.

[0129] Optionally, performing fusion processing on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image includes:

[0130] According to the pre-processed infrared image and I 01 =G11(I1)+G12(I1)+G13(I1), I′ 01 =G14(I 01 )+G14(I 01 )×I 01 、f1=G15(I′ 01 )+G15(I′ 01)×I 01 +G15(I′ 01 )×I′ 01 , get the infrared image output features;

[0131] According to the preprocessed visible light image and I 02 =G21(I2)+G22(I2)+G23(I2), I′ 02 =G24(I 02 )+G24(I 02 )×I 02 、f2=G25(I′ 02 )+G25(I′ 02 )×I 02 +G25(I′ 02 )×I′ 02 , obtain the visible light image output features;

[0132] According to the infrared image output feature, the visible light image output feature and f3=G7(G6(f c ))+G7(G6(f c ))×f1+G7(G6(f c ))×f2, to obtain enhanced fusion features;

[0133] Performing dimensionality reduction processing on the enhanced fusion features to obtain a fused image;

[0134] Among them, f c =Concat(f1,f2);

[0135] Among them, G11 and G21 are 1×1 convolution kernels, G12 and G22 are 3×3 convolution kernels, G13 and G23 are 5×5 convolution kernels, the input dimension is 1 and the output dimension is 64; G14 is a 5×5 convolution kernel, G24 is a 7×7 convolution kernel, the input dimension and output dimension are 64; G15 and G25 are both 3×3 convolution kernels, the input dimension and output dimension are 64; G6 and G7 are both 1×1 convolution kernels, the input dimension is 128 and the output dimension is 64; I 01 , I 02 are the multi-scale features of pre-processed infrared images and the multi-scale features of pre-processed visible light images, I′ 01 , I′ 02 are the deep features of the preprocessed infrared image and the deep features of the preprocessed visible light image, respectively. f1 and f2 are the output features of the infrared image and the output features of the visible light image, respectively. c is the fusion feature, and f3 is the enhanced fusion feature.

[0136] Optionally, performing fault identification on the insulator according to the fused image to obtain a fault identification result includes:

[0137] performing fault identification on the insulator according to the fused image to obtain an initial identification result;

[0138] Screening is performed based on the initial identification result and preset screening conditions to obtain a fault identification result.

[0139] The insulator fault identification device based on image fusion proposed in an embodiment of the present invention obtains an infrared image and a visible light image of the insulator, and preprocesses them respectively to obtain a preprocessed infrared image and a preprocessed visible light image, then fuses the preprocessed infrared image and the preprocessed visible light image to obtain a fused image, and finally performs fault identification on the insulator based on the fused image to obtain a fault identification result, thereby improving the comprehensiveness of image information and facilitating improving the accuracy of insulator fault identification.

[0140] It should be noted that the device is a device corresponding to the above method, and all implementations in the above method embodiment are applicable to the embodiment of the device and can achieve the same technical effects, which will not be described in detail in this embodiment.

[0141] An embodiment of the present invention further provides a computing device comprising: a processor and a memory storing a computer program. When the computer program is executed by the processor, the computer program performs the method described in any of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effects. These are not further described in this embodiment.

[0142] An embodiment of the present invention further provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in any of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of the present invention and can achieve the same technical effects. These are not further described in this embodiment.

[0143] It should be noted that, in the apparatus and method of the present invention, it is apparent that each component or step can be decomposed and / or recombined. Such decomposition and / or recombination should be considered equivalent solutions of the present invention. Furthermore, the steps of performing the above series of processes can naturally be performed in chronological order according to the order described, but do not necessarily need to be performed in chronological order. Certain steps can be performed in parallel, interleaved, or independently of each other.

[0144] It should be noted that, in the above embodiments, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the implementation methods of the above embodiments is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0145] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An insulator fault identification method based on image fusion, characterized in that: include: Acquire infrared and visible light images of insulators; Preprocessing the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image; Performing fusion processing on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image; Fault identification is performed on the insulator according to the fused image to obtain a fault identification result.

2. The insulator fault identification method based on image fusion according to claim 1, characterized in that: Preprocessing the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image, including: performing filtering processing on the infrared image and the visible light image respectively to obtain a filtered infrared image and a filtered visible light image; performing enhancement processing on the filtered infrared image and the filtered visible light image respectively to obtain an enhanced infrared image and an enhanced visible light image; The enhanced infrared image and the enhanced visible light image are respectively registered to obtain a preprocessed infrared image and a preprocessed visible light image.

3. The insulator fault identification method based on image fusion according to claim 2, characterized in that: Performing filtering processing on the infrared image and the visible light image respectively to obtain a filtered infrared image and a filtered visible light image, including: Obtain a preset window; the size of the preset window is N×N; Performing filtering according to the preset window, the infrared image, the visible light image, and g(x,y)=median{f(x+i,y+j)}, (i,j)∈N to obtain a filtered infrared image and a filtered visible light image; Where g(x,y) is the pixel value of the filtered image at position (x,y), where the filtered image is a filtered infrared image or a filtered visible light image; f(x+i,y+j) is the pixel value in the window N centered at (x,y) in the original image, where the original image is an infrared image or a visible light image; N is the preset window, which is an odd number; and median is the median of all pixel values in the window after sorting.

4. The insulator fault identification method based on image fusion according to claim 2, characterized in that: The filtered infrared image and the filtered visible light image are enhanced to obtain an enhanced infrared image and an enhanced visible light image, respectively, including: performing grayscale normalization processing on the filtered infrared image and the filtered visible light image respectively to obtain a grayscale infrared image and a grayscale visible light image; According to the grayscale infrared image, the grayscale visible light image and Obtain infrared gray level probability and visible light gray level probability; According to the infrared gray level probability, the visible light gray level probability and Obtain infrared cumulative probability and visible light cumulative probability; According to the infrared cumulative probability, the visible light cumulative probability and q k =round[(L-1)·S k ], get the new infrared gray level and the new visible light gray level; Obtaining an enhanced infrared image and an enhanced visible light image according to the new infrared grayscale and the new visible light grayscale; Among them, n k is the number of pixels at each gray level in the original image, k = 0, 1, ..., L-1, L is the total number of gray levels, M and O are the width and height of the original image, f(i, j) is the gray value of the pixel at the i-th row and j-th column in the original image, δ(a, b) is the exponential function, p k is the probability that the gray level is the gray level of the pixel in the original image, where the original image is a grayscale infrared image or a grayscale visible light image; S k is the infrared cumulative probability or the visible light cumulative probability; q k is the new infrared grayscale or the new visible light grayscale, round is rounded to the nearest integer; (L-1) is the value of S k Normalized to the range [0, L-1].

5. The insulator fault identification method based on image fusion according to claim 2, characterized in that: Performing registration processing on the enhanced infrared image and the enhanced visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image, including: Extracting feature points from the enhanced infrared image and the enhanced visible light image to obtain feature points; Matching the feature points to obtain a corresponding relationship between the enhanced infrared image and the enhanced visible light image; A preprocessed infrared image and a preprocessed visible light image are obtained according to the enhanced infrared image, the enhanced visible light image and the corresponding relationship.

6. The insulator fault identification method based on image fusion according to claim 1, characterized in that: Performing fusion processing on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image includes: According to the pre-processed infrared image and I 01 =G11(I1)+G12(I1)+G13(I1), I′ 01 =G14(I 01 )+G14(I 01 )×I 01 、f1=G15(I′ 01 )+G15(I′ 01 )×I 01 +G15(I′ 01 )×I′ 01 , get the infrared image output features; According to the preprocessed visible light image and I 02 =G21(I2)+G22(I2)+G23(I2), I′ 02 =G24(I 02 )+G24(I 02 )×I 02 、f2=G25(I′ 02 )+G25(I′ 02 )×I 02 +G25(I′ 02 )×I′ 02 , obtain the visible light image output features; According to the infrared image output feature, the visible light image output feature and f3=G7(G6(f c ))+G7(G6(f c ))×f1+G7(G6(f c ))×f2, to obtain enhanced fusion features; Performing dimensionality reduction processing on the enhanced fusion features to obtain a fused image; Among them, f c =Concat(f1,f2); Among them, G11 and G21 are 1×1 convolution kernels, G12 and G22 are 3×3 convolution kernels, G13 and G23 are 5×5 convolution kernels, the input dimension is 1 and the output dimension is 64; G14 is a 5×5 convolution kernel, G24 is a 7×7 convolution kernel, the input dimension and output dimension are 64; G15 and G25 are both 3×3 convolution kernels, the input dimension and output dimension are 64; G6 and G7 are both 1×1 convolution kernels, the input dimension is 128 and the output dimension is 64; I 01 , I 02 are the multi-scale features of pre-processed infrared images and the multi-scale features of pre-processed visible light images, I′ 01 , I′ 02 are the deep features of the preprocessed infrared image and the deep features of the preprocessed visible light image, respectively. f1 and f2 are the output features of the infrared image and the output features of the visible light image, respectively. c is the fusion feature, and f3 is the enhanced fusion feature.

7. The insulator fault identification method based on image fusion according to claim 1, characterized in that: Performing fault identification on the insulator according to the fused image to obtain a fault identification result includes: performing fault identification on the insulator according to the fused image to obtain an initial identification result; Screening is performed based on the initial identification result and preset screening conditions to obtain a fault identification result.

8. An insulator fault identification device based on image fusion, characterized in that: include: an acquisition module, used for acquiring infrared images and visible light images of insulators; a processing module, configured to preprocess the infrared image and the visible light image respectively to obtain a preprocessed infrared image and a preprocessed visible light image; Performing fusion processing on the preprocessed infrared image and the preprocessed visible light image to obtain a fused image; Fault identification is performed on the insulator according to the fused image to obtain a fault identification result.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium, characterized in that The device stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

Images