A method for detecting power faults based on visible light and infrared fusion images

Abstract

The present application discloses a method for detecting electric faults based on visible light and infrared fusion images, including: step 1, obtaining a first image and a second image of a region to be detected under the same viewing angle; step 2, respectively decomposing the first image and the second image step by step, generating a fusion image according to preset fusion conditions and the images at the same position in the first image and the second image after decomposition; step 3, according to a fault detection model, performing fault identification on the fusion image, adding a prediction box to the fusion image, with a prediction label marked in the prediction box, and the prediction label is used to determine whether the electric equipment in the fusion image is faulty, wherein the model parameters of the fault detection model are determined by a neural network model and a sample fusion image. Through the technical solution in the present application, it is possible to effectively detect faults in electric equipment, has strong practicability, is easy to promote, has high economic and social benefits, and improves the stability of power grid operation.

Description

Technical Field

[0001] The present application relates to the technical field of power fault detection, and in particular to a power fault detection method based on visible light and infrared fusion images. Background Art

[0002] In order to meet the increasing demand for electric energy in China, it is necessary to continue to expand power lines in the direction of high voltage and large capacity, which makes the maintenance of power equipment an issue that cannot be ignored. On the one hand, accurate maintenance and high-efficiency maintenance across terrain are difficult to solve; on the other hand, the transmission equipment on the power lines is prone to damage, especially in remote areas. These transmission equipment are located in remote areas and have a relatively poor operating environment. When these equipment are damaged, it is very easy to cause large-scale power supply failures and accidents. Therefore, it is necessary to conduct regular inspections of power lines, and traditional power circuit inspections are usually manual inspections, which leads to heavy workloads for inspectors, high risks, and low inspection efficiency.

[0003] In the prior art, the good detection performance of infrared imaging sensors for thermal targets can be used to detect faults in power lines. However, the background information of the image formed by the infrared imaging sensor is blurred and cannot accurately identify the specific location of overheating, that is, it cannot tell which component has a fault, and long-distance fault detection is impossible, which is not conducive to accurate maintenance of faulty equipment. Summary of the invention

[0004] The purpose of this application is to solve the problems existing in the existing power equipment fault detection process, improve the effectiveness of power equipment fault detection, and enhance the stability of power grid operation.

[0005] The technical solution of the present application is to provide a method for detecting electric faults based on visible light and infrared fusion images, the method comprising: step 1, obtaining a first image and a second image of a to-be-detected area at the same viewing angle; step 2, respectively decomposing the first image and the second image step by step, and generating a fusion image according to preset fusion conditions and the images at the same position in the first image and the second image after decomposition; step 3, according to a fault detection model, performing fault identification on the fusion image, adding a prediction box to the fusion image, wherein a prediction label is marked in the prediction box, and the prediction label is used to determine whether the electric equipment in the fusion image is faulty, wherein the model parameters of the fault detection model are determined by a neural network model and a sample fusion image, specifically comprising: step 31, obtaining a sample image at the same viewing angle when the electric equipment is in operation, and generating a sample fusion image according to the sample image; step 32, constructing a neural network model, wherein the neural network model comprises a forward propagation path and a reverse propagation path, introducing a momentum factor in the reverse propagation path, and adjusting the weights and thresholds in the reverse propagation path, wherein the calculation formula for adjusting the weights and thresholds is:

[0006] Δω ji (k+1)=(1-mc(k))η(k)δ j x ji +mc(k)Δω ji (k)

[0007] Δb j (k+1)=(1-mc(k))η(k)δ j +mc(k)Δb j (k)

[0008]

[0009] In the formula, Δω ji (k+1) is the weight adjustment for the k+1th iteration, Δb j (k+1) is the threshold adjustment amount for the k+1th iteration, mc(k) is the momentum factor corresponding to the kth iteration, η(k) is the learning rate corresponding to the kth iteration, and δ j is the error term of node j, x ji is the input from node i to node j, i and j are the node numbers, E(k) is the sum of squared errors between the actual output and the expected output at the kth iteration; the learning rate η is calculated as:

[0010]

[0011] Step 33, introduce the sample fusion image input into the neural network model after the momentum factor, perform the k+1th iteration according to the weights and thresholds returned by the back propagation path during the kth iteration, and when it is determined that the k+1th iteration converges, the neural network model is recorded as the fault detection model, and the parameters of the neural network model are recorded as model parameters.

[0012] In any of the above technical solutions, further, the preset fusion condition includes a low-frequency preset fusion condition and a high-frequency preset fusion condition, and step 2 specifically includes:

[0013] Step 21, filtering the first image and the second image respectively according to a Gaussian filter function;

[0014] Step 22, using a wavelet transform method to decompose the filtered first image and the second image level by level, wherein the images obtained by each level of decomposition include a first low-frequency image, a first high-frequency image, a second low-frequency image, and a second high-frequency image, and the first low-frequency image and the second low-frequency image of the current level can be decomposed into a first low-frequency image, a first high-frequency image, a second low-frequency image, and a second high-frequency image of the next level;

[0015] Step 23, according to the preset low-frequency fusion condition, the first low-frequency image and the second low-frequency image at the same position are low-frequency fused, and recorded as a fused low-frequency image;

[0016] Step 24, according to the high-frequency preset fusion condition and the positions of the first high-frequency image and the second high-frequency image after decomposition, the first high-frequency image or the second high-frequency image at the same position obtained by each level of decomposition is sequentially selected and recorded as the fused high-frequency image at the position;

[0017] Step 25, using an inverse wavelet transform method, generating a fused image based on the fused low-frequency image and the fused high-frequency images at different positions.

[0018] In any of the above technical solutions, further, step 23 specifically includes:

[0019] Step 231, splitting the first low-frequency image at the same position into a plurality of first sub-block images of equal size, and splitting the second low-frequency image into a plurality of second sub-block images of equal size, wherein the number of the first sub-block images is equal to the number of the second sub-block images;

[0020] Step 232, respectively calculate the correlation coefficients of the first sub-block image and the second sub-block image at the same position, and the calculation formula of the correlation coefficient ρ(A, B) is:

[0021] ρ(A,B)=Cov(A,B) / σ A σ B

[0022] Cov(A,B)=E[(AE[A])(BE[B])]

[0023] Where A is the pixel value in the first sub-block image, B is the pixel value in the second sub-block image, Cov(·) is the covariance operation, E(·) is the variance operation, σ A is the standard deviation of the first sub-block image, σ B is the standard deviation of the second sub-block image;

[0024] Step 233: Generate a fused low-frequency image according to the correlation coefficient, the first sub-block image, and the second sub-block image.

[0025] In any of the above technical solutions, further, step 24 specifically includes:

[0026] Step 241, splitting the first high-frequency image of the same level into a plurality of third sub-block images of equal size, and splitting the second high-frequency image into a plurality of fourth sub-block images of equal size, wherein the number of the third sub-block images is equal to the number of the fourth sub-block images;

[0027] Step 242, calculate the average pixel values ​​in the third sub-block image and the fourth sub-block image, and select the image with the larger average pixel value in the third sub-block image and the fourth sub-block image at the same level and the same position as the fused high-frequency image at the position.

[0028] In any of the above technical solutions, further, a device label is marked in the sample fusion image.

[0029] In any of the above technical solutions, further, the device label includes a first coordinate, a second coordinate and a device type of the power device to be detected in the fused image, and the second coordinate is located on the opposite side of the first coordinate.

[0030] The beneficial effects of this application are:

[0031] Through the technical solution in this application, it is possible to effectively detect faults in power equipment, it is highly practical, easy to promote, has high economic and social benefits, and improves the stability of power grid operation.

[0032] When the image of the detection area is fused in the present application, in order to make full use of the characteristics of the visible light image and the infrared image, a multi-level decomposition and same-level fusion image processing and fusion mechanism is adopted to fuse the visible light image and the infrared image, so that the final result can not only retain the color and clear detail contours and edges of the visible light image, but also retain the brightness information of the infrared image in the infrared image, so that the infrared target object stands out relative to the background brightness, making it easier to identify the target.

[0033] This application also uses an improved BP algorithm that combines the additional momentum method and the adaptive learning rate adjustment method to update the model parameters, which can not only effectively suppress the network from falling into the local minimum, but also help shorten the learning time, improve the fault detection model's recognition efficiency of the fused image of the area to be detected, and optimize the real-time performance of power equipment fault detection.

[0034] In a preferred implementation of the present application, a visible light imaging device and an infrared imaging device are simultaneously mounted on a drone, and visible light images and infrared images of the same viewing angle can be obtained through personnel monitoring, making sufficient preparations for obtaining fused images, and image information of longer lines and equipment can be captured, avoiding the problem of low efficiency of traditional methods. At the same time, an image processing and fusion mechanism can be built on the drone, and on this mechanism, real-time image acquisition of infrared and visible light images based on the V4L2 interface and real-time image transmission based on the TCP / IP protocol are performed, and efficient fusion of visible light and infrared images can be achieved, thereby improving the real-time performance of power equipment fault detection.

[0035] Furthermore, in the process of fault identification of the fused image by the fault detection model, in order to reduce overlapping prediction boxes and improve the model operation efficiency, the present application, based on the improved NMS method Soft-NMS, dynamically changes the threshold to reduce multiple prediction boxes around the same object, thereby improving the accuracy of fault identification and optimizing the display effect of fault identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The advantages of the above and / or additional aspects of the present application will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:

[0037] Figure 1 is a schematic flow chart of a power fault detection method based on visible light and infrared fusion images according to an embodiment of the present application;

[0038] Figure 2 is a schematic diagram of an image acquisition device according to an embodiment of the present application;

[0039] Figure 3 is a schematic flowchart of a fused image generation process according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0041] In the following description, many specific details are elaborated to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited to the specific embodiments disclosed below.

[0042] like Figure 1 As shown, this embodiment provides a power fault detection method based on visible light and infrared fusion images, the method comprising:

[0043] Step 1, obtaining a first image and a second image of the area to be detected at the same viewing angle;

[0044] Specifically, Figure 2 As shown, in this embodiment, a drone is used as an image acquisition device, and two fixed seats are respectively arranged at the lower end of the drone body. Each fixed seat is provided with a rotating gimbal, and the rotating gimbal can be freely rotated up and down and left and right to acquire two images from the same perspective through two cameras, namely, the first image and the second image of the area to be detected.

[0045] In this embodiment, the first image is set to be a visible light image, and the second image is set to be an infrared image.

[0046] Step 2, respectively decomposing the first image and the second image level by level, and generating a fused image according to a preset fusion condition and the images at the same position in the decomposed first image and the second image;

[0047] It should be noted that the fusion of the first image and the second image can be achieved by an image processing and fusion mechanism built on the drone, or it can be achieved by an image processing and fusion mechanism built on the terminal after the drone sends it to the terminal through a wireless transmission network. This embodiment is not limited to this.

[0048] In this embodiment, the drone collects images in real time through the V4L2 interface and transmits images in real time based on the TCP / IP protocol.

[0049] Through the image processing and fusion mechanism in this embodiment, the fused image obtained can not only retain the color and clear contour edges of the visible light image, but also retain the brightness information of the infrared image.

[0050] It should be noted that during the training and testing of the fault detection model in this embodiment, it is also necessary to use an image acquisition device to collect sample images, and after the image processing and fusion mechanism is used to generate a sample fusion image, the equipment labels of the power equipment in the sample fusion image need to be calibrated to calibrate: the coordinates of the power equipment in the image (which may include the coordinates of the upper left corner and the coordinates of the lower right corner); the type of power equipment, such as an insulator can be marked as jyz; whether the power equipment is faulty, such as an insulator fault can be marked as jyzgz, if not, only the type needs to be marked.

[0051] In this embodiment, when calibrating the equipment label of the electric power equipment, the labelImg tool may be used to perform manual data labeling.

[0052] It should be noted that the existing image fusion method has a large amount of calculation and a slow fusion rate. In order to apply image fusion to the power fault detection in this embodiment, this embodiment shows a method for generating a fused image, which specifically includes:

[0053] Step 21: filter the first image and the second image respectively according to the Gaussian filter function, and convert their resolutions into 320*320.

[0054] In the filtering process, a user-specified template (or convolution, mask) is used to scan each pixel in the image, and the weighted average gray value of the pixels in the neighborhood determined by the template is used to replace the value of the central pixel of the template. Since the image is two-dimensional, a two-dimensional Gaussian function is often used in image processing. Here, the calculation formula of the two-dimensional Gaussian distribution function is:

[0055]

[0056] Where x and y are the pixel coordinates, and the preset parameter θ can usually be set to 1.5, and its size determines the width of the Gaussian function.

[0057] Step 22, using wavelet transform method, decompose the filtered first image and second image level by level, and the images obtained after each level of decomposition include a first low-frequency image, a first high-frequency image, a second low-frequency image, and a second high-frequency image.

[0058] After decomposition, the same wavelet transform method is used again to decompose the first low-frequency image and the second low-frequency image of the current level into the first low-frequency image, the first high-frequency image, the second low-frequency image, and the second high-frequency image of the next level. That is, the low-frequency image of each level can be decomposed step by step to improve the accuracy of image fusion.

[0059] The specific process of decomposing the image using the wavelet transform method is not limited in this embodiment.

[0060] The decomposed images at different frequency component levels can be fused using different fusion rules to obtain fused images.

[0061] It should be noted that the positions of the images obtained by each level of decomposition have not changed, and when fusion is performed, the images at the same position are fused to obtain a fused image corresponding to the first image and the second image.

[0062] In this embodiment, the low-frequency part of the image is used to express the general appearance and average characteristics of the image, and the high-frequency part of the image reflects the detailed characteristics of the image.

[0063] Step 23, according to the low-frequency preset fusion condition in the preset fusion condition, the first low-frequency image and the second low-frequency image at the same position are low-frequency fused step by step, and recorded as a fused low-frequency image;

[0064] Step 24, according to the high-frequency preset fusion condition in the preset fusion condition and the positions of the first high-frequency image and the second high-frequency image at each level after decomposition, sequentially select the first high-frequency image or the second high-frequency image at the same position decomposed at each level, and record it as the fused high-frequency image at the position;

[0065] like Figure 3 As shown, when decomposing, taking the first image as an example, the first image is firstly decomposed into a first low-frequency image and a first high-frequency image (including images 8-10) of a first level, and the first low-frequency image of the first level is decomposed again to obtain a first low-frequency image and a first high-frequency image (including images 5-7) of a second level, and the first low-frequency image of the second level is decomposed again to obtain a first low-frequency image (including image 1) and a first high-frequency image (including images 2-4) of a third level.

[0066] When performing fusion, firstly, low-frequency fusion is performed on image 1 of the two decomposed images according to the low-frequency preset fusion condition, and images 2-4 are selected according to the high-frequency preset fusion condition to obtain a fused high-frequency image corresponding to the third level; then, images 5-7 are selected according to the same high-frequency preset fusion condition to obtain a fused high-frequency image corresponding to the second level; finally, images 8-10 are selected according to the high-frequency preset fusion condition to obtain a fused high-frequency image corresponding to the first level.

[0067] It should be noted that the decomposition of the first image and the second image is determined by a preset image decomposition condition and pixel information in the image, and this embodiment does not limit the process of image decomposition.

[0068] When low-frequency fusion is performed on the first low-frequency image and the second low-frequency image according to the preset low-frequency fusion condition, the method specifically includes:

[0069] Step 231, splitting the first low-frequency image at the same position into a plurality of M×N first sub-block images of equal size, and splitting the second low-frequency image into a plurality of M×N second sub-block images of equal size, wherein the number of the first sub-block images is equal to the number of the second sub-block images;

[0070] In this embodiment, since the size of the first sub-block image is equal to the size of the second sub-block image, and the shooting angles of the original images (the first image and the second image) are the same, the scene in the split first sub-block image corresponds to the scene in the second sub-block image, and the two can be numbered one-to-one, that is, the qth first sub-block image corresponds to the qth second sub-block image.

[0071] Taking the sub-block image as a unit, the numerical distribution statistics of each sub-block image are performed, and its variance, standard deviation, and covariance are calculated, and then the correlation coefficient is calculated.

[0072] Step 232, respectively calculate the correlation coefficient ρ(A, B) of the first sub-block image and the second sub-block image at the same position, ρ(A, B)∈[0,1], and the calculation formula of the correlation coefficient ρ(A, B) is:

[0073] ρ(A,B)=Cov(A,B) / σ A σ B

[0074] Cov(A,B)=E[(AE[A])(BE[B])]

[0075] Where A is the pixel value in the first sub-block image, B is the pixel value in the second sub-block image, Cov(·) is the covariance operation, E(·) is the variance operation, σ A is the standard deviation of the first sub-block image, σ B is the standard deviation of the second sub-block image;

[0076] Step 233: Generate the fused low-frequency image according to the correlation coefficient, the first sub-block image, and the second sub-block image.

[0077] Specifically, by setting a fusion threshold, it is determined in turn whether the correlation coefficient ρ(A, B) is within the range of the fusion threshold. When the correlation coefficient ρ(A, B) is within the range of the fusion threshold, the first sub-block image is selected as the fused sub-image, otherwise, the second sub-block image is selected as the fused sub-image.

[0078] Then, a fused low-frequency image is generated according to the selected fused sub-image and the position of the image.

[0079] Furthermore, in order to improve the accuracy of the fused image and make full use of the color and clear contour edges of the visible light image and the brightness information of the infrared image, this embodiment also shows an improved low-frequency image fusion method, which specifically includes:

[0080] When the correlation coefficient ρ(A, B) is determined to belong to the first threshold, that is, ρ(A, B)∈[0,0.19), if the standard deviation σ of the first sub-block image A Greater than or equal to the standard deviation σ of the second sub-block image B When , the first sub-block image is recorded as the fused low-frequency sub-image, otherwise, the second sub-block image is recorded as the fused low-frequency sub-image;

[0081] When the correlation coefficient ρ(A, B) is determined to belong to the second threshold, that is, ρ(A, B)∈[0.19,0.5), the calculation formula for the fused low-frequency sub-image F is:

[0082]

[0083] When the correlation coefficient ρ(A, B) is determined to belong to the third threshold, that is, ρ(A, B)∈[0.5,1), the calculation formula for the fused low-frequency sub-image F is:

[0084]

[0085] Furthermore, when selecting and merging high-frequency images according to the high-frequency preset fusion condition, it specifically includes:

[0086] Step 241, splitting the first high-frequency image of the same level into a plurality of third sub-block images of equal size, and splitting the second high-frequency image into a plurality of fourth sub-block images of equal size, wherein the number of the third sub-block images is equal to the number of the fourth sub-block images;

[0087] Step 242, calculating the average pixel values ​​in the third sub-block image and the fourth sub-block image, and selecting the image with the larger average pixel value in the third sub-block image and the fourth sub-block image at the same level and the same position as the fused high-frequency image at the corresponding position.

[0088] Specifically, Figure 3 As shown, first, the images 2-4 in the first image are split to obtain the third sub-block images corresponding to the third level, and the pixel average values ​​corresponding to each third sub-block image are calculated. At the same time, the images 2-4 in the second image are split to obtain the fourth sub-block images corresponding to the third level, and the pixel average values ​​corresponding to each fourth sub-block image are calculated. According to the positions of the sub-block images, images with larger pixel average values ​​are selected in turn as the fused high-frequency images corresponding to the third level.

[0089] Then, in the same manner, each fused high-frequency image corresponding to the second level is determined in images 5-7, and each fused high-frequency image corresponding to the first level is determined in images 8-10.

[0090] Step 25, using the inverse wavelet transform method, generates a fused image based on the fused low-frequency image and the fused high-frequency image.

[0091] Step 3: According to the fault detection model, fault identification is performed on the fused image, and a prediction box is added to the fused image. The prediction box is marked with a prediction label. The prediction label is used to determine whether the power equipment in the fused image is faulty. The model parameters of the fault detection model are determined by the neural network model and the sample fused image.

[0092] In this embodiment, the model parameters of the fault detection model are determined by inputting the sample fusion image into a network model with self-learning ability and utilizing the self-learning ability of the network model. The network model after the model parameters are determined is recorded as the fault detection model. The fault detection model can perform fault identification on the fusion image obtained in the above process.

[0093] In this embodiment, a BP network model in a neural network is used as a network model. The process of determining the model parameters in the BP network model specifically includes:

[0094] Step 31, obtaining sample images at the same viewing angle when the power equipment is in operation, and generating a sample fusion image based on the sample images;

[0095] Step 32, constructing a neural network model, the neural network model includes a forward propagation path and a reverse propagation path, introducing a momentum factor in the reverse propagation path, adjusting the weights and thresholds in the reverse propagation path, wherein the calculation formula for adjusting the weights and thresholds is:

[0096] Δω ji (k+1)=(1-mc(k))η(k)δ j x ji +mc(k)Δω ji (k)

[0097] Δb j (k+1)=(1-mc(k))η(k)δ j +mc(k)Δb j (k)

[0098]

[0099] In the formula, Δω ji (k+1) is the weight adjustment for the k+1th iteration, Δb j (k+1) is the threshold adjustment amount for the k+1th iteration, mc(k) is the momentum factor corresponding to the kth iteration, η(k) is the learning rate corresponding to the kth iteration, and δ j is the error term of node j, x ji is the input from node i to node j, i and j are the node numbers, and E(k) is the sum of squared errors between the actual output and the expected output at the kth iteration;

[0100] Specifically, by introducing the momentum factor, the network not only considers the effect of the error on the gradient when correcting the weights, but also considers the influence of the changing trend on the error surface. Without the effect of additional momentum, the network may fall into shallow local minima, and with the effect of additional momentum, it is possible to slide through these minima.

[0101] Since in the flat area of ​​the error surface, if the learning rate η(k) is too small, the number of training times will increase, so it is hoped to increase the value of η(k); in the area where the error changes drastically, if η(k) is too large, the adjustment amount will be too large and cross the narrow "pit", causing the number of iterations to increase and the training to oscillate. Therefore, the learning rate η(k) is adaptively adjusted to accelerate the convergence of the BP network. The calculation formula of the learning rate η(k) is:

[0102]

[0103] By introducing the momentum factor and adaptively adjusting the learning rate η, we can effectively prevent the network from falling into the local minimum and shorten the learning time.

[0104] Step 33, introduce the sample fusion image input into the neural network model after the momentum factor, perform the k+1th iteration according to the weights and thresholds returned by the back propagation path during the kth iteration, and when it is determined that the k+1th iteration converges, the neural network model is recorded as the fault detection model, and the parameters of the neural network model are recorded as model parameters.

[0105] Specifically, the entire sample data set is labeled, and then the sample data set is divided into a training set and a test set (the test set is generally smaller, such as 20% of the sample data set) according to a certain ratio based on actual conditions.

[0106] The training set is input into the above model for iterative training. According to the error between the calibrated device label and the model output result, the improved BP algorithm is used to adjust the weight and threshold until the model converges, that is, the error between the calibrated device label and the model output result is stabilized within a certain range.

[0107] Afterwards, the test set is input into the above-mentioned trained model for model testing, and the model accuracy is output. If the accuracy meets expectations, the model can be used as a fault detection model.

[0108] The fault detection model obtained in this embodiment can add a prediction label on the fused image. The prediction label corresponds to the device label calibrated in the sample fused image and is predicted by the BP neural network algorithm after introducing a momentum factor and changing the learning rate to adaptive adjustment.

[0109] In this embodiment, the original image (the first image and the second image) of the area to be inspected is obtained by a drone, and then the first image and the second image are fused through an image processing and fusion mechanism to obtain a fused image, which is used as an input and input into the above-mentioned fault detection model. The fused image is output with a prediction box through a BP neural network algorithm, wherein a prediction label (equipment label) is indicated on the prediction box. The prediction label in the prediction box can indicate the predicted equipment status. For example, if the label is "jyzgz", it represents an equipment insulator fault in the area to be inspected.

[0110] It should be noted that the present embodiment also uses the Soft-NMS method with dynamically changing thresholds to reduce overlapping prediction boxes, leaving prediction boxes with confidence scores higher than the threshold t, and improving the recognizability of the output results of the fault detection model, wherein the adjustment formula of the threshold t is:

[0111]

[0112] Where IoU is the intersection over union ratio.

[0113] The above describes in detail the technical solution of the present application in combination with the accompanying drawings. The present application proposes a method for detecting power faults based on visible light and infrared fusion images, including: step 1, obtaining a first image and a second image of the area to be detected under the same viewing angle; step 2, respectively decomposing the first image and the second image step by step, and generating a fused image according to preset fusion conditions and the images at the same position in the first image and the second image after decomposition; step 3, according to the fault detection model, fault identification is performed on the fused image, and a prediction box is added to the fused image. The prediction box is marked with a prediction label, and the prediction label is used to determine whether the power equipment in the fused image is faulty, wherein the model parameters of the fault detection model are determined by the neural network model and the sample fusion image. Through the technical solution in the present application, it is possible to effectively detect faults in power equipment, which is highly practical and easy to promote, with high economic and social benefits, and improves the stability of power grid operation.

[0114] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0115] The units in the device of the present application can be combined, divided and deleted according to actual needs.

[0116] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. A power fault detection method based on visible light and infrared fusion images, characterized in that: The method comprises: Step 1, obtaining a first image and a second image of the area to be detected at the same viewing angle; Step 2, respectively decomposing the first image and the second image level by level, and generating a fused image according to a preset fusion condition and an image at the same position in the first image and the second image after decomposition, wherein the preset fusion condition includes a low-frequency preset fusion condition and a high-frequency preset fusion condition, and specifically includes: Step 21, filtering the first image and the second image respectively according to a Gaussian filter function; Step 22, using a wavelet transform method to decompose the filtered first image and the second image level by level, wherein the images obtained by each level of decomposition include a first low-frequency image, a first high-frequency image, a second low-frequency image, and a second high-frequency image, and the first low-frequency image and the second low-frequency image of the current level can be decomposed into a first low-frequency image, a first high-frequency image, a second low-frequency image, and a second high-frequency image of the next level; Step 23, according to the low-frequency preset fusion condition, low-frequency fusion is performed on the first low-frequency image and the second low-frequency image at the same position, recorded as a fused low-frequency image, which specifically includes: Step 231: split the first low-frequency image at the same position into a plurality of first sub-block images of equal size, and split the second low-frequency image into a plurality of second sub-block images of equal size, wherein the number of the first sub-block images is equal to the number of the second sub-block images; Step 232, sequentially calculate the correlation coefficients of the first sub-block image and the second sub-block image at the same position, and the calculation formula of the correlation coefficient ρ(A, B) is: p(A,B)=Cov(A,B) / σ A s B Cov(A,B)=E[(AE[A])(BE[B])] Where A is the pixel value in the first sub-block image, B is the pixel value in the second sub-block image, Cov(·) is the covariance operation, E(·) is the variance operation, σ A is the standard deviation of the first sub-block image, σ B is the standard deviation of the second sub-block image; Step 233, generating the fused low-frequency image according to the correlation coefficient, the first sub-block image, and the second sub-block image; Step 24, according to the high-frequency preset fusion condition and the positions of the first high-frequency image and the second high-frequency image after decomposition, sequentially selecting the first high-frequency image or the second high-frequency image at the same position decomposed at each level, and recording it as the fused high-frequency image at the position, specifically includes: Step 241, splitting the first high-frequency image of the same level into a plurality of third sub-block images of equal size, and splitting the second high-frequency image into a plurality of fourth sub-block images of equal size, wherein the number of the third sub-block images is equal to the number of the fourth sub-block images; Step 242, calculating the average pixel values ​​of the third sub-block image and the fourth sub-block image, and selecting the image with the larger average pixel value in the third sub-block image and the fourth sub-block image at the same level and the same position as the fused high-frequency image at the position; Step 25, using an inverse wavelet transform method to generate the fused image according to the fused low-frequency image and the fused high-frequency images at different positions; Step 3: perform fault identification on the fused image according to the fault detection model, add a prediction box in the fused image, and mark the prediction box with a prediction label, and the prediction label is used to determine whether the power equipment in the fused image is faulty, wherein the model parameters of the fault detection model are determined by the neural network model and the sample fused image, and specifically include: Step 31, obtaining sample images at the same viewing angle when the power equipment is in operation, and generating the sample fusion image according to the sample images; Step 32, constructing a neural network model, the neural network model includes a forward propagation path and a reverse propagation path, introducing a momentum factor in the reverse propagation path, adjusting the weights and thresholds in the reverse propagation path, wherein the calculation formula for adjusting the weights and thresholds is: Give ji (k+1)=(1-mc(k))η(k)δ j x ji +mc(k)Dη ji (k) Δb j (k+1)=(1-mc(k))η(k)δ j +mc(k)Δb j (k) In the formula, Δω ji (k+1) is the weight adjustment for the k+1th iteration, Δb j (k+1) is the threshold adjustment amount for the k+1th iteration, mc(k) is the momentum factor corresponding to the kth iteration, η(k) is the learning rate corresponding to the kth iteration, and δ j is the error term of node j, x ji is the input from node i to node j, i and j are the node numbers, E(k) is the square sum of the error between the actual output and the expected output of the kth iteration; the learning rate η is calculated as: Step 33, input the sample fusion image into the neural network model after introducing the momentum factor, perform the k+1th iteration according to the weights and thresholds returned by the back propagation path during the kth iteration, and when it is determined that the k+1th iteration converges, the neural network model is recorded as the fault detection model, and the parameters of the neural network model are recorded as the model parameters.

2. The power fault detection method based on visible light and infrared fusion image according to claim 1 is characterized in that: The sample fusion image is marked with a device label.

3. The power fault detection method based on visible light and infrared fusion images as claimed in claim 2 is characterized in that: The device tag includes a first coordinate, a second coordinate, and a device type of the power device to be detected in the fused image, and the second coordinate is located on the opposite side of the first coordinate.

Images