Power grid line defect analysis method integrating visible light and infrared image processing

Abstract

This invention proposes a power grid line defect analysis method that integrates visible light and infrared image processing to achieve visible light illumination enhancement, discharge point identification, and intelligent target temperature extraction from infrared images. The method also integrates the intelligent image processing algorithm with drone on-site inspections, achieving the following effects: First, the enhanced image can assist operators in patrols and inspections in low-light environments, and identify abnormal discharge points; second, it automatically identifies equipment targets and extracts temperature values from infrared images, making it easier for operators to detect abnormal temperature rises in equipment; and third, by integrating the processing results of infrared and visible light images, it further subdivides equipment fault types. This invention can address the problems of low patrol efficiency, difficulty in defect detection, and low defect analysis efficiency caused by insufficient illumination in traditional night patrols. It improves the efficiency and intelligence level of drone night patrols, ensuring the reliable operation of power equipment.

Description

Technical Field

[0001] The present invention relates to the technical fields of power equipment inspection, visible light image illumination enhancement, infrared image recognition, and the like, and in particular to a power grid line defect analysis method that integrates visible light and infrared image processing. Background Art

[0002] During power grid inspections, taking infrared images during the day presents numerous challenges, including interference from sunlight and equipment temperature rise caused by sunlight. Therefore, infrared photography needs to be performed during periods of low sunlight, but this also creates issues with insufficient illumination that can affect visibility.

[0003] Furthermore, current power grid line inspections primarily rely on visible light and infrared imagery for fault diagnosis. However, relying on only one type of imagery makes it difficult to determine the cause of a defect. For example, visible light images primarily reveal surface defects in equipment, while infrared images only detect abnormal temperature rises. Different defect causes require different troubleshooting methods. For example, internal defects and temperature rises require equipment replacement, while external contamination and heat generation require cleaning. Summary of the Invention

[0004] To address the conflicting environmental requirements for infrared and visible light image capture, visible light images are enhanced for illumination, enabling both inspection and infrared capture in low-light conditions. Furthermore, during deep learning segmentation, infrared image segmentation can be incomplete or involve excessive regions, necessitating fine-grained segmentation for target recognition and extraction in infrared images.

[0005] At the same time, by fusing visible light and infrared images, discharge and temperature rise can be used together for defect diagnosis and analysis, further improving the accuracy of defect classification.

[0006] Therefore, to address the shortcomings of existing technologies, this invention proposes a drone-assisted night patrol method based on visible light and infrared image processing. This method addresses the issues of insufficient illumination and inefficient infrared defect analysis during nighttime line inspections. First, by enhancing the illumination of low-light visible light images, inspectors can visually identify and inspect line equipment without the need for additional lighting. Second, it automatically segments and identifies equipment targets in infrared images, extracting the target component temperature. Finally, the visible light and infrared image processing are integrated to comprehensively diagnose equipment defect types.

[0007] The purpose of this invention is to achieve visible light illumination enhancement, discharge point identification, and intelligent target temperature extraction from infrared images, and to integrate the image intelligent processing algorithm with drone on-site inspections. This can achieve the following effects: First, the enhanced image can assist operators in patrolling and inspecting in low-light environments and identify abnormal discharge points; second, it automatically identifies equipment targets and extracts temperature values from infrared images, making it easier for operators to detect abnormal temperature rises in equipment; third, it integrates the processing results of infrared and visible light images to further subdivide the type of equipment failure. This invention can solve the problems of low patrol efficiency, difficulty in defect detection, and low defect analysis efficiency caused by insufficient illumination in traditional night patrols. It improves the efficiency and intelligence level of drone night patrols and ensures the reliable operation of power equipment.

[0008] The present invention specifically adopts the following technical solutions:

[0009] A method for analyzing power line defects by integrating visible light and infrared image processing, characterized by comprising the following steps:

[0010] Step S1: The edge computing terminal extracts visible light and infrared images captured by the drone;

[0011] Step S2: The edge computing terminal performs adaptive illumination enhancement on the visible light image, identifies the discharge points in the image, and transmits the image to the patrol device to assist operators in observing the on-site situation.

[0012] Step S3: The edge computing terminal extracts target equipment including insulators and tension clamps from the infrared image through the intelligent segmentation algorithm of "image + data" fusion, and extracts temperature values to assist in the diagnosis of equipment status.

[0013] Step S4: Match and associate the target device identified by the infrared image with the discharge point position of the visible light image, and combine the target device discharge point in the visible light image and the target device temperature value in the infrared image to determine the fault status of the device.

[0014] Furthermore, step S2 specifically includes the following steps:

[0015] Step S21: using a convolutional neural network trained on visible light image samples with normal illumination as a decomposition network to decompose the visible light image S to obtain a reflected light image R and an incident light image I;

[0016] Step S22: Using a convolutional neural network as an enhancement network, the illumination distribution of the incident light image I is estimated and the illumination of the incident light image I is enhanced to obtain an image I'. Simultaneously, the noise in the reflected image R is suppressed to obtain an image R'. The adjusted reflected image R' is multiplied by the incident light image I' to obtain an enhanced visible light image S'.

[0017] Step S23: After obtaining the enhanced visible light image, discharge points are detected and positions of the discharge points are marked.

[0018] Furthermore, step S3 specifically includes the following steps:

[0019] Step S31: performing temperature matrix analysis on the extracted infrared image to obtain an infrared image and a temperature matrix;

[0020] Step S32: Construct an encoder-decoder convolutional network. The encoder encodes the original infrared image and obtains image features. The decoder fuses the image features to achieve preliminary segmentation of the target and obtain image F1.

[0021] Step S33: Considering the deviation between the segmented target in image F1 and the actual target, i.e., the presence of missing or excessive targets, further processing is required to improve the accuracy of the segmented targets. The center point P on the segmented target in F1 is selected, and P is used as the center to expand horizontally or vertically to obtain a center point set TL. The points in the set TL are fitted to obtain the centerline slope.

[0022] Step S34: Move one point from P in the direction perpendicular to the center line to obtain point P2, confirm whether the temperature difference between point P2 and P exceeds 5°C, and repeat the process until the segmented target set M is obtained; select a rectangular box Mi along the center line with w*h, and calculate the similarity with the rectangular box M0 selected by the center point P. If the similarity value between Mi and M0 is greater than Set, the point in the rectangular box Mi belongs to the segmentation target, and continue to move to the next point and repeat steps S33 and S34; if the similarity value between Mi and M0 is less than Set, the point in the rectangular box Mi does not belong to the segmentation target;

[0023] Step S35: The finely extracted center point set TLC is obtained according to the above steps, and is further mapped into a temperature matrix to obtain a temperature value representing the target area.

[0024] Furthermore, step S4 specifically includes the following steps:

[0025] Step S41: mapping the discharge point positions in the visible light image to the infrared image according to the pixel ratio between the infrared image and the visible light image;

[0026] Step S42: Based on the device identified in the infrared image, the temperature of the device and the position of the discharge point are integrated to comprehensively determine the discharge heating defect, the internal heating defect and their severity.

[0027] Furthermore, a polling method is used to extract the latest visible light images and infrared images taken by the drone, and wireless transmission is used to return the processed results to the ground terminal to assist operators in patrol and inspection.

[0028] Furthermore, in step S1, an edge computing terminal carried on the drone is used to extract the visible light and infrared images taken by the drone, and calculate the illumination value of the visible light image.

[0029] Furthermore, it also includes step S5: the edge computing terminal feeds back the processed captured images and inspection results in real time through the wireless communication module, and the ground handheld terminal displays and stores the images and inspection results.

[0030] Compared to existing technologies, this invention and its preferred solution address practical issues faced during nighttime patrols: insufficient illumination of visible light images prevents operators from viewing them; and infrared images require manual use of specialized software for component temperature analysis and condition diagnosis, resulting in low efficiency and prone to errors and omissions. By intelligently processing visible light and infrared images, this approach addresses these challenges during nighttime patrols. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0032] Figure 1 Flowchart of the process of an embodiment of the present invention.

[0033] Figure 2 4 is a flow chart of visible light image processing according to an embodiment of the present invention.

[0034] Figure 3 This is a structural diagram of infrared image processing according to an embodiment of the present invention.

[0035] Figure 4 This is a defect diagnosis image that fuses visible light and infrared images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] Hereinafter, specific embodiments of the present application will be described in detail with reference to the accompanying drawings. Based on these detailed descriptions, those skilled in the art will be able to clearly understand the present application and implement the present application. Without violating the principles of the present application, the features of different embodiments may be combined to obtain new implementations, or certain features of certain embodiments may be substituted to obtain other preferred implementations.

[0037] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0038] To make the features and advantages of this patent more clearly understood, the following embodiments are specifically described in detail as follows:

[0039] like Figures 1-4 As shown, this embodiment provides a power grid line defect analysis method that integrates visible light and infrared image processing. For visible light images captured during nighttime drone inspections, illumination enhancement is used to provide visible light images suitable for visual inspection of power grid equipment. For infrared inspection images, image segmentation and defect diagnosis are used to analyze the status of target components. This method not only assists drones in capturing on-site images during nighttime inspections, but also improves the efficiency of equipment status defect analysis, significantly enhancing the quality and effectiveness of drone nighttime inspections.

[0040] Taking the nighttime inspection of composite insulators as an example, the implementation steps include the following:

[0041] In step S1, a DJI M300 drone is equipped with visible light and infrared lenses to capture visible light and infrared images simultaneously during the patrol process. At the same time, the hardware connection between the edge computing terminal and the drone is realized through the OSDK expansion board. At the same time, a self-starting script program is written in the edge computing terminal, and the terminal automatically polls the images captured by the drone. After discovering newly captured infrared and visible light images, the images are extracted into the terminal's memory.

[0042] In step S2, in order to reduce the amount of calculation, the visible light image is cropped and compressed to the same size as the infrared image. At the same time, the terminal performs image illumination detection on the visible light image. If the illumination meets the requirements, the discharge point is searched; if the illumination does not meet the requirements, the visible light image is enhanced. In the visible light image enhancement process, a convolutional neural network (decomposition network) trained on visible light image samples with normal illumination is used to decompose the visible light image S to obtain a reflected image R and an incident light image I; the convolutional neural network (enhancement network) is further used to estimate the illumination distribution of the incident light image I and improve the illumination of the incident light image I to obtain image I'; at the same time, the noise in the reflected image R is suppressed to obtain image R'; and the adjusted reflected image R' and the incident light image I' are multiplied to obtain the enhanced visible light image S';

[0043] After visible light image enhancement processing, the discharge point appears as a light blue area in the visible light image. In order to further identify the discharge point area, the following method is used to identify the discharge area:

[0044] (1) In the historical images with discharge points, the image area containing the discharge points is extracted as a reference sample; in order to improve the recognition accuracy, the discharge point area in the image is divided into a core area C and an extended area T, that is, one discharge point corresponds to a set of {C, T} and The radius of the expansion area is λ times the radius of the core area, where C = {(x c1 ,y c1 ),...,(x cm ,y cm )},T={(x t1 ,y t1 ),...,(x tn ,y tn )}. At the same time, the image feature values of the core area and the extended area are analyzed, and the maximum deviation values δ1 and δ2 are set;

[0045] (2) Scan the image to be processed with an area of radius R to obtain the core area R i , R i Calculate the matching degree with the core area C. If the matching degree deviation is greater than δ1, continue scanning the next area. If the matching degree deviation is less than δ1, expand the ratio λ outward with the core area as the center to obtain the undetermined expansion area G. i , G i The matching degree is calculated with the extended area T. If the matching degree deviation is greater than δ2, the scanning of the next area is continued; if the matching degree deviation is less than δ2, the core area is selected as the discharge point area;

[0046] (3) Repeat step 2 until the entire image is scanned and the set of discharge points F = {(x fi ,y fi )}.

[0047] In step S3, the edge computing terminal uses the "image + data" intelligent segmentation algorithm to identify power equipment such as insulators and tension clamps in the infrared image and extract the temperature value of the target equipment;

[0048] (1) The edge computing terminal starts the polling program, extracts infrared images from the drone through the Onboard expansion board interface, and performs temperature analysis on the infrared images to obtain a temperature value matrix;

[0049] (2) The edge computing module runs the image intelligent segmentation model to obtain the coarse segmentation target area;

[0050] (3) The “rough” segmentation of infrared images has problems such as incomplete target extraction, uneven edges, and too large a segmentation range. The temperature value matrix is used to perform fine extraction of the target area, as follows:

[0051] (3.1) First, extract the coordinates of the roughly segmented image. According to all the coordinate points, the center point c(x i ,y j ), and calculate the average temperature t of the coarse segmentation image mean ;

[0052] (3.2) Map the center point segmented image to the temperature matrix to obtain the center point temperature value of the segmented image and the average temperature of the segmented image area;

[0053] (3.3) With the center point as the starting point, expand in four directions: up, down, left, and right. Take the left expansion method as an example: find the nearest point to the left of the center point (x i-1 ,y j ), and find the upper and lower adjacent points of this point (x i-1 ,y j-1 ), (x i-1 ,y j+1 ), if the temperature values of each point are greater than the average temperature of the region t mean , determine the next center point as (x i-1 ,y j ); if point (x i-1 ,y j-1 ) corresponds to a temperature value less than t mean , then the alternative point is (x i-1 ,y j ), (x i-1 ,y j+1 ), (x i-1 ,y j+2 ), if the temperature values of each point are greater than the average temperature of the region t mean , then the center point is determined to be (x i-1 ,y j+1 ); if point (x i-1 ,y j+1 ) corresponds to a temperature value less than t mean , then the alternative point is (x i-1 ,y j-2 ), (x i-1 ,y j-1 ), (x i-1 ,y j ), if the temperature values of each point are greater than the average temperature of the region t mean , then determine the next center point as (x i-1 ,y j-1 ); if point (xi -1,y j ) corresponds to a temperature value less than t mean , then stop expanding; repeat the above steps to get the expanded center point set, that is, LC={(x c,i ,y c,j )}, the set can form a central point connecting line;

[0054] (3.4) The center point connecting line is further fitted using the least squares method to obtain the slope of the center point connecting line. Then, the endpoint of one end of the center point connecting line is selected and extended in the direction perpendicular to the connecting line. If the temperature value between two adjacent points exceeds 5°C, it is considered to be a component boundary. After point-by-point extension, a new segmented component image can be obtained.

[0055] (3.5) In the new segmented component image, select the center point of the end and use the w*h rectangular box to select the temperature value of the area DL1. At the same time, use point c(x i ,y j ) as the center, and also select a w*h rectangular box D c .

[0056] Calculate D c Similarity to the image in DL1:

[0057] (3.5.1) First, normalize the temperature values in the two boxes, and the resulting two box areas are X and Y;

[0058] (3.5.2) Then, calculate the similarity between the two frames

[0059]

[0060]

[0061]

[0062]

[0063]

[0064] The similarity judgment criteria are:

[0065] Take C3 = (K*L)^2 / 2, generally K = 0.03, L = 1

[0066] (3.5.3) If the similarity value S is greater than δ3, the endpoint belongs to the target device; if the similarity value is less than δ3, the endpoint does not belong to the target device, and the next center point is selected to the right and the similarity calculation is performed;

[0067] (3.5.4) Finally, the center points of the rectangular boxes with similarity matching are summarized to determine the target device center line TLC after fine extraction = {(x tc,i ,y tc,i )}, i∈[0,N), N is the number of center points, and the corresponding center line temperature value is obtained according to the coordinates of the points on the center line.

[0068] In step S4, the target device identified by the infrared image is matched and associated with the discharge point position of the visible light image, and the target device discharge point in the visible light image and the target device temperature value in the infrared image are combined to determine the fault status of the device.

[0069] Point (x fi ,y fi ) discharge condition is v=disch(x fi ,y fi ), if v = 0, there is no discharge at this point, and if v = 1, there is discharge at this point; the coordinates of the point mapped to the infrared temperature matrix are (x i ,y i ), that is, (x i ,y i )=tran((x fi ,y fi )); and (x i ,y i ) corresponds to the temperature value w(x i ,y i ).

[0070] Design df function for point (x i ,y i ) corresponds to the temperature value w(x i ,y i ) and the discharge in the visible light image disch(x fi ,y fi ) to make a comprehensive judgment:

[0071]

[0072] Set value is set according to the ambient temperature. 0 indicates normal temperature and no discharge, 1 indicates temperature rise but no discharge, 2 indicates normal temperature but discharge, and 3 indicates temperature rise and discharge.

[0073] Therefore, the corresponding df value at the point on the center line TLC is: DF = {df i}.

[0074] Further comprehensive analysis of the DF sequence is conducted, and the judgment criteria are as follows:

[0075] (1) If in DF, the proportion p of points where dfi = 3 in the entire center line df＝3 > setp3, then determine the status of the component as 3;

[0076] (2) If in DF, the proportion p of points where dfi = 3 in the entire center line df＝3 < setp3, and p df＝1 > setp1, then determine the status of the component as 2;

[0077] (3) If in DF, p df＝3 < setp3 and p df＝1 < setp1, p df＝2 > setp2, then determine the status of the component as 1;

[0078] (4) In other cases, determine the status of the component as 0

[0079] Output the result. 0 indicates that the component has no fault, 1 indicates surface pollution flash temperature rise of the component, 2 indicates internal defect temperature rise of the component, and 3 indicates temperature rise caused by both pollution flash and internal defects.

[0080] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications are also regarded as the protection scope of the present invention.

[0081] This patent is not limited to the above best embodiment. Anyone can obtain other various forms of power grid line defect analysis methods that integrate visible light and infrared image processing under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope covered by this patent.

Claims

1. A method for analyzing power line defects by integrating visible light and infrared image processing, characterized in that: The following steps are involved: Step S1: The edge computing terminal extracts visible light and infrared images captured by the drone; Step S2: The edge computing terminal adaptively enhances the visible light image and identifies the discharge points in the image. The image is then transmitted to the patrol device to assist operators in observing the on-site situation. The core area C and the extended area T are trained using historical samples, and the discharge points are dynamically identified using deviation thresholds δ1 and δ2. Step S3: The edge computing terminal extracts target equipment, including insulators and tension clamps, from the infrared image using an intelligent segmentation algorithm that uses "image + data" fusion. It also extracts temperature values to assist in diagnosing the equipment status. Perform temperature matrix analysis on the extracted infrared image to obtain the infrared image and temperature matrix; Construct an encoder-decoder convolutional network. The encoder encodes the original infrared image and obtains image features. The decoder fuses the image features to achieve preliminary segmentation of the target and obtain image F1. Select the center point P on the segmented target in F1 and expand horizontally or vertically with P as the center to obtain the center point set TL; fit the points in the set TL to obtain the centerline slope; Move one point from P in the direction perpendicular to the center line to obtain point P2. Check whether the temperature difference between point P2 and point P exceeds 5°C, and repeat the process until the segmented target set M is obtained. Select a rectangular box Mi along the center line with w*h, and calculate the similarity with the rectangular box M0 selected by the center point P. If the similarity value between Mi and M0 is greater than Set, the point in the rectangular box Mi belongs to the segmentation target, and continue to move to the next point and repeat steps S33 and S34. If the similarity value between Mi and M0 is less than Set, the point in the rectangular box Mi does not belong to the segmentation target. The TLC of the center point set after fine extraction is obtained and further mapped to the temperature matrix to obtain the temperature value representing the target area; Step S4: Match and associate the target device identified by the infrared image with the discharge point position of the visible light image, and combine the target device discharge point in the visible light image and the target device temperature value in the infrared image to determine the fault status of the device: Map the discharge point position to the infrared temperature matrix and determine the fault type based on the df value distribution ratio on the center line TLC: Through the df function to point (x i ,y i ) corresponds to the temperature value w(x i ,y i ) and the discharge in the visible light image disch(x fi ,y fi ) to make a comprehensive judgment: , Setvalue is set according to the ambient temperature, where 0 means the temperature is normal and there is no discharge, 1 means there is a temperature rise but no discharge, 2 means the temperature is normal but there is discharge, and 3 means there is a temperature rise and discharge.

2. The power grid line defect analysis method integrating visible light and infrared image processing according to claim 1 is characterized by: Step S2 specifically includes the following steps: Step S21: using a convolutional neural network trained on visible light image samples with normal illumination as a decomposition network to decompose the visible light image S to obtain a reflected light image R and an incident light image I; Step S22: Using a convolutional neural network as an enhancement network, the illumination distribution of the incident light image I is estimated and the illumination of the incident light image I is enhanced to obtain an image I'. Simultaneously, the noise in the reflected image R is suppressed to obtain an image R'. The adjusted reflected image R' is multiplied by the incident light image I' to obtain an enhanced visible light image S'. Step S23: After obtaining the enhanced visible light image, discharge points are detected and positions of the discharge points are marked.

3. The power grid line defect analysis method integrating visible light and infrared image processing according to claim 1 is characterized by: Step S4 specifically includes the following steps: Step S41: mapping the discharge point positions in the visible light image to the infrared image according to the pixel ratio between the infrared image and the visible light image; Step S42: Based on the device identified in the infrared image, the temperature of the device and the position of the discharge point are integrated to comprehensively determine the discharge heating defect, the internal heating defect and their severity.

4. The power grid line defect analysis method integrating visible light and infrared image processing according to claim 1 is characterized in that: The latest visible light and infrared images taken by the drone are extracted using a polling method, and the processed results are returned to the ground terminal using wireless transmission to assist operators in patrolling and checking.

5. The power grid line defect analysis method integrating visible light and infrared image processing according to claim 1 is characterized in that: In step S1, the edge computing terminal carried by the drone is used to extract the visible light and infrared images taken by the drone, and calculate the illumination value of the visible light image.

6. The power grid line defect analysis method integrating visible light and infrared image processing according to claim 5 is characterized in that: It also includes step S5: the edge computing terminal feeds back the processed captured images and inspection results in real time through the wireless communication module, and the ground handheld terminal displays and stores the images and inspection results.

Images