A high-voltage power line infrared image processing method and system based on an infrared camera

Abstract

The application provides a high-voltage wire infrared image processing method and system based on an infrared camera, wherein the highest temperature, the lowest temperature and the average temperature and other information of a corresponding marking area are obtained by clicking the marking on the picture according to point, line and rectangular marking; the line marking is used for displaying the temperature point trend graph on the line segment in the temperature trend display module; the pictures are automatically processed after setting the early warning temperature and threshold parameters through the batch processing picture function, and the pictures with temperature anomaly, normal and not meeting the processing requirements are saved separately, and the current processing progress is prompted through a progress bar; the function of providing an interactive interface for the infrared image shot by the unmanned aerial vehicle is realized, the operation is simple, the function is intuitive and the running is stable. After selecting a source folder, the application can quickly detect the wire and the drainage line area in the picture. The application is interactive, can quickly respond to various instructions and can intuitively display the final processing result.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for processing infrared images of high-voltage power lines based on an infrared camera. Background Technology

[0002] With the booming development of my country's drone industry, drones, with their advantages of small size, flexibility, and high flight altitude, are increasingly being used for high-altitude photography. Currently, using drones equipped with thermal imagers to detect the temperature of high-voltage power lines to ensure their safety has become an important technological means. However, the challenge lies in how to intuitively display and process the temperature information extracted from the captured images. This requires an interactive interface, which needs to be simple to operate, intuitive in function, and stable in operation. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for processing infrared images of high-voltage power lines based on an infrared camera, which provides an interactive interface for infrared images captured by a drone.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows: a method for processing infrared images of high-voltage power lines based on an infrared camera, comprising the following steps:

[0005] S1: The drone's thermal imager acquires infrared images;

[0006] S2: Preprocess the infrared image to clarify the visual features of linear targets in the infrared image and calibrate the object topology in the image pixel space; the specific steps are as follows:

[0007] S21: Perform grayscale conversion on the infrared image to obtain the corresponding grayscale image; then perform preprocessing on the grayscale image including denoising and smoothing; then remove non-target objects by determining the connected components of the grayscale image and extract the ROI region of the conductor and the drainage line.

[0008] S22: Perform morphological operations on the image based on the ROI regions of the conductors and drain lines to connect the insulator connected regions into one piece; then search for the largest connected region region in the image, which is the visually largest insulator part in the image;

[0009] S23: In the grayscale image, based on the position of the insulator and the number of conductors, starting from the boundary of the insulator on the conductor side and ending at the intersection of the conductor and the image boundary, count the number of pixels with higher grayscale values ​​on these lines. The line with the most pixels is the detected conductor.

[0010] S24: Extract the position of the conductor, and calculate the gray value of each pixel on the conductor in the original grayscale image for a second positioning, remove pixels with errors, and obtain accurate single-pixel conductor position information;

[0011] S3: Based on the actual features of linear targets in the image, perform targeted processing to complete semantic segmentation of linear targets and refined extraction of target regions.

[0012] According to the above scheme, the specific steps in step S21 are as follows:

[0013] S211: Separate the RGB channels of the infrared image to obtain a grayscale image;

[0014] S212: An adaptive grayscale thresholding algorithm is used to perform binary processing on the grayscale image for image denoising.

[0015] S213: The K-means algorithm is used to extract the ROI region from the denoised grayscale image, which is used to delineate and segment the main target contour of linear target classes.

[0016] Furthermore, in step S213, the specific steps are as follows:

[0017] S2131: Utilize the frequency characteristics of gray-level distribution in the histogram and set the number of peaks, valleys, and flat areas to determine the number of clusters k that the gray-level image needs to iterate.

[0018] S2132: Use the quantile strategy to set the center point of the initial grouping, and gradually realize the mean iteration within each unit to complete the segmentation;

[0019] All ungrouped gray values ​​in the processed grayscale image are arranged into a vector set X according to the principle of monotonically increasing;

[0020] Divide the set X into k equal parts according to the set value k. The position between each part is the position of the quantile. P i :

[0021] ;

[0022] S2133: Calculate the ideal number of classifications using the optimization criterion, verify the preset k value, and export the segmented infrared image after the ideal segmentation effect is met.

[0023] In a set of pixels, the better the segmentation effect is, the better the segmentation effect is, the smaller the internal difference of each group after classification, that is, the closer the value of each element is to the average value of the group, and the larger the difference between groups, that is, the greater the difference between the average values ​​of different groups.

[0024] The internal difference value within the same group is expressed as the standard deviation between the pixel values ​​of all elements in a group and the pixel value corresponding to the center point, i.e., the mean value within the group.

[0025] Let n be the number of all pixels in the set. C i Let x be the pixel value of each pixel in the i-th group, and Z be the average of all pixel values ​​in the i-th group. Then, the internal difference value of the entire pixel set is... S in for:

[0026] .

[0027] Furthermore, in step S22, the specific steps are as follows:

[0028] S221: The Canny operator is used to detect straight lines in the image, transforming the detection of a set of points with significant local changes in a grayscale image into edge detection;

[0029] S222: The Seed-Filling method is used to process the connected components to obtain the largest connected component, which is the location of the insulator.

[0030] Furthermore, in step S221, the specific steps are as follows:

[0031] S2211: Let (x, y) be the coordinates of a pixel in the image. Let be the Gaussian blur radius. σ Let I be the standard deviation of the normal distribution. A two-dimensional Gaussian function is used to smooth the image to obtain a data array I(x, y). This data array is used to suppress the adverse effects of image noise on edge detection performance. The formula for the two-dimensional Gaussian function is:

[0032] ;

[0033] In two-dimensional space, the contour lines of the surface generated by the above formula are concentric circles that are normally distributed starting from the center; the convolution matrix composed of pixels with non-zero distribution is transformed with the original image, and the value of each pixel is the weighted average of the values ​​of its surrounding neighboring pixels; the value of the original pixel has the maximum Gaussian distribution value and the maximum weight; the farther the neighboring pixel is from the original pixel, the smaller its weight;

[0034] S2212: Select 2 The gradient magnitude and direction of the smoothed data array I(x, y) are calculated by taking the finite difference mean of each pixel in the neighborhood, and the corresponding gradient magnitude image and gradient direction image are obtained.

[0035] The partial derivatives in the x and y directions are respectively:

[0036] ;

[0037] The gradient magnitude and gradient direction are as follows:

[0038] ;

[0039] S2213: The Canny algorithm is used to perform non-maximum suppression, detection and edge connection to obtain the edge point array of the image, and further edge thresholding is performed to remove false edges;

[0040] Perform orientation angle normalization on the gradient direction of the image to normalize it to four angles;

[0041] Suppress the amplitude of all non-ridge peaks existing in the gradient direction to achieve the effect of refining the gradient amplitude ridge in the amplitude array A[i,j].

[0042] Select a 3 A template with a 3-pixel window size and containing an 8-directional neighborhood is applied to all points of the magnitude array A[i,j]. At each point, the center pixel is compared with two pixels along the gradient line. If the magnitude A[i,j] at the center pixel is not greater than the magnitudes of two adjacent points along its gradient line, then A[i,j] is set to zero.

[0043] According to the above scheme, the specific steps in step S23 are as follows:

[0044] S231: Determine the location information of the insulator based on the largest connected component;

[0045] S232: Estimate the position of the boundary point of the conductor at the edge of the image based on the insulator's tilt.

[0046] S233: Determine the final conductor position by counting the high grayscale pixels between the conductor endpoint and the boundary endpoint at the insulator.

[0047] According to the above scheme, the specific steps in step S3 are as follows:

[0048] S31: For multiple images in the same folder, save the images to different result folders according to different temperature results for the same part;

[0049] S32: For a single image, select different point markers, line markers, and rectangle markers to mark the image; mark the drainage lines, conductors, and corresponding highest, lowest, and average temperatures in the single infrared image.

[0050] Furthermore, in step S32, the specific steps are as follows:

[0051] S321: For point marking, after the user clicks on the point marking in the image, the temperature value of the point is displayed. After clicking the point marking multiple times, the temperature information display area displays the information of the point marking, including the highest temperature, lowest temperature and average temperature.

[0052] S322: Line marking. After the user clicks on the start and end points on the image, a line segment is generated and displayed. The temperature trend display area displays the temperature trend map of the points on the line segment. The temperature information display area displays the information of the line segment, including the highest temperature, lowest temperature and average temperature. The specific locations of the highest and lowest temperatures are marked in the image.

[0053] S323: For rectangular marking, after the user clicks the start point and end point on the image, a rectangle is generated and displayed. The temperature information display area displays the information of the rectangle, including the highest temperature, lowest temperature and average temperature, and marks the specific location of the highest temperature and lowest temperature on the image.

[0054] An infrared image processing system for high-voltage power lines based on an infrared camera includes an image folder selection module, an image saving module, a folder directory tree display module, an image processing display module, a marker type selection module, a temperature trend display module, a temperature information display module, an image information display module, and an image batch processing parameter setting module.

[0055] The image folder selection module is used to provide images for subsequent operations;

[0056] The image saving module is used to save the current image according to the selected path;

[0057] The folder directory tree display module is used to select images in the image folder by path;

[0058] The image processing and display module is used to label the drainage line, conductor, and corresponding maximum, minimum, and average temperature information in the selected image;

[0059] The marker category module works in conjunction with the image processing and display module. By clicking on the marker, the corresponding highest temperature, lowest temperature, and average temperature information are displayed on the image.

[0060] The temperature trend display module is used to display line segments on the image based on the line marking instructions, the positions of the start and end points, and to display the temperature distribution trend of all points on the line segment;

[0061] The temperature information display module is used to display the highest, lowest, and average temperatures of all points on an image based on point marking instructions and the positions of multiple points; it is also used to display the highest, lowest, and average temperatures of each line based on line marking instructions and the positions of multiple lines, and to display the temperature trends of different lines; it is also used to display the highest, lowest, and average temperatures of each rectangular area based on rectangular marking instructions and the positions of rectangular areas, and to mark the locations of the highest and lowest temperatures in the corresponding rectangular areas.

[0062] The image information display module is used to display relevant information about the current image;

[0063] The image batch processing parameter setting module is used to store images whose highest temperature meets the sum of the warning temperature value and the threshold.

[0064] A computer storage medium storing a computer program executable by a computer processor, the computer program executing a method for infrared image processing of high-voltage power lines based on an infrared camera.

[0065] The beneficial effects of this invention are as follows:

[0066] 1. This invention provides a method and system for processing infrared images of high-voltage power lines based on an infrared camera. By marking a single image with points, lines, and rectangles, clicking on a marker in the image retrieves information such as the highest, lowest, and average temperatures of the corresponding marked area. Line markers are used to display a temperature trend graph along the line segment in the temperature trend display module. A batch image processing function automatically processes images after setting warning temperatures and threshold parameters, separating and saving images with abnormal, normal, and unsuitable temperatures, while a progress bar indicates the current processing progress. This provides an interactive interface for infrared images captured by drones, offering simple operation, intuitive functionality, and stable performance.

[0067] 2. After selecting the source folder, this invention can quickly detect the wire and drainage line areas in the image.

[0068] 3. This invention is user-friendly, can quickly respond to various commands, and displays the final processing results intuitively. Attached Figure Description

[0069] Figure 1 This is a flowchart of the algorithm in an embodiment of the present invention.

[0070] Figure 2 This is a preprocessing flowchart of an embodiment of the present invention.

[0071] Figure 3 This is a flowchart illustrating the segmentation process according to an embodiment of the present invention.

[0072] Figure 4This is a diagram showing the ROI extraction effect of an embodiment of the present invention.

[0073] Figure 5 This is an edge detection effect diagram of an embodiment of the present invention.

[0074] Figure 6 This is a schematic diagram of the 4 and 8 neighborhoods in an embodiment of the present invention.

[0075] Figure 7 This is a flowchart of the seed filling algorithm according to an embodiment of the present invention.

[0076] Figure 8 This is the maximum connected region graph of the insulator according to an embodiment of the present invention.

[0077] Figure 9 This is a flowchart of the secondary positioning process according to an embodiment of the present invention.

[0078] Figure 10 This is a diagram showing the locations of the detected insulators and the found wires in an embodiment of the present invention.

[0079] Figure 11 This is an overall block diagram of the infrared image processing software according to an embodiment of the present invention.

[0080] Figure 12 This is an illustration of the effect of selecting an image folder by clicking the file button in the menu according to an embodiment of the present invention.

[0081] Figure 13 This is a diagram illustrating the effect of displaying images by clicking on a file directory tree diagram according to an embodiment of the present invention.

[0082] Figure 14 This is an image showing the effect of the algorithm in this embodiment of the invention finding the drainage line and the wire, displaying temperature information, and marking the positions of the highest and lowest temperatures on the image.

[0083] Figure 15 This is a diagram illustrating the effect of selecting point markers and displaying temperature information according to an embodiment of the present invention.

[0084] Figure 16 This is a diagram illustrating the effect of selecting line markers and displaying temperature information according to an embodiment of the present invention.

[0085] Figure 17 This is a diagram illustrating the effect of selecting a rectangular marker and displaying temperature information according to an embodiment of the present invention.

[0086] Figure 18 This is a diagram showing the batch image processing parameter settings according to an embodiment of the present invention.

[0087] Figure 19 This is a diagram showing the batch processing effect of an embodiment of the present invention.

[0088] Figure 20This is a diagram showing the end effect of batch processing classification in an embodiment of the present invention. Detailed Implementation

[0089] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0090] For infrared images captured by a drone's thermal imager, embodiments of this invention automatically mark drainage lines, conductors, and their highest, lowest, and average temperatures within a single infrared image. For multiple images in the same folder, this invention places images of different temperature results for the same location into different result folders; for a single image, different point markers, line markers, and rectangle markers are selected to mark the image. Users can click on point markers in the image to see the point's temperature value; after multiple clicks, the highest, lowest, and average temperatures of these points are displayed in the temperature information display area. For line markers, after the user clicks on the start and end points in the image, the software automatically draws a straight line, simultaneously displaying the temperature trend graph of points along that line in the temperature trend display area, and displaying the highest, lowest, and average temperatures of that line segment in the temperature information display area, while also marking the specific locations of the highest and lowest temperatures in the image. Similarly, for rectangular markers, after the user clicks on the start and end points on the image, the software automatically draws a rectangle. The temperature information display area shows the highest, lowest, and average temperatures of the line segment, and marks the specific locations of the highest and lowest temperatures on the image.

[0091] See Figure 1 and Figure 2 The high-voltage power line infrared image processing algorithm of this invention mainly includes two major processes: image preprocessing (feature extraction) and linear classification recognition. Preprocessing includes image channel separation, ROI extraction, edge detection, and connected component processing. Each linear classification requires targeted feature processing based on the characteristics of the conductor, ultimately obtaining the semantic segmentation region of each linear target. The specific steps are as follows:

[0092] S1: Acquire infrared images using the drone's built-in thermal imager;

[0093] S2: RGB channel separation;

[0094] S3: Adaptive grayscale thresholding is used to denoise images;

[0095] Noise in images often manifests as isolated pixels or pixel blocks with strong visual effects. Image noise can severely interfere with subsequent algorithmic processing. There are generally two sources of noise: First, during image acquisition, the sensor material properties, operating environment, electronic components, and circuit structure can introduce various types of noise, such as thermal noise caused by resistance, channel thermal noise of field-effect transistors, photon noise, dark current noise, and non-uniformity noise in photoresponse. Second, during image preprocessing, the non-uniform intensity amplification caused by feature enhancement can result in isolated pixels or pixel patches. In a grayscale image, if the probability density function of a certain type of noise exhibits a normal distribution, it can be called Gaussian noise. The time-varying average and the covariance function between two instants can be used to determine Gaussian noise. If the noise is stationary, the average value is independent of time and is equivalent to the power spectral density. Gaussian noise can be generated by a large number of independent pulses, so that within any finite time interval, the value of each pulse is negligible compared to the sum of all pulse values. The stationary noise represents an average value that is unrelated to time, while the covariance function becomes a correlation function that depends only on the difference between the two instants being considered. It functions similarly to the power spectral density, and its probability density function is as follows:

[0096] ;

[0097] In addition to the noise in linear objects, there are also interference items in the image background, such as clouds and the sun. After statistically analyzing the grayscale values ​​in the image, it was found that the tower, clouds, and sun in the image are layered, with the sun generally having the highest grayscale value, followed by the tower, and the clouds having relatively lower grayscale values. Therefore, an adaptive grayscale thresholding algorithm was used for image binary processing to filter out these background interference items.

[0098] S4: Image ROI Region Extraction; ROI extraction mainly involves delineating and segmenting the contours of main targets within linear object classes, providing a foundation for further image preprocessing. Several image segmentation methods have been developed, including threshold-based, edge-based, and clustering analysis or fuzzy set theory-based methods. Clustering refers to classifying a group of research objects into several groups of units with the same characteristics. Clustering is commonly used to partition unlabeled datasets. The K-means algorithm is a classic and commonly used clustering method.

[0099] Linear target recognition can utilize an improved k-means algorithm in infrared image processing. The target infrared image has already undergone simple preprocessing, and the processed image serves as the grayscale image imported in the first step of the algorithm. Based on the K-means algorithm, the number of clusters (k) to be iterated is set using the frequency characteristics of the grayscale distribution in the histogram and the number of peaks, valleys, and flat areas. Next, a quantile strategy is used to set the initial group center points, and then the mean within each unit is iterated step by step until segmentation is complete. Finally, an optimization criterion is used to calculate the ideal number of classifications, verifying the preset k value from the first step. Once the ideal segmentation effect is achieved, the segmented infrared image is exported. This criterion is used to overcome the shortcomings of randomly selecting cluster values ​​and initial cluster centers, verifying more suitable cluster values ​​and more complete segmentation results, thus improving practicality and reliability. The process is as follows: Figure 3 As shown.

[0100] First, arrange all ungrouped grayscale values ​​in the simply processed grayscale image into a vector set X according to the principle of monotonically increasing values. Then, divide this set into k equal parts according to a calculated value k. The position between each part is the position of the quantile, and its corresponding value is Pi. Its calculation formula is as follows:

[0101] ;

[0102] Assuming the selected image has a k value of 6, after sorting and dividing, we obtain 5 quartiles. The value of the first quartile is equal to the value at one-sixth of the total gray values ​​in the set, arranged in monotonically ascending order. This value corresponds to P1, and so on, with P2, P3, P4, and P5 calculated in the same way. To achieve a more ideal segmentation effect, within a pixel set, the internal difference of each group after classification should be minimized (i.e., the value of each element should be as close as possible to the average of that group), while the difference between groups should be maximized (i.e., the larger the difference between the average values ​​of different groups). The internal difference value of the same group is represented by the standard deviation between the pixel values ​​of all elements in that group and the pixel value corresponding to the center point, i.e., the average value within the group. The expression for the internal difference value of the entire pixel set is:

[0103] ;

[0104] In the formula: n is the total number of pixels in the set, x represents the pixel value of each element in the i-th group, and Z is the average of all pixel values ​​in the i-th group. The result is as follows: Figure 4 As shown.

[0105] S5: Edge Detection; The Canny operator is used to detect straight lines in the image, transforming the detection of a set of points with significant local changes in a grayscale image into edge detection. The local peaks of the first derivative amplitude of the image's grayscale profile correspond to step edges, which are also the zero-crossing points of the corresponding second derivative. Numerically, the gradient is a measure of the change in a function, and an image can be viewed as a series of sampling points where the image intensity function changes continuously. Therefore, if there are significantly changing grayscale values ​​in an image, they are detected using a discrete approximation function of the gradient. The gradient is expressed as a two-dimensional equivalent of the first derivative, namely:

[0106] ;

[0107] Specifically, absolute values ​​are used to approximate gradient magnitudes. The gradient direction is represented as .

[0108] a) Image smoothing

[0109] Image edge detection algorithms are mainly based on the calculation of the first and second derivatives of image grayscale. However, the calculation of derivatives is easily affected by noise. Therefore, filters must be used first to suppress the adverse effects of noise on the relevant edge detection performance.

[0110] The Canny edge detector is an optimal approximation operator for the product of signal-to-noise ratio and localization. It derives the first derivative form of a Gaussian function using the normative differentiation method. Due to the commutative and associative properties of convolution operations, the detection method first uses a two-dimensional Gaussian function... This is used to smooth the image, thereby suppressing image noise.

[0111] b. Calculation of gradient magnitude and direction

[0112] After image smoothing, to obtain the corresponding gradient magnitude and gradient direction images, traditional detection algorithms use a selection of 2... The gradient magnitude and direction of the smoothed data array I(x, y) are calculated by taking the finite difference mean of each pixel in the neighborhood. The partial derivatives in the two directions are:

[0113] ;

[0114] The formulas for calculating gradient magnitude and gradient direction are respectively

[0115] ;

[0116] c. Nonmaximum suppression, detection, and edge connection

[0117] The Canny algorithm first normalizes the gradient directions of the image, then normalizes them to four angles. Next, it suppresses the amplitude of all non-ridge (feature contour lines along the gradient direction) peaks along the gradient direction, thus thinning the gradient amplitude ridges in the amplitude array A[i,j]. A 3... A template with a 3-pixel window size and containing an 8-directional neighborhood is applied to all points in the magnitude array A[i,j]. At each point, the center pixel is compared with two pixels along the gradient line. If the magnitude A[i,j] at the center pixel is not greater than the magnitudes of the two adjacent points along its gradient line, then A[i,j] is set to zero. After the above processing, the wide ridge can be thinned to only a single pixel width. Therefore, in the non-maximum suppression process, the width of the ridge is thinned, while its height is preserved.

[0118] Non-maximum suppression ultimately yields an array of edge points in the image. However, this edge array inevitably contains various false edge information caused by noise or texture. Therefore, further edge thresholding is required to further remove false edges. The results are as follows... Figure 5 As shown.

[0119] S6: Select the Seed-Filling method to obtain the largest connected component, which is the location of the insulator;

[0120] After ROI extraction and edge detection, linear images can be basically divided into regions for the main target objects in the image. Based on the maximum connected component, the location information of the insulator can be determined first. After determining the location, the boundary point position of the conductor at the edge of the image can be estimated based on the insulator's tilt. Finally, the final conductor position is determined by counting the high grayscale pixels between the conductor endpoint at the insulator and the boundary endpoint.

[0121] Since binary images typically only have two grayscale values, 0 and 255, common connected regions are 4-adjacent and 8-adjacent (e.g., ...). Figure 6 (As shown) There are two main methods for analyzing connected components: the Two-Pass method and the Seed-Filling method. We choose the Seed-Filling method to find the largest connected component. The flowchart is as follows. Figure 7 As shown.

[0122] Insulators and conductors have a strong spatial topological correlation. Processing the maximum connected component of the insulator can improve the positional reference for conductor / drainage line determination, thereby enhancing the robustness of linear target detection. The results are shown in the figure below. Figure 8 As shown.

[0123] S7: Conductor Recognition; Through the above image preprocessing, the visual features of linear targets in the infrared image are clarified, and the object topology in the image pixel space is labeled. Subsequent processing can be tailored to the actual characteristics of the linear targets to complete semantic segmentation and refined extraction of the target region. Based on the analysis of linear target samples, the specific processing flow of the conductor recognition algorithm is as follows:

[0124] a. By acquiring the infrared image, the image is first converted to grayscale to obtain the corresponding grayscale image. Then, the grayscale image is preprocessed by denoising and smoothing. Finally, the ROI regions of the conductor and drain line are extracted by judging the connected components in the grayscale image and removing towers, etc.

[0125] b. Based on the ROI region images of the extracted conductors and drain lines, perform morphological operations on the images to connect the insulator connected regions into one piece. Then, search for the largest connected region region in the image, which is the visually largest insulator part in the image.

[0126] c. In the preprocessed grayscale image, after determining the position of the insulator and counting the number of conductors, the starting point is the boundary of the insulator on the conductor side, and the ending point is the intersection of the conductor and the image boundary. The number of pixels with higher grayscale values ​​on these lines is counted. The conductor with the most pixels is the detected conductor.

[0127] d. Since the grayscale image obtained through preprocessing undergoes morphological dilation, the resulting guideline information may deviate slightly from the guideline information in the original image. Therefore, the guideline positions are extracted, and the grayscale values ​​of the pixels on the guideline are sequentially calculated in the original grayscale image for a second positioning process. Pixels with errors are then removed to obtain more accurate single-pixel guideline position information. The secondary positioning process is as follows: Figure 9 As shown, the final algorithm detection results are as follows: Figure 10 As shown.

[0128] See Figure 11 The embodiments of the present invention include an image folder selection module, an image saving module, a folder directory tree display module, an image processing display module, a tag type selection module, a temperature trend display module, a temperature information display module, an image information display module, an image batch processing parameter setting module, and a fast detection module.

[0129] The image folder selection module provides images for subsequent operations. There are two ways to select an image folder. The first is to click the "File" button in the menu, then click "Open Folder," select the image folder path in the pop-up file directory, and click "OK." You can then see the expanded path tree in the file tree view on the left side of the interface. (See [link]). Figure 12The second method is to directly click on the file tree directory on the left, expanding layer by layer until the image is opened. See [link / reference]. Figure 13 .

[0130] To save the image, click the "File" button in the menu, then click the "Save As" button. In the pop-up file directory, select the path and click "OK" to save the current image to that path.

[0131] The folder directory tree display module is used to select other images under the image folder and to select the image folder path and display it.

[0132] The image processing and display module allows you to select an image, click the "Process Image" button, and the background algorithm will automatically mark the drainage lines and conductors on the image. Additionally, clicking "Clear Image," then selecting different markers, and clicking on the image will display the highest, lowest, and average temperature information accordingly. See [link to relevant documentation]. Figure 14 .

[0133] The marker category module works in conjunction with the image processing and display module. Clicking on a marker will display the highest, lowest, and average temperature information on the image.

[0134] The temperature trend display module is used to select line markers and then click on the start and end points on the image; a straight line will then appear on the image. Simultaneously, the lower center area of ​​the software interface will display the temperature distribution trend of all points along that line.

[0135] The temperature information display module, when multiple points are selected and clicked on in the image, displays the highest, lowest, and average temperatures for all points. (See also...) Figure 15 When you select line markers and draw multiple lines on the image, this module displays the highest, lowest, and average temperatures for each line. Additionally, selecting different lines will display their temperature trends in the temperature trend module. See also... Figure 16 When a rectangular marker is selected, this module displays the highest, lowest, and average temperatures for each rectangular area. It also marks the locations of the highest and lowest temperatures within the corresponding rectangular areas. See [link / reference]. Figure 17 .

[0136] The image information display module is used to display relevant information about the current image.

[0137] See Figure 18 and Figure 20 The image batch processing parameter settings module, after selecting the path by clicking the "..." button, automatically creates three folders: ok, ng, and temp. These folders store images whose highest temperature matches the sum of the warning temperature value and the threshold, images that do not meet the threshold, and abnormal images that do not satisfy the algorithm's processing requirements. After clicking the "Start Batch Processing" button, a processing progress dialog box appears. You can click the cancel button to end the batch processing early. See [link to documentation]. Figure 19 .

[0138] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for processing infrared images of high-voltage power lines based on an infrared camera, characterized in that: Includes the following steps: S1: The drone's thermal imager acquires infrared images; S2: Preprocess the infrared image to clarify the visual features of linear targets in the infrared image and calibrate the object topology in the image pixel space; the specific steps are as follows: S21: Perform grayscale conversion on the infrared image to obtain the corresponding grayscale image; then perform preprocessing on the grayscale image including denoising and smoothing; then remove non-target objects by determining the connected components of the grayscale image and extract the ROI region of the conductor and the drainage line. S22: Perform morphological operations on the image based on the ROI regions of the conductors and drain lines to connect the insulator connected regions into one piece; then search for the largest connected region region in the image, which is the visually largest insulator part in the image; S23: In the grayscale image, based on the position of the insulator and the number of conductors, starting from the boundary of the insulator on the conductor side and ending at the intersection of the conductor and the image boundary, count the number of pixels with higher grayscale values ​​on these lines. The line with the most pixels is the detected conductor; the specific steps are as follows: S231: Determine the location information of the insulator based on the largest connected component; S232: Estimate the position of the boundary point of the conductor at the edge of the image based on the insulator's tilt. S233: The final conductor position is determined by statistically analyzing the high grayscale pixels between the conductor endpoint and the boundary endpoint at the insulator. S24: Extract the position of the conductor, and calculate the gray value of each pixel on the conductor in the original grayscale image for a second positioning, remove pixels with errors, and obtain accurate single-pixel conductor position information; S3: Based on the actual features of linear targets in the image, perform targeted processing to complete semantic segmentation of linear targets and refined extraction of target regions.

2. The infrared image processing method for high-voltage power lines based on an infrared camera according to claim 1, characterized in that: The specific steps in step S21 are as follows: S211: Separate the RGB channels of the infrared image to obtain a grayscale image; S212: An adaptive grayscale thresholding algorithm is used to perform binary processing on the grayscale image for image denoising. S213: The K-means algorithm is used to extract the ROI region from the denoised grayscale image, which is used to delineate and segment the main target contour of linear target classes.

3. The infrared image processing method for high-voltage power lines based on an infrared camera according to claim 2, characterized in that: The specific steps in step S213 are as follows: S2131: Utilize the frequency characteristics of gray-level distribution in the histogram and set the number of peaks, valleys, and flat areas to determine the number of clusters k that the gray-level image needs to iterate. S2132: Use the quantile strategy to set the center point of the initial grouping, and gradually realize the mean iteration within each unit to complete the segmentation; All ungrouped gray values ​​in the processed grayscale image are arranged into a vector set X according to the principle of monotonically increasing; Divide the set X into k equal parts according to the set value k. The position between each part is the position of the quantile. P i : ； S2133: Calculate the ideal number of classifications using the optimization criterion, verify the preset k value, and export the segmented infrared image after the ideal segmentation effect is met. In a set of pixels, the better the segmentation effect is, the better the segmentation effect is, the smaller the internal difference of each group after classification, that is, the closer the value of each element is to the average value of the group, and the larger the difference between groups, that is, the greater the difference between the average values ​​of different groups. The internal difference value within the same group is expressed as the standard deviation between the pixel values ​​of all elements in a group and the pixel value corresponding to the center point, i.e., the mean value within the group. Let n be the number of all pixels in the set. C i Let x be the pixel value of each pixel in the i-th group, and Z be the average of all pixel values ​​in the i-th group. Then, the internal difference value of the entire pixel set is... S in for: 。 4. The infrared image processing method for high-voltage power lines based on an infrared camera according to claim 2, characterized in that: The specific steps in step S22 are as follows: S221: The Canny operator is used to detect straight lines in the image, transforming the detection of a set of points with significant local changes in a grayscale image into edge detection; S222: The Seed-Filling method is used to process the connected components to obtain the largest connected component, which is the location of the insulator.

5. The infrared image processing method for high-voltage power lines based on an infrared camera according to claim 4, characterized in that: The specific steps in step S221 are as follows: S2211: Let (x, y) be the coordinates of a pixel in the image. For the fuzzy radius, σ Let I be the standard deviation of the normal distribution. A two-dimensional Gaussian function is used to smooth the image to obtain a data array I(x, y). This data array is used to suppress the adverse effects of image noise on edge detection performance. The formula for the two-dimensional Gaussian function is: ； In two-dimensional space, the contour lines of the surface generated by the above formula are concentric circles that are normally distributed starting from the center; the convolution matrix composed of pixels with non-zero distribution is transformed with the original image, and the value of each pixel is a weighted average of the values ​​of its surrounding neighboring pixels; the value of the original pixel has the maximum Gaussian distribution value and the maximum weight, and the farther the neighboring pixel is from the original pixel, the smaller its weight; S2212: Select 2 The gradient magnitude and direction of the smoothed data array I(x, y) are calculated by taking the finite difference mean of each pixel in the neighborhood, and the corresponding gradient magnitude image and gradient direction image are obtained. The partial derivatives in the x and y directions are respectively: ； The gradient magnitude and gradient direction are as follows: ； S2213: The Canny algorithm is used to perform non-maximum suppression, detection and edge connection to obtain the edge point array of the image, and further edge thresholding is performed to remove false edges; Perform orientation angle normalization on the gradient direction of the image to normalize it to four angles; Suppress the amplitude of all non-ridge peaks existing in the gradient direction to achieve the effect of refining the gradient amplitude ridge in the amplitude array A[i,j]. Select a 3 A template with a 3-pixel window size and containing an 8-directional neighborhood is applied to all points of the magnitude array A[i,j]. At each point, the center pixel is compared with two pixels along the gradient line. If the magnitude A[i,j] at the center pixel is not greater than the magnitudes of two adjacent points along its gradient line, then A[i,j] is set to zero.

6. The infrared image processing method for high-voltage power lines based on an infrared camera according to claim 1, characterized in that: The specific steps in step S3 are as follows: S31: For multiple images in the same folder, save the images to different result folders according to different temperature results for the same part; S32: For a single image, select different point markers, line markers, and rectangle markers to mark the image; mark the drainage lines, conductors, and corresponding highest, lowest, and average temperatures in the single infrared image.

7. The infrared image processing method for high-voltage power lines based on an infrared camera according to claim 6, characterized in that: The specific steps in step S32 are as follows: S321: For point marking, after the user clicks on the point marking in the image, the temperature value of the point is displayed. After clicking the point marking multiple times, the temperature information display area displays the information of the point marking, including the highest temperature, lowest temperature and average temperature. S322: Line marking. After the user clicks on the start and end points on the image, a line segment is generated and displayed. The temperature trend display area displays the temperature trend map of the points on the line segment. The temperature information display area displays the information of the line segment, including the highest temperature, lowest temperature and average temperature. The specific locations of the highest and lowest temperatures are marked in the image. S323: For rectangular marking, after the user clicks the start point and end point on the image, a rectangle is generated and displayed. The temperature information display area displays the information of the rectangle, including the highest temperature, lowest temperature and average temperature, and marks the specific location of the highest temperature and lowest temperature on the image.

8. A system for the infrared image processing method for high-voltage power lines based on an infrared camera as described in any one of claims 1 to 7, characterized in that: It includes modules for selecting image folders, saving images, displaying folder directories in a tree structure, processing images, selecting tag types, displaying temperature trends, displaying temperature information, displaying image information, and setting batch image processing parameters. The image folder selection module is used to provide images for subsequent operations; The image saving module is used to save the current image according to the selected path; The folder directory tree display module is used to select images in the image folder by path; The image processing and display module is used to label the drainage line, conductor, and corresponding maximum, minimum, and average temperature information in the selected image; The marker category module works in conjunction with the image processing and display module. By clicking on the marker, the corresponding highest temperature, lowest temperature, and average temperature information are displayed on the image. The temperature trend display module is used to display line segments on the image based on the line marking instructions, the positions of the start and end points, and to display the temperature distribution trend of all points on the line segment; The temperature information display module is used to display the highest, lowest, and average temperatures of all points on an image based on point marking instructions and the positions of multiple points; it is also used to display the highest, lowest, and average temperatures of each line based on line marking instructions and the positions of multiple lines, and to display the temperature trends of different lines; it is also used to display the highest, lowest, and average temperatures of each rectangular area based on rectangular marking instructions and the positions of rectangular areas, and to mark the locations of the highest and lowest temperatures in the corresponding rectangular areas. The image information display module is used to display relevant information about the current image; The image batch processing parameter setting module is used to store images whose highest temperature meets the sum of the warning temperature value and the threshold.

9. A computer storage medium, characterized in that: It contains a computer program that can be executed by a computer processor, which performs a high-voltage power line infrared image processing method based on an infrared camera as described in any one of claims 1 to 7.

Images