Transmission line linear infrared-like image processing and recognition system and method

Abstract

The application discloses a kind of transmission line linear infrared image processing and identification system, RGB channel separation module is used to carry out RGB channel separation to high-voltage electric wire infrared image, and the corresponding G channel picture is obtained;ROI extraction module is used to extract the ROI area of the G channel picture;Adaptive gray threshold processing module carries out OTSU method processing to ROI area picture, and obtains the ROI area picture after adaptive gray threshold processing;Edge extraction module obtains the edge contour picture of ROI area using edge detection canny operator;Insulator position acquisition module obtains ROI area difference image, and finds the maximum contour of ROI area difference image.The application can efficiently analyze and identify the transmission line picture collected by unmanned aerial vehicle, and can intuitively show the change of temperature.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a power transmission line linear infrared image processing and recognition system and method. BACKGROUND

[0002] With the vigorous development of China's unmanned aerial vehicle industry, unmanned aerial vehicles have the advantages of being small, flexible and having a high flight altitude, so many high-altitude shooting operations are carried out by unmanned aerial vehicles. Nowadays, using unmanned aerial vehicles to carry thermal imagers to detect whether the temperature of high-voltage power lines is normal so as to ensure the safety of high-voltage power lines has become an important technical means. However, how to intuitively display and recognize the image information parsed from the collected pictures is an urgent development field. Currently, desktop infrared temperature display software only displays the maximum, minimum and average temperature values and cannot intuitively display the temperature conditions of power transmission lines. SUMMARY

[0003] The purpose of the present application is to provide a power transmission line linear infrared image processing and recognition system and method, which can efficiently and accurately recognize the power transmission line pictures collected by unmanned aerial vehicles, thereby facilitating the intuitive display of the temperature conditions of power transmission lines.

[0004] To achieve this purpose, the power transmission line linear infrared image processing and recognition system designed by the present application comprises an RGB channel separation module, an ROI extraction module, an adaptive gray threshold processing module, an edge extraction module and an insulator position acquisition module.

[0005] The RGB channel separation module is used for RGB channel separation of high-voltage power line infrared images to obtain corresponding G channel pictures.

[0006] The ROI extraction module is used for extracting the ROI region (region of interest) of the G channel picture to obtain an ROI region picture.

[0007] The adaptive gray threshold processing module is used for OTSU method processing of the ROI region picture to determine an adaptive gray threshold t, and the adaptive gray threshold t is used for binarization of the ROI region picture to obtain an adaptive gray threshold processed ROI region picture.

[0008] The edge extraction module is used for edge extraction of the adaptive gray threshold processed ROI region picture by using an edge detection canny operator to obtain an edge contour picture of the ROI region.

[0009] The insulator location acquisition module is used to subtract the pixel value of each corresponding pixel in the edge contour image of the ROI region after adaptive grayscale threshold processing from the pixel value of the ROI region to obtain the ROI region difference image. The maximum contour found in the ROI region difference image is the location of the maximum insulator.

[0010] The beneficial effects of this invention are:

[0011] This invention comprises two main processes: image preprocessing (feature extraction) and linear classification recognition. Preprocessing includes image channel separation, ROI extraction, edge detection, and connected component processing. Each linear classification requires targeted feature processing based on the characteristics of the power lines, ultimately yielding the semantic segmentation regions of each linear target. This invention can efficiently analyze and recognize images of power transmission lines captured by drones, thus providing a clear view of the power transmission line temperature. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the structure of the present invention;

[0013] Figure 2 This is a schematic diagram of the RGB channel separation results of the present invention;

[0014] Figure 3 This is a schematic diagram illustrating the effect of ROI extraction according to the present invention;

[0015] Figure 4 This is a schematic diagram of the edge detection processing results of the present invention;

[0016] Figure 5 This is a schematic diagram of the adaptive threshold processing results of this method;

[0017] Figure 6 This is a schematic diagram of the maximum connected region of the insulator of the present invention;

[0018] Figure 7 This is a schematic diagram of the insulator detected by the present invention;

[0019] Figure 8 This is a schematic diagram of the wire location found by the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0021] like Figures 1-8 The linear infrared image processing and recognition system for power transmission lines shown is characterized by comprising an RGB channel separation module, an ROI extraction module, an adaptive grayscale threshold processing module, an edge extraction module, and an insulator position acquisition module.

[0022] The RGB channel separation module is configured to separate the high-voltage wire infrared image into RGB channels to obtain a corresponding G channel picture (R refers to red, G refers to green, and B refers to blue).

[0023] The ROI extraction module is configured to extract an ROI region of the G channel picture to obtain an ROI region picture (the ROI region includes insulators and wires, etc.).

[0024] The adaptive gray threshold processing module is configured to perform OTSU processing on the ROI region picture to determine an adaptive gray threshold t (since the shooting habits of the person shooting are different, the gray of the G channel image obtained in step S2 can be overall high or low, and setting a fixed threshold can result in poor robustness of the algorithm), perform binarization on the ROI region picture by using the adaptive gray threshold t, and obtain an ROI region picture processed by adaptive gray threshold processing (after adaptive gray threshold processing, the region that is not needed is excluded, and after otsu processing, the picture obtained does not contain clouds, only insulators and wires. Since the gray value of the clouds is much lower than that of the insulators and wires, the clouds are filtered out by the otsu algorithm).

[0025] The edge extraction module is configured to perform edge extraction on the ROI region picture processed by adaptive gray threshold processing by using an edge detection canny operator to obtain an edge contour picture of the ROI region. The canny edge detection only detects edges, and only retains the outermost layer of pixels, and also retains a long and thin single-pixel line, since it is considered as an edge.

[0026] The insulator position acquisition module is configured to subtract the pixel value of each corresponding pixel point in the ROI region picture processed by adaptive gray threshold processing from the edge contour picture of the ROI region to obtain an ROI region difference image. The maximum contour found in the ROI region difference image is the position of the largest insulator. Since the canny detection only retains the outermost layer of pixels, for the insulator, the removal of the layer of pixels has little effect, since the insulator is a large connected domain, and for the wire, since the wire is basically a single-pixel long and thin pixel line in the image, the wire is removed after the subtraction. Therefore, in the difference image, there is no wire, only the insulator is left.

[0027] The technical scheme further comprises a wire position acquisition module and a temperature value acquisition module, the wire position acquisition module is configured to determine the wire position by region growing method according to the position of the insulator. The basic idea of region growing is to collect pixels with similar properties to form a region. A seed pixel is found in the region as the starting point of growth, and then the pixels with pixel values of 250-255 in the neighborhood of the seed pixel are merged into the region where the seed pixel is located. These new pixels are taken as new seed pixels to continue the above process until there is no pixel meeting the condition to be included. Because the pixel values of the wire region change little, the whole wire can be found by using the method.

[0028] The temperature value acquisition module is configured to acquire the temperature value of any pixel point in the high-voltage wire infrared image, and acquire the temperature values of all points at the position of the largest insulator and the wire position in combination with the position of the largest insulator and the wire position.

[0029] The RGB channel separation module is configured to separate the channels of the high-voltage wire infrared image by using the split() function of opencv, the brightness of the high-temperature position in the infrared image is high, the image is whitened, and the details are not clear. By separating the G channel of the infrared image, the G channel image most sensitive to temperature change can be obtained, and the details of the high-temperature position image can be seen clearly. In the obtained G channel image, the image is not whitened, and the details of the elements in the wire and cloud images are clearer, which is convenient for subsequent algorithm processing.

[0030] The insulator position acquisition module is configured to subtract the pixel value of each pixel point in the ROI region image after adaptive grayscale threshold processing from the edge contour image of the ROI region, to obtain a ROI region difference image, and find the largest contour of the ROI region difference image by using the findcontours() function in the opencv software, which is the position of the insulator. The ROI region image after threshold processing has wires, and the difference image has no or only a few pixels on the wires, and the remaining large connected regions are insulators. The largest insulator can be found more easily by using the difference image.

[0031] In the technical solution, the specific method of the ROI extraction module for extracting the ROI region of the G channel picture is as follows: since the tower always appears on the left side or the right side of the image and the pixel value is 255, starting from the leftmost side or the rightmost side of the G channel picture, the sum of pixel values on the same vertical line of the G channel picture is calculated in sequence, and when the difference between the sum of pixel values of a vertical line and the sum of pixel values of an adjacent vertical line is greater than a threshold value, the vertical line pixel value sum is a mutation point, the horizontal coordinate x of the mutation point is recorded, the length of the G channel picture is set as m, and an image with a length of (m-x) is intercepted as the ROI region picture. Since the tower is not the target to be detected and appears regularly (only on the leftmost side or the rightmost side of the image), the interference of the tower is excluded by the obtained image, so that the subsequent detection of the conductor is more convenient.

[0032] In the technical solution, the specific method of the adaptive gray threshold processing module for processing the ROI region picture by OTSU method (Otsu method-maximum between-class variance method) to determine the adaptive gray threshold t is as follows: assuming that the adaptive gray threshold t n , the adaptive gray threshold t n is obtained, the between-class variance of the two images after the ROI region picture is segmented at each adaptive gray threshold t i , and the adaptive gray threshold t that makes the between-class variance of the two images after the ROI region picture is segmented maximum at each adaptive gray threshold t i is the adaptive gray threshold t.

[0033] In the technical solution, the specific method of the adaptive gray threshold processing module for performing binaryzation on the ROI region picture by using the adaptive gray threshold t is as follows: the pixels in the ROI region picture are converted into white pixels with a pixel value of 255 and black pixels with a pixel value of 0 by using the adaptive gray threshold t. Each pixel of the image is traversed from left to right and from top to bottom, and if the pixel value of the current pixel point is greater than t, the pixel value of the current pixel point is re-assigned as 255, otherwise, the pixel value is re-assigned as 0. After the traversal is completed, the obtained image only contains two kinds of pixel values of 0 and 255. The image becomes a binary image, and the process is called binaryzation of the image.

[0034] In the technical solution, the specific method of the edge extraction module for performing edge extraction on the ROI region picture after the adaptive gray threshold processing by using the edge detection canny operator to obtain the edge contour picture of the ROI region is as follows:

[0035] Firstly, Gaussian filter is used to smooth the image. The edge profile picture of ROI region refers to the part of the picture where the local region brightness changes more than the preset value. The gray profile of this region is regarded as a step, that is, it sharply changes from one gray value to another gray value with a large difference in a small buffer area. The ROI region picture is smoothed by two-dimensional Gaussian function to remove noise by Gaussian filter.

[0036]

[0037] wherein G(x, y) represents the gradient of the pixel value represented by the point (x, y) in the ROI region picture, x represents the horizontal coordinate, y represents the vertical coordinate, and σ represents the standard deviation. The Gaussian filter denoising is to perform weighted average on the pixel values of the entire ROI region picture. The value of each pixel point is obtained by weighted average of its own value and other pixel values in the neighborhood.

[0038] Then, gradient and gradient direction calculation is performed. Assuming that the gradient values of the filtered image in the horizontal and vertical directions are G x (i, j) and G y (i, j), respectively, the gradient value and the gradient direction of each pixel point are:

[0039]

[0040]

[0041] Non-maximum suppression. In the gradient direction, if the gradient amplitudes of the two adjacent (horizontally adjacent to the left and right, and vertically adjacent to the top and bottom) pixel points of the current pixel point are both greater than the gradient amplitude of the current pixel point, the current pixel point is marked as a non-edge point. Otherwise, the current pixel point is marked as an edge candidate point.

[0042] The ROI region picture is traversed. If the gradient amplitude of the pixel point is greater than the high threshold value of the canny operator, it is determined as an edge point. If the gradient amplitude of the pixel point is less than the low threshold value of the canny operator, it is determined as a non-edge point. If the gradient amplitude of the pixel point is between the high and low threshold values of the canny operator, it is determined whether there is a point greater than the high threshold value of the canny operator in its 8-neighborhood. If there is, it is an edge point. Otherwise, it is a non-edge point.

[0043] The Canny operator is provided with two threshold values, a low threshold value and a high threshold value, the Canny operator can calculate a gradient value at each pixel position, referred to as an "actual gradient", when the "actual gradient > high threshold value", the point is taken as an "edge point"; when "actual gradient < low threshold value", the point is taken as a "non-edge point (also called a background point)"; when "low threshold value < actual gradient < high threshold value", it needs to be distinguished, if the surrounding adjacent points are all greater than the low threshold value, the point is taken as a background point (non-edge point). If the surrounding adjacent points are all greater than the low threshold value, the gradient of the adjacent point and the point is compared, and the one with the greater gradient is taken as an edge point.

[0044] In the technical solution, the edge detection algorithm of the image is mainly based on the first-order and second-order derivative calculation of the image gray scale, but the calculation process of the derivative is easily affected by noise, and therefore a filter must be used to suppress the adverse effects of the noise factor on the edge detection performance.

[0045] In the technical solution, the specific method for acquiring the position of the conductor by the insulator position acquisition module through the region growing method is as follows:

[0046] Any point in the maximum insulator position determined in the G channel image is taken as one end of the conductor, and is defined as a seed, the pixel value of the seed is a (255), the pixel points with pixel values of a-5~a in the 8-neighborhood of the seed are added to the seed, so that the number and range of the seed are continuously increased, until the boundary of the image is found, that is, the position of the conductor is obtained. After the position of the maximum insulator is found, as long as an arbitrary pixel point is taken, the region growing algorithm can continuously find the pixel region similar to the maximum insulator pixel, and the conductor is not only connected to the maximum insulator region, but also has a similar pixel value, so it is contained.

[0047] A power line linear infrared image processing and recognition method, which comprises the following steps:

[0048] Step 1: collecting a high-voltage power line infrared image by using a UAV, performing RGB channel separation on the high-voltage power line infrared image, and obtaining a corresponding G channel image;

[0049] Step 2: extracting an ROI region of the G channel image to obtain an ROI region image;

[0050] Step 3: performing OTSU method processing on the ROI region image to determine an adaptive gray threshold t, performing binaryzation on the ROI region image by using the adaptive gray threshold t to obtain an ROI region image processed by the adaptive gray threshold;

[0051] Step 4: using edge detection canny operator to extract the edge of the ROI region picture after adaptive gray threshold processing, get the edge profile picture of the ROI region;

[0052] Step 5: subtract the pixel value of each pixel point in the ROI region picture after adaptive gray threshold processing and the edge profile picture of the ROI region, get the ROI region difference image, find the maximum profile of the ROI region difference image, which is the position of the maximum insulator.

[0053] There is a coordinate-temperature coordinate system in the image processing interface. For the collected infrared image, the temperature in the image is obtained through the SDK provided by the unmanned aerial vehicle manufacturer. Through the SDK, the temperature value of any pixel point on the image can be obtained. In the image processing interface, draw a straight line on the infrared image, and a curve can be obtained in the coordinate-temperature coordinate system. When the angle between the straight line and the horizontal line is 45°-135°, the x-axis of the coordinate-temperature coordinate system is the y-coordinate value of the point on the straight line. When the angle between the straight line and the horizontal line is 0°-45°, 135°-180°, the x-axis of the coordinate-temperature coordinate system is the y-coordinate value of the point. The y-axis of the coordinate system is the temperature value. The coordinate direction of the points on the curve is from left to right and from top to bottom. At the same time, mark the highest temperature point coordinate and the lowest temperature point coordinate on the straight line in the infrared image. Combined with the recognition algorithm part of the present application, the temperature of all points on the power transmission line can be obtained.

[0054] The contents not described in detail in the specification belong to the prior art known to those skilled in the art.

Claims

1. A linear infrared image processing and recognition system for power transmission lines, characterized in that: It includes an RGB channel separation module, an ROI extraction module, an adaptive grayscale threshold processing module, an edge extraction module, and an insulator position acquisition module; The RGB channel separation module is used to separate the RGB channels of the infrared image of the high-voltage power line to obtain the corresponding G channel image. The ROI extraction module is used to extract the ROI region of the G channel image to obtain the ROI region image; The adaptive grayscale threshold processing module is used to process the ROI region image using the OTSU method, determine the adaptive grayscale threshold t, and use the adaptive grayscale threshold t to binarize the ROI region image to obtain the ROI region image after adaptive grayscale threshold processing. The edge extraction module is used to extract edges from the ROI region image after adaptive grayscale thresholding using the edge detection Canny operator to obtain the edge contour image of the ROI region. The insulator position acquisition module is used to subtract the pixel value of each corresponding pixel in the edge contour image of the ROI region after adaptive grayscale threshold processing from the pixel value of the ROI region after adaptive grayscale threshold processing to obtain the ROI region difference image. The maximum contour found by the ROI region difference image is the position of the maximum insulator. The specific method for the ROI extraction module to extract the ROI region of the G channel image is as follows: starting from the leftmost or rightmost side of the G channel image, calculate the sum of pixel values ​​on the same vertical line of the G channel image in sequence. When the difference between the sum of pixel values ​​on a certain vertical line and the sum of pixel values ​​on adjacent vertical lines is greater than a threshold, it is a sudden change point in the sum of pixel values ​​on the vertical line. Record the horizontal coordinate x of the sudden change point. Let the length of the G channel image be m, and extract an image of length (mx) as the ROI region image. The adaptive grayscale thresholding module processes the ROI region image using the OTSU method. The specific method for determining the adaptive grayscale threshold t is as follows: assuming the adaptive grayscale threshold t... n Calculate the adaptive grayscale threshold t n Each adaptive grayscale threshold t i The inter-class variance of two images after ROI region segmentation is calculated at each adaptive grayscale threshold t. i The adaptive gray threshold t is the adaptive gray threshold that maximizes the inter-class variance between the two images after segmentation of the ROI region. It also includes a conductor position acquisition module, which is used to determine the conductor position based on the position of the insulator according to the region growth method. The region growth method finds a seed pixel in the region as the starting point for growth, and then merges the pixels with pixel values ​​of 250 to 255 in the neighborhood of the seed pixel into the region where the seed pixel is located. These new pixels are used as new seed pixels to continue the above process until no more pixels that meet the conditions can be included.

2. The linear infrared image processing and recognition system for power transmission lines according to claim 1, characterized in that: The temperature value acquisition module is used to acquire the temperature value of any pixel in the infrared image of the high-voltage power line, and combine it with the position of the maximum insulator and the position of the conductor to acquire the temperature values ​​of all points at the positions of the maximum insulator and the conductor.

3. The linear infrared image processing and recognition system for power transmission lines according to claim 1, characterized in that: The RGB channel separation module is used to separate the image channels of the infrared image of the high-voltage power line using the split() function built into OpenCV.

4. The linear infrared image processing and recognition system for power transmission lines according to claim 1, characterized in that: The insulator location acquisition module is used to subtract the pixel value of each pixel in the edge contour image of the ROI region after adaptive grayscale thresholding from the pixel value of the edge contour image of the ROI region to obtain the ROI region difference image. The maximum contour found by the findcontours() function in OpenCV software is the location of the insulator.

5. The linear infrared image processing and recognition system for power transmission lines according to claim 1, characterized in that: The adaptive grayscale threshold processing module uses an adaptive grayscale threshold t to binarize the ROI region image. Specifically, the pixels in the ROI region image are transformed into white pixels with a pixel value of 255 and black pixels with a pixel value of 0 using the adaptive grayscale threshold t.

6. The linear infrared image processing and recognition system for power transmission lines according to claim 1, characterized in that: The edge extraction module uses the Canny edge detection operator to extract edges from the ROI region image after adaptive grayscale thresholding. The specific method for obtaining the edge contour image of the ROI region is as follows: The edge contour image of the ROI region refers to the part of the image where the brightness change in a local area is greater than a preset value. The grayscale profile of this region is regarded as a step. The ROI region image is smoothed by a two-dimensional Gaussian function and Gaussian filtering is performed to remove noise. Where G(x,y) represents the gradient of the pixel value represented by point (x,y) in the ROI region image, x represents the horizontal coordinate, y represents the vertical coordinate, and σ represents the standard deviation. Gaussian filtering denoising is to perform a weighted average of the pixel values ​​of the entire ROI region image. For each pixel, the value is obtained by weighting its own value and the values ​​of other pixels in its neighborhood. Let the gradient values ​​of the filtered image in the horizontal and vertical directions be G and G, respectively. x (i, j) and G y (i, j), then the gradient value and gradient direction of each pixel are: In the gradient direction, if the gradient magnitudes of the two adjacent pixels of the current pixel are both greater than the gradient magnitude of the current pixel, then the current pixel is marked as a non-edge point; otherwise, the current pixel is marked as an edge candidate point. Traverse the ROI region image. If the gradient magnitude of a pixel is greater than the high threshold of the Canny operator, it is determined to be an edge point; if the gradient magnitude of a pixel is less than the low threshold of the Canny operator, it is determined to be a non-edge point; if the gradient magnitude of a pixel is between the high and low thresholds of the Canny operator, check if there are any points in its 8-neighborhood that are greater than the high threshold of the Canny operator. If there are, it is an edge point; otherwise, it is a non-edge point.

7. The linear infrared image processing and recognition system for power transmission lines according to claim 2, characterized in that: The specific method by which the conductor position acquisition module determines the conductor position based on the insulator position using a region growth method is as follows: Take any point in the determined maximum insulator position in the G channel image as one end of the conductor and define it as a seed. The pixel value of the seed is a. Add the pixels with pixel values ​​from a-5 to a in the 8-neighborhood of the seed to the seed, so that the number and range of seeds continue to increase until the boundary of the image is found, that is, the position of the conductor is obtained.

8. A method for processing and recognizing linear infrared images of power transmission lines according to claim 1, characterized in that, It includes the following steps: Step 1: Separate the RGB channels of the infrared image of the high-voltage power line to obtain the corresponding G channel image; Step 2: Extract the ROI region of the G channel image to obtain the ROI region image; Step 3: Process the ROI region image using the OTSU method, determine the adaptive grayscale threshold t, and use the adaptive grayscale threshold t to binarize the ROI region image to obtain the ROI region image after adaptive grayscale threshold processing. Step 4: Use the Canny edge detection operator to extract edges from the ROI region image after adaptive grayscale thresholding to obtain the edge contour image of the ROI region; Step 5: Subtract the pixel value of each pixel in the edge contour image of the ROI region after adaptive grayscale thresholding from the pixel value of the ROI region after adaptive grayscale thresholding to obtain the ROI region difference image. The maximum contour found in the ROI region difference image is the position of the maximum insulator.

Images