A power inspection method and system based on drone

Abstract

The present invention relates to the field of image processing, and in particular to a method and system for electric power inspection based on a drone, the method comprising: obtaining an original image of electric power equipment during inspection, preprocessing the original image to obtain a grayscale image; calculating the possibility that each pixel in the grayscale image is located on the electric power equipment; adjusting the pixel value of each pixel in the original image based on the possibility that the pixel is located on the electric power equipment to obtain the latest image; compressing and decompressing the latest image to obtain a decompressed image, and performing defect detection on the decompressed image. The present invention adjusts the pixel values ​​of ineffective pixels, and in the adjusted image, the number of pixels with the same pixel value increases, thereby improving the compression rate of the image, so that the working efficiency of the drone during the electric power inspection process is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to a power inspection method and system based on a drone. Background Art

[0002] Drone power inspection is a new technology application that uses drone technology to inspect power facilities to ensure the safe and efficient operation of the facilities. Drones are equipped with multiple sensors such as high-definition cameras and infrared thermal imagers. They fly along power lines in the air and obtain high-definition images and temperature data of the lines in real time, so as to promptly discover safety hazards in the lines. For example, drones can detect problems such as overheating of wires, corrosion of insulators, and aging of lines.

[0003] Prior art, such as the patent application document with publication number CN116612065A, discloses a method for intelligently identifying defects in power line inspection images based on YOLOv5. The method includes: inputting transmission line drone inspection images; preprocessing the aerial images using an image denoising algorithm; establishing a data set using labeled transmission line drone inspection images; training a YOLOv5 model using the data set to automatically mine defect features; detecting drone inspection images using the trained YOLOv5 model to identify equipment defect types and locations; analyzing the recognition results and generating a report.

[0004] The above patent application document uses drone inspection to collect visible light images of key equipment in the power transmission lines and uses deep learning algorithms to extract features. However, when the drone acquires images, there are some external factors, such as power equipment in remote areas and poor network information, which leads to slow data transmission speed of the drone and reduced work efficiency. Summary of the invention

[0005] In order to solve the technical problem of reduced working efficiency of the above-mentioned drone, the present invention provides solutions in the following aspects.

[0006] In a first aspect, a power inspection method based on a drone comprises:

[0007] The original image of the power equipment during inspection is obtained, and the original image is preprocessed to obtain a grayscale image; the possibility of each pixel in the grayscale image being on the power equipment is calculated; the pixel value of each pixel in the original image is adjusted based on the possibility of the pixel being on the power equipment to obtain the latest image; the latest image is compressed and decompressed to obtain a decompressed image, and the decompressed image is subjected to defect detection;

[0008] The adjustment process includes:

[0009] When the probability of any pixel point is greater than or equal to a preset threshold, the pixel value of the corresponding pixel point of the original image is not adjusted;

[0010] When the possibility of any pixel point is less than a preset threshold, the pixels in the row where the pixel point is located and whose possibility of being on the power equipment is greater than or equal to the preset threshold are eliminated, and the mode of the pixel values ​​of the remaining pixels in the row where the pixel point is located on any channel of the original image is replaced by the pixel value of the pixel point.

[0011] The present invention obtains a grayscale image by preprocessing the original image, and further adjusts the pixel value based on the possibility that the pixel point is on the power equipment, so that the power equipment part in the image is more prominent, and the non-power equipment part (such as the background) is properly suppressed or adjusted, thereby improving the overall quality of the image.

[0012] During the adjustment process, for pixels with a high probability of being on the power equipment (i.e., pixels with a probability greater than or equal to a preset threshold), their pixel values ​​remain unchanged, retaining the original features of the power equipment. For pixels with a low probability, by replacing them with the majority pixel values ​​of pixels with a high probability in their row, noise and interference can be reduced to a certain extent, while maintaining the continuity of the edge of the power equipment, making the power equipment clearer and easier to identify in the image.

[0013] Furthermore, by compressing and decompressing the images, the amount of data stored and transmitted can be reduced without losing too much image information, thereby improving the working efficiency of the drone.

[0014] Preferably, the probability that the pixel point is on the power device satisfies the relationship:

[0015] ; In the formula, For the The probability that a pixel is on a power device, For the The maximum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, is the grayscale image with The number of pixels with the same feature value. It is the maximum number of pixels with the same eigenvalue in the grayscale image.

[0016] By considering the maximum value of the probability that a pixel in the neighborhood around the pixel is located at the edge of the power equipment, it helps to highlight the features of the edge of the power equipment, and thus the outline and shape of the power equipment can be more clearly identified; when a pixel has the same feature value as multiple surrounding pixels, it is more likely to be part of the power equipment.

[0017] Preferably, the process of acquiring the probability that the pixel point is located at the edge of the power equipment includes:

[0018] Perform edge detection on the grayscale image to obtain all edges;

[0019] The average value of the curvature of all the pixels on the edge is calculated, and the average value of the curvature is normalized to obtain the probability that the pixel is located at the edge of the power equipment.

[0020] Through edge detection and curvature calculation, the noise and non-edge information in the image can be filtered, unnecessary calculations in subsequent processing steps can be reduced, and the entire image processing process can be optimized.

[0021] Preferably, the compression uses run length encoding.

[0022] Preferably, the compression adopts JPEG lossy compression.

[0023] Preferably, the probability that the pixel point is on the power device satisfies the relationship:

[0024] ; For the The probability that a pixel is on a power device, For the The probability that a pixel is located at the edge of the power equipment, For the The maximum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, For the The minimum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, is the grayscale image with The number of pixels with the same feature value. It is the maximum number of pixels with the same eigenvalue in the grayscale image.

[0025] By considering the maximum and minimum values ​​of probability information, the edge features in the local area can be captured; by introducing and , which can capture the global edge features.

[0026] Preferably, edge detection is performed on the grayscale image to obtain the bounding rectangles of all edges, and the aspect ratio of each bounding rectangle is calculated. Then, the probability that the pixel point is located at the edge of the power equipment satisfies the relationship:

[0027] ; In the formula, For the The probability that a pixel on the edge of a strip is located at the edge of a power device, For the The average curvature of all pixels on the edge of the strip, For the The difference between the aspect ratio of the circumscribed rectangle corresponding to the strip edge and all other edges is less than or equal to the number of edges of the recognition accuracy of the insulator preset in the grayscale image. is the total number of edges, represents normalization processing, It is an exponential function with the natural constant e as its base.

[0028] The geometric shape characteristics (curvature) of the edge and the similarity with other edges (width-to-height ratio of the bounding rectangle) are comprehensively considered to calculate the probability that each edge pixel belongs to the edge of the power equipment. This comprehensive consideration helps to improve the accuracy and robustness of edge detection of power equipment.

[0029] In a second aspect, a drone-based power inspection system includes: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned drone-based power inspection method is implemented.

[0030] The beneficial effects of the present invention are:

[0031] By calculating the probability that a pixel is at the edge of or on the power equipment, it is possible to determine which pixels in the image are useful for power inspection, and then adjust the pixel values ​​of ineffective pixels.

[0032] In the adjusted image, the number of pixels with the same pixel value increases, thereby improving the compression rate of the image and improving the work efficiency of the drone during power inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0034] Figure 1 It is a method flow chart of steps S1 to S4 in a power inspection method based on a drone in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0036] The embodiment of the present invention discloses a power inspection method based on a drone, referring to Figure 1 , including steps S1 to S4, which are specifically as follows:

[0037] S1: Obtain the original image of the power equipment during inspection, and pre-process the original image to obtain a grayscale image.

[0038] When using drones for power inspections, the high-definition cameras on board are used to capture the original images of the power equipment (i.e., three-channel color images), and the captured original images are subjected to denoising and image equalization to obtain grayscale images.

[0039] S2: Calculate the probability that each pixel in the grayscale image is located on the power equipment.

[0040] When using drones for power inspections, the original images collected will contain a large amount of image information that is irrelevant to the power inspection. This is because during the inspection process, drones will not only capture power equipment, but also other elements such as the natural environment, buildings, and vegetation around the power equipment.

[0041] Since power equipment is usually made of specific materials such as metal, it has significant differences in texture features from the surrounding environment and other objects. Therefore, the difference in texture features between power equipment and the surrounding environment can be analyzed to improve the accuracy of image power equipment recognition. The specific analysis steps are generally as follows:

[0042] First, edge detection (such as Canny edge detection algorithm, Sobel operator) is performed on the grayscale image obtained in the above step S1 to identify all edges in the grayscale image;

[0043] After edge detection, some wider or fuzzy edges may be obtained. Therefore, edge thinning algorithms (such as non-maximum suppression, morphological processing, etc.) can be used to thin the edges and turn them into lines with a width of one pixel.

[0044] Track the refined edge and extract the pixels on the edge. For each pixel on the edge, calculate its curvature, add up the curvatures of all edge pixels, and then divide it by the total number of pixels on the edge to get the average curvature.

[0045] The curvature can be calculated by fitting a local curve of edge pixels (such as using polynomial fitting, circle fitting, etc.), or by using other methods. The curvature calculation is a prior art and will not be described in detail here.

[0046] Since the average value of the curvature may vary due to different image sizes and resolutions, the average value of the curvature needs to be normalized, and the normalized average value of the curvature is used as the probability that each pixel is located at the edge of the power equipment.

[0047] Exemplarily, the probability that a pixel point is located at the edge of a power device satisfies the following relationship:

[0048]

[0049] In the formula, For the The probability that a pixel on the edge of a strip is located at the edge of a power device, For the The average curvature of all pixels on the edge of the strip, represents normalization processing, It is an exponential function with the natural constant e as its base.

[0050] in, The bigger, The smaller it is, the smaller the probability that the pixel on the edge is on the edge of the power equipment. The smaller it is, the more straight-line the edge is, that is, the greater the probability of being on the edge of the power equipment.

[0051] In another embodiment, it is also taken into account that the insulator of the power equipment may have a large edge curvature due to its umbrella-like structure. Using the first method of calculating the edge curvature mentioned above alone will cause the pixels on its edge to be underestimated. Therefore, by introducing a correction factor, this can be compensated so that the pixels on the edge of the insulator can also be correctly identified as part of the edge of the power equipment.

[0052] Exemplarily, the probability that a pixel point is located at the edge of a power device satisfies the following relationship:

[0053]

[0054] In the formula, For the The probability that a pixel on the edge of a strip is located at the edge of a power device, For the The average curvature of all pixels on the edge of the strip, For the The difference between the aspect ratio of the circumscribed rectangle corresponding to the strip edge and all other edges is less than or equal to the number of edges of the recognition accuracy of the insulator preset in the grayscale image. is the total number of edges, represents normalization processing, It is an exponential function with the natural constant e as its base.

[0055] Among them, the recognition accuracy of the insulator can be set to 0.1, and the staff can also adjust it according to the specific situation.

[0056] in, The larger it is, the more edges have a similar aspect ratio to this edge, further confirming the possibility that this edge is an insulator.

[0057] Secondly, in order to further improve the distinction between power equipment and background in the acquired image, a local binary pattern (LBP) algorithm with a window such as 3×3 is used to obtain the feature value of each pixel in the grayscale image. Each feature value corresponds to multiple pixels in the image.

[0058] Count the number of pixels corresponding to each eigenvalue, find the maximum number of pixels with the same eigenvalue, and for each pixel, obtain the maximum value of the probability that the pixels in the surrounding 11×11 range are located at the edge of the power equipment;

[0059] Then the probability of the pixel being on the power equipment is calculated to satisfy the following relationship:

[0060]

[0061] In the formula, For the The probability that a pixel is on a power device, For the The maximum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, is the grayscale image with The number of pixels with the same feature value. It is the maximum number of pixels with the same eigenvalue in the grayscale image.

[0062] in, The larger the value is, the more likely the pixel is to be close to the edge of the power device, so the greater the possibility that it is on the power device; since one of the pixels on both sides of the edge of the power device is on the power device and the other is in the background, May lead to misidentification, therefore, The smaller it is (i.e. the fewer pixels have the same characteristic value as the current pixel), the more likely the current pixel is unique, and therefore the more likely it is to be on a power device. Corrections are made to reduce the possibility of misidentifying background pixels as power equipment pixels.

[0063] In another embodiment, the probability that the pixel point is located on the power device satisfies the relationship:

[0064]

[0065] For the The probability that a pixel is on a power device, For the The probability that a pixel is located at the edge of the power equipment, For the The maximum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, For the The minimum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, is the grayscale image with The number of pixels with the same feature value. It is the maximum number of pixels with the same eigenvalue in the grayscale image.

[0066] Among them, not only the maximum and minimum probability values ​​in the neighborhood are considered, but also the probability information of the pixel itself is considered to further enhance the robustness of the calculation.

[0067] S3: Adjust the pixel value of each pixel of the original image based on the possibility that the pixel is on the power equipment to obtain the latest image.

[0068] Based on the possibility that the pixel is located on the power equipment, the pixel value of each pixel in the original image is adjusted to obtain the latest image. Through this adjustment, the areas related to the power equipment can be highlighted, making the images of these areas clearer and easier to identify. This helps to more accurately extract the characteristic information of the power equipment in the subsequent image processing and analysis process, thereby improving the accuracy and efficiency of power equipment fault detection and diagnosis.

[0069] Moreover, during drone power inspections, the efficiency of data transmission after the drone takes images is reduced due to external factors, and faults in power equipment cannot be detected in a timely manner.

[0070] It should be noted that in the embodiment of the present invention, the original image is a three-channel color image, so each pixel is composed of three color channels, namely a red channel, a green channel and a blue channel. When performing image processing, the pixel value of each channel may need to be adjusted separately, and the adjusted pixel value will also have three channels.

[0071] Specifically, a pixel value adjustment rule is formulated based on the possibility that each pixel point is located on the power device. The rule is as follows:

[0072] When the probability of any pixel point is greater than or equal to a preset threshold, the pixel value of the corresponding pixel point of the original image is not adjusted;

[0073] For example, if ≥0.8, then it is considered that this pixel is likely to be part of the power equipment, so its original pixel value is retained unchanged;

[0074] When the probability of any pixel point is less than a preset threshold, the pixels in the row where the pixel point is located and whose probability of being on the power equipment is greater than or equal to the preset threshold are removed, and the mode of the pixel values ​​of the remaining pixels in the row where the pixel point is located in any channel of the original image is replaced by the pixel value of the pixel point;

[0075] For example, if <0.8, then it is considered that this pixel is likely not part of the power equipment, so its pixel value is adjusted to the mode of the pixel values ​​of non-power equipment pixels in the row (that is, the mode of all the pixel values ​​of the pixel in the row where the pixel is located). Pixels with a value ≥ 0.8 are removed, and the mode of the pixel values ​​of the remaining pixels in the row on the corresponding channel is used as the new pixel value of the pixel). The purpose of this operation is to increase the number of pixels with the same consecutive pixel value for easy compression.

[0076] In another embodiment, the adjustment rules are supplemented.

[0077] When the probability of any pixel point is less than a preset first threshold (corresponding to the preset threshold 0.8 in the above initial adjustment rule) and greater than or equal to a preset second threshold (the second threshold can be set to 0.5), it is divided into two cases:

[0078] When the maximum value of the probability that all pixels in the 11×11 range around a pixel are on the power device is greater than or equal to 0.8, it is considered that this pixel may also be part of the power device, so its original pixel value is retained;

[0079] When the maximum value of the probability that all pixels within the 11×11 range around a pixel are on the power device is less than 0.8, it is considered that this pixel may not be part of the power device, so its pixel value is adjusted to the mode of the pixel values ​​of non-power device pixels in the row.

[0080] The adjustment rule is formulated to maintain the quality of the power inspection image while increasing the image compression efficiency by adjusting the pixel value.

[0081] Based on the adjustment rule, all adjusted pixel values ​​are obtained, all adjusted pixel values ​​are recombined to construct a new image, and then an latest image is obtained. This latest image can be used for further processing, such as compression, storage, transmission or analysis.

[0082] S4: compress and decompress the latest image to obtain a decompressed image, and perform defect detection on the decompressed image.

[0083] After obtaining the latest image, use the compression algorithm to compress the latest image, store or transmit the compressed image file to the place where the image analysis is performed, further apply the decompression algorithm to the compressed image file to restore the image, apply the defect detection algorithm to the decompressed image, identify and mark the defects in the image, and then complete the UAV's power inspection.

[0084] The compression algorithm may use existing technologies such as run length encoding and JPEG lossy compression.

[0085] An embodiment of the present invention further discloses a drone-based power inspection system, comprising a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a drone-based power inspection method according to the present invention is implemented.

[0086] The system also includes other components familiar to those skilled in the art, such as a communication bus and a communication interface. The configuration and functions of these components are known in the art and will not be described in detail here.

[0087] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module or both. Any such computer storage medium may be part of a device or accessible or connectable to a device.

[0088] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0089] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A power inspection method based on drone, characterized in that: include: Obtain the original image of the power equipment during inspection, and pre-process the original image to obtain a grayscale image; Calculate the probability that each pixel in the grayscale image is located on the power equipment, satisfying the relationship: ; In the formula, For the The probability that a pixel is on a power device, For the The maximum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, is the grayscale image with The number of pixels with the same feature value. is the maximum number of pixels with the same eigenvalue in the grayscale image; The pixel value of each pixel of the original image is adjusted based on the possibility that the pixel is located on the power equipment to obtain the latest image; Compress and decompress the latest image to obtain a decompressed image, and perform defect detection on the decompressed image; The adjustment process includes: When the probability of any pixel point is greater than or equal to a preset threshold, the pixel value of the corresponding pixel point of the original image is not adjusted; When the possibility of any pixel point is less than a preset threshold, the pixels in the row where the pixel point is located and whose possibility of being on the power equipment is greater than or equal to the preset threshold are eliminated, and the mode of the pixel values ​​of the remaining pixels in the row where the pixel point is located on any channel of the original image is replaced by the pixel value of the pixel point.

2. The power inspection method based on drone according to claim 1 is characterized in that: The process of obtaining the probability that the pixel point is located at the edge of the power equipment includes: Perform edge detection on the grayscale image to obtain all edges; The average value of the curvature of all the pixels on the edge is calculated, and the average value of the curvature is normalized to obtain the probability that the pixel is located at the edge of the power equipment.

3. The power inspection method based on drone according to claim 1 is characterized in that: The compression uses run length encoding.

4. The power inspection method based on drone according to claim 1 is characterized in that: The compression adopts JPEG lossy compression.

5. The power inspection method based on drone according to claim 1 is characterized in that: The probability that the pixel point is located on the power device satisfies the relationship: ; For the The probability that a pixel is on a power device, For the The probability that a pixel is located at the edge of the power equipment, For the The maximum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, For the The minimum value of the probability that the pixel point in the neighborhood around the pixel point is located at the edge of the power equipment, is the grayscale image with The number of pixels with the same feature value. It is the maximum number of pixels with the same eigenvalue in the grayscale image.

6. The power inspection method based on drone according to claim 5 is characterized in that: Perform edge detection on the grayscale image, obtain the bounding rectangles of all edges, and calculate the aspect ratio of each bounding rectangle. The probability that the pixel point is located at the edge of the power equipment satisfies the relationship: ; In the formula, For the The probability that a pixel on the edge of a strip is located at the edge of a power device, For the The average curvature of all pixels on the edge of the strip, For the The difference between the aspect ratio of the circumscribed rectangle corresponding to the strip edge and all other edges is less than or equal to the number of edges of the recognition accuracy of the insulator preset in the grayscale image. is the total number of edges, represents normalization processing, It is an exponential function with the natural constant e as its base.

7. A power inspection system based on drones, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the drone-based power inspection method according to any one of claims 1 to 6 is implemented.

Images