A power tower and power line association correction method

Abstract

A method for power tower and power line detection with correlation correction involves capturing images of the power tower and power lines from above and preprocessing the images to grayscale. Edge detection is then performed on the preprocessed images to obtain a binarized edge map. This edge map is used for power line and power tower detection, resulting in power lines represented by parallel line groups and power tower regions defined by rectangular boxes. The rectangular boxes are mapped onto the edge map, excluding background areas on either side of the power tower rectangles, and power lines are re-detected within the power tower rectangles. After obtaining new power line detection results, the proportion of the power tower region in the edge map is estimated based on the left and right widths of the power line distribution area, and the power tower detection parameters are updated accordingly. The process of re-detecting power towers continues until the requirements are met or the required number of iterations is reached. This invention does not rely on large-scale datasets, resulting in faster detection speeds and better performance.

Description

Technical Field

[0001] This invention belongs to the field of power facility safety inspection technology, and relates to the inspection of power towers and power lines, and particularly to a power tower and power line detection method for correlation correction of power towers and power lines. Background Technology

[0002] Traditional power line inspections primarily employ manual and helicopter inspections. Manual inspection involves workers climbing power towers to inspect the condition of the power lines. However, manual inspections have several drawbacks. Firstly, they are inaccurate and inefficient, especially for long-distance transmission lines, requiring significant manpower and resources. Secondly, they pose low safety risks, particularly for lines in complex environments, where adverse weather conditions make manual inspections extremely difficult and dangerous. Helicopter inspection is another commonly used method, utilizing helicopters equipped with professional personnel and various sensors to conduct inspections over the power lines. Helicopter inspections have addressed some of the problems of manual inspections. For example, helicopters can reach complex terrains inaccessible to humans, such as canyons, and significantly improve inspection efficiency. However, helicopters are not flexible enough, their large size limits their access to many specific areas, and their inspection and maintenance costs are high, limiting their use to specific situations. In conclusion, current traditional inspection methods are insufficient to meet the ever-growing demand for power line inspections.

[0003] In recent years, with the development of artificial intelligence and robotics technologies, a technical solution has emerged that relies on drones equipped with simple cameras to collect data and uses AI technology for image understanding to inspect power lines. Drone inspection is less affected by weather and can operate for extended periods, significantly increasing inspection frequency and quality while reducing labor costs and ensuring worker safety, thus demonstrating clear advantages in power line inspection. However, the key to drone-camera inspection lies in how to utilize computer vision recognition technology for the detection and identification of power lines and towers.

[0004] Currently, the main methods for detecting electric lines in images, both domestically and internationally, can be broadly categorized into two types: traditional image processing methods and data-driven machine learning methods. Image processing methods rely on the physical and geometric characteristics of the electric lines themselves, defining feature patterns based on human experience to perform pattern recognition on the image to be detected. For example, based on prior knowledge, the electric line is treated as a continuous straight line, and classic line segment detection methods are used to complete the detection, such as Hough transform, Radon transform, directional filtering, and line detection algorithms based on gradient and edge information. Traditional image processing methods are easily affected by noise, and their human experience is often specific to particular scenes. For different environments, frequent parameter adjustments or even redefinition of feature patterns are needed to achieve good detection results. Data-driven machine learning methods, on the other hand, learn and train machine learning models on large amounts of labeled data to obtain detection models that effectively describe the features of electric lines. Among machine learning methods, those based on deep learning models (usually convolutional neural networks) have shown better performance. Data-driven machine learning methods typically require a large amount of manually labeled datasets to train the model and post-processing optimization using electric line structure information.

[0005] Meanwhile, the main technical approach for inspecting power towers currently employs object detection methods from the field of computer vision. Moreover, the most effective methods are those based on deep learning models, such as two-stage detection methods based on Faster R-CNN and Mask R-CNN, and one-stage object detection methods based on YOLO. Two-stage detection algorithms offer high accuracy but are slow, making them unsuitable for real-time UAV inspections. One-stage object detection algorithms based on regression, such as YOLO and SSD, have higher detection efficiency, but accuracy cannot be guaranteed, resulting in higher false positive and false negative rates.

[0006] Even so, current methods for detecting power lines and towers for applications such as power line inspection have limited overall accuracy and do not fully meet the requirements for high reliability and availability. This is due to insufficient labeled training data leading to inadequate training of machine learning models, as well as challenges arising from the complexity and diversity of power lines, towers, and their environments, resulting in poor versatility of traditional image processing methods. In particular, power line inspection is a domain-specific problem with limited available image data, and even less labeled image data. Therefore, while data-driven machine learning methods are theoretically more effective, there is a lack of sufficient data to train highly usable detection models. Furthermore, existing methods typically perform power line or tower detection separately, without fully utilizing prior knowledge in the power transmission field, such as the structural relationships between power lines and towers. Summary of the Invention

[0007] In order to overcome the shortcomings of the prior art, the present invention aims to provide a power tower and power line detection method for power tower and power line association correction, so as to at least solve at least one of the problems of insufficient training due to limited image data and poor detection accuracy of power towers and power lines in complex environments.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A method for detecting power towers and power lines to correct the correlation between power towers and power lines, comprising the following steps:

[0010] Step 1: Take an image of the power tower and power lines from above, and perform grayscale preprocessing on the image;

[0011] Step 2: Perform edge detection on the preprocessed image to obtain a binarized edge map;

[0012] Step 3: Power line detection is performed using the edge map, which includes line segment detection, merging line segments, and parallel line group clustering to obtain power lines represented by parallel line groups.

[0013] Step 4: Use the edge map to detect power towers and obtain the power tower area defined by a rectangular frame;

[0014] Step 5: Map the rectangle to the edge map, exclude the background areas on both sides of the power tower rectangle, and re-detect the power lines in the area inside the power tower rectangle. After obtaining new power line detection results, estimate the proportion of the power tower area in the edge map based on the left and right width of the power line distribution area, and then update the power tower detection parameters. Re-detect the power tower until the requirements are met or the number of iterations is reached.

[0015] In one embodiment, in step 1, each captured image contains only one power tower target and power lines; the preprocessing includes adjusting image resolution, image grayscale conversion, histogram equalization, and noise reduction; wherein, during the grayscale conversion process, only the red and blue channels are retained to generate a grayscale image.

[0016] In one embodiment, step 3 involves using a probabilistic Hough transform to detect the two endpoints of the line segments in the edge image; then calculating the coordinates of the line segment center point, the line segment slope, and the intercept; grouping the line segments according to their geometric distance; after grouping, using the least squares method to fit the line segments in each group to a straight line and extending them to traverse the image; finally, grouping the lines with similar slopes into parallel line groups; and finally, clustering the parallel line groups according to their slopes, retaining the line with the highest count in the clustering results, and filtering out messy lines with slopes that differ greatly from the electric field lines.

[0017] In one embodiment, in the electric field lines represented by the resulting set of parallel lines, the distance between the two outermost electric field lines is considered. Estimate the width of the power tower, and then estimate the area where the power tower is located based on its length-to-width ratio. The edge map is rotated as a whole according to the power line detection results. Make the power tower area in the diagram horizontal, where The angle between the clustered line group and the vertical direction.

[0018] In one embodiment, step 4 involves using a corner detection method based on curvature scale space to detect corners in the edge map, obtaining a corner distribution map; then, the corner distribution map is integrally projected in the horizontal and vertical directions using pixel statistical projection to obtain horizontal and vertical projection histograms; finally, each projection value in the horizontal direction is replaced with its left and right adjacent values ​​using median smoothing. The median of the projections is used to replace each vertical projection value with its adjacent values ​​above and below. The median of each projection is used to obtain a smoothed projection histogram. The proportion of the power tower in the edge map is estimated by the width and height of the power line region. Based on the proportion, the projection histograms in the horizontal and vertical directions are divided into multiple small regions. The drop between each small region and the adjacent region is calculated. The regions with the largest drop have the left and right boundaries and the top and bottom boundaries of the power tower, respectively.

[0019] In one embodiment, updating the power tower detection parameters means updating the number of divisions of the small area.

[0020] In one embodiment, in step 5, the degree of change in the results during the iterative detection of power towers uses IoU as an evaluation index, and the number of iterations is determined based on the IoU. The IoU is the result of dividing the overlapping portion of two regions by the sum of the two regions, used to measure the degree of overlap between two power tower detection results. When the IoU is greater than a set threshold... If the two detection results are consistent, the current result is output as the final result, and the task ends; if the IoU is less than or equal to the threshold... If the number of iterations exceeds the given maximum number of iterations, the task is forcibly terminated, and the detection results of the last round are output.

[0021] The present invention also provides a power tower and power line detection system for power tower and power line correlation correction, comprising:

[0022] The preprocessing module performs grayscale preprocessing on the images of power towers and power lines captured from above;

[0023] The edge detection module performs edge detection on the preprocessed image to obtain a binarized edge map.

[0024] The power line detection module uses the edge map to detect power lines, which includes line segment detection, merging line segments, and parallel line group clustering to obtain power lines represented by parallel line groups.

[0025] The power tower detection module uses the edge map to detect power towers and obtains a power tower area defined by a rectangular frame.

[0026] The correlation correction module maps the rectangle to the edge map, excludes the background areas on both sides of the power tower rectangle, and re-detects power lines in the area inside the power tower rectangle. After obtaining new power line detection results, the proportion of the power tower area in the edge map is estimated based on the left and right widths of the power line distribution area, and then the power tower detection parameters are updated. The power tower is re-detected until the requirements are met or the number of iterations is reached.

[0027] In one embodiment, the detection system is deployed on an embedded platform of a drone to provide the drone flight control program with the function of detecting and locating the positions of power towers and power lines; the images of the power towers and power lines are acquired by the drone.

[0028] Compared with existing methods for independently detecting power lines and power towers, this invention does not rely on large-scale datasets, has a faster detection speed, and performs correlation correction on the detection results of power lines and power towers, resulting in better detection performance. Attached Figure Description

[0029] Figure 1 This is a flowchart of the detection algorithm of the present invention.

[0030] Figure 2 This is a schematic diagram of power towers and power lines in the clustered image of this invention.

[0031] Figure 3 This is a schematic diagram of the power tower area obtained after correction according to the present invention.

[0032] Figure 4 This is a schematic diagram of the corner projection of the present invention.

[0033] Figure 5 This is a schematic diagram of the power lines that are excluded from re-inspection in the areas on both sides of the power tower area according to the present invention.

[0034] Figure 6 The detection area is obtained by the IOU (Intersection over Union) calculation method of this invention, where the left figure is the intersection and the right figure is the union. Detailed Implementation

[0035] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings and examples.

[0036] Traditional target detection methods face challenges such as limited data and consistently low accuracy in complex environments. To address these issues, this invention provides a method for detecting power towers and power lines. This method employs an image processing approach to effectively overcome the challenge of limited image data. Furthermore, by correlating the detection of power lines and towers, it better addresses the accuracy issues associated with detecting power towers and power lines in complex environments.

[0037] The objective of this invention for detecting power towers and power lines is to detect power towers and power lines in an image, where power towers are defined by rectangular frames and power lines are marked with straight lines. Following the main processing flow of this invention, refer to... Figure 1 The main steps include the following:

[0038] Step 1: Take images of the power towers and power lines, and perform necessary preprocessing on the images, including at least grayscale conversion. Generally, the images should be taken from above the power towers and power lines.

[0039] Step 2: Perform edge detection on the preprocessed image to obtain a binarized edge map.

[0040] Step 3 involves detecting power lines using the edge map, which includes segment detection, merging segment pairs, and clustering parallel line groups to obtain power lines represented by parallel line groups. Specifically, the ends of a segment are detected, fitted to a straight line, and then clustered according to the similarity of their slopes.

[0041] Step 4: Use the edge map to detect power towers and obtain the power tower area defined by the rectangular frame.

[0042] Step 5: Map the rectangle to the edge map, exclude the background areas on both sides of the power tower rectangle, and re-detect the power lines in the area inside the power tower rectangle. After obtaining new power line detection results, estimate the proportion of the power tower area in the edge map based on the left and right width of the power line distribution area, and then update the power tower detection parameters. Re-detect the power tower until the requirements are met or the number of iterations is reached.

[0043] The power line detection in step 3, the power tower detection in step 4, and the correlation correction in step 5 are performed iteratively, such as... Figure 1 As shown.

[0044] The method of this invention is mainly based on the above-mentioned approach. Firstly, this invention utilizes a more efficient traditional image processing method to achieve initial detection of power lines and power towers. Then, leveraging knowledge in the power field, such as the correlation between power lines and power towers, it achieves an iterative and organic fusion and mutual correction of line detection and tower detection. Ultimately, it simultaneously or at least overcomes one of the two problems mentioned above.

[0045] Accordingly, this invention provides a power tower and power line detection system for power tower and power line correlation correction, comprising: a preprocessing module, an edge detection module, a power line detection module, a power tower detection module, and a correlation correction module. Steps 1 to 5 described above are executed respectively. In the following description, the execution content of the preprocessing module is the same as in step 1, the execution content of the edge detection module is the same as in step 2, the execution content of the power line detection module is the same as in step 3, the execution content of the power tower detection module is the same as in step 4, and the execution content of the correlation correction module is the same as in step 5.

[0046] This invention can be specifically used for detecting power towers and power lines in images captured by drones during power line inspections. Specifically, the detection method or system is deployed on an embedded platform of the drone, providing the drone's flight control program with the function of detecting and locating the positions of power towers and power lines. The drone power line inspection scenario described in this invention is as follows: the drone, carrying a camera, flies above the power towers and power lines and takes pictures of them from a top-down perspective. Considering the significant distances between power towers in long-distance power transmission scenarios, this invention ensures that only one tower target is included in the drone's field of view; that is, each image captured should contain only one power tower target, and should also include the power lines. The following describes this invention in detail using examples of the execution of each module.

[0047] 1. Preprocessing module (execute step 1)

[0048] The preprocessing module's functions include adjusting image resolution, image grayscale conversion, histogram equalization, and noise reduction. The preprocessing module receives video data captured by the drone and breaks the video down into single frames for preprocessing. Typically, images captured by drone lenses are high-resolution color RGB images, which are slow to process directly. Therefore, they are first scaled down to a resolution suitable for processing.

[0049] In the process of image grayscale conversion, the three channels of the RGB image must first be separated. Drone aerial images typically use the ground as a background, mainly consisting of green vegetation, yellow soil, blue-green water, and power lines that appear silvery-white under normal lighting conditions. The silvery-white color composition of the power lines is RGB(192, 192, 192), containing more blue (B) primary color information compared to the background color. To eliminate the influence of the complex ground environment on power line extraction, this invention only retains the red and blue channels to generate the grayscale image.

[0050] Since power line detection is performed outdoors, changes in environment or lighting can cause overexposure or underexposure, resulting in an overall dark image. This can also lead to inconsistent contrast of the target line within the image, affecting subsequent image segmentation and power line extraction. Therefore, in digital image processing, a grayscale histogram is used to count the frequency of pixels at different gray levels. The horizontal axis of this histogram represents the 0-255 gray levels, and the vertical axis represents the number of pixels at each gray level in the image. The grayscale histogram can be used to observe the grayscale distribution of an image. Histogram equalization involves non-linearly stretching the image, redistributing pixel values ​​to achieve approximately equal numbers of pixel values ​​within a certain area. Because a more even distribution of grayscale values ​​in an image results in a greater amount of information contained within the image, the color levels obtained through histogram equalization are more distinct, and the linear features in the image are more easily discernible.

[0051] The image is then smoothed using methods such as Gaussian filtering. Gaussian filtering is a linear smoothing filter suitable for eliminating Gaussian noise and is widely used in image processing noise reduction. Simply put, Gaussian filtering is a process of weighted averaging across the entire image. For each pixel's value, a convolution is performed using a template, and then the weighted average value is used to replace the value at the template's center.

[0052] 2. Edge detection module (execute step 2)

[0053] Edge detection is a parallel boundary segmentation technique based on gray-level discontinuities, and it is the first step in all boundary segmentation methods. Edges are the boundaries between the target and the background, and edge extraction is a crucial step in distinguishing between them. Generally, edge detection methods utilize the differences in color, texture, and gray-level features between the background and the target. Edge detection calculations are typically performed using first or second derivatives, but in actual digital images, differentiation is approximated using difference operations. Points located on either side of an edge in an image experience abrupt changes in gray-level values, resulting in larger differential values. The differential value is maximized when the direction of the differential is perpendicular to the boundary. This characteristic allows us to obtain image edges. This module uses the Canny operator and includes the following steps:

[0054] 1) Gradient and orientation calculation: The gradient and orientation of each pixel are calculated using the Sobel operator.

[0055] 2) Non-maximum suppression: Eliminate stray responses caused by edge detection.

[0056] 3) Dual threshold: Detects both real and potential edges.

[0057] 4) Lag technique: Edge detection and boundary tracking are completed by suppressing weak edges.

[0058] Edge detection serves two purposes: the edges of power line images are important features of power lines, and edge detection is a necessary step in power line detection; in addition, after extracting the edges of the image, filling, extending and smoothing the edges can be performed to detect corner points based on the rate of change of pixels on the edges, and thus detect power towers.

[0059] 3. Power line detection module (execute step 3)

[0060] The input to this module is the binarized edge map obtained after edge detection. The main steps of power line detection include line segment detection, line segment merging, and parallel line group clustering.

[0061] 1) Line segment detection employs probabilistic Hough transform. Hough transform is a feature extraction technique in image processing that detects objects with specific shapes using a voting algorithm. This process calculates the local maxima of the accumulated results in a parameter space to obtain a set of objects conforming to that specific shape as the Hough transform result, a common method for detecting line segments in images. The standard Hough transform essentially maps the image to its parameter space, requiring the calculation of all M edge points, resulting in a large computational load and memory requirements. If only processing is needed in the input image... ( If there are ) edge points, then this The selection of edge points is probabilistic, which is the basis of the probabilistic Hough transform. In actual UAV-captured images, power lines typically do not appear as clear, continuous long straight lines, but rather as many short, broken segments. A key feature of the probabilistic Hough transform is its ability to detect the two endpoints of line segments in an image, accurately locating straight lines. Therefore, this invention uses the probabilistic Hough transform, combined with distance metrics, to group and merge lines based on traditional line detection algorithms. This means detecting the two endpoints of line segments in the edge image, thereby fitting broken line segments into a straight line and improving the completeness of power line detection.

[0062] 2) Merging line segments: After obtaining the coordinates of the endpoints of the line segments through probabilistic Hough transform, the coordinates of the center point, slope, and intercept of the line segments are calculated, and the line segments are grouped according to their geometric distance. After grouping, the line segments in each group are fitted to straight lines using the least squares method, and the fitted lines are extended to penetrate the image. Finally, lines with similar slopes are grouped into parallel line groups.

[0063] 3) Parallel line clustering: In images captured by drones, power lines typically exhibit the following characteristics: approximately straight lines, a certain range of tilt angles, approximately parallel lines, traversing the entire image, and perpendicularly passing through power tower areas. Therefore, clustering parallel line groups based on their slopes and retaining the lines with the highest counts in the clustering results can filter out messy lines with slopes significantly different from those of the power lines.

[0064] In the embodiments of this invention, the K-means++ algorithm is used for parallel line group clustering. K-means is a commonly used clustering algorithm based on Euclidean distance, which assumes that the closer two targets are, the greater their similarity. The K-means++ algorithm optimizes the selection of initial cluster centers. The clustering process is roughly as follows: i) First, randomly select K samples (K parallel line groups with different slopes) from the sample set (a set of parallel line groups, where each sample is a parallel line group) as cluster centers, and calculate the distance between all samples and these K "cluster centers"; ii) For each sample, assign it to the cluster containing the nearest "cluster center", and calculate the new "cluster center" for each new cluster; iii) Repeat the above process until the "cluster centers" no longer move.

[0065] like Figure 2 As shown, after clustering, the angle between the detected electric field lines and the vertical direction is calculated. Used for image correction, and the location and size of power tower areas can be estimated based on the distribution area of ​​power lines. Specifically, as follows... Figure 3 As shown, in the electric field lines represented by the obtained set of parallel lines, the distance between the two outermost electric field lines is... The width of the power tower can be estimated, and then the area where the power tower is located can be estimated based on the length-to-width ratio of the power tower. The reason for image correction before detecting power towers is that target detection tasks typically use horizontal bounding boxes to represent the approximate range of targets in an image. However, objects in drone images are usually oriented arbitrarily. Using horizontal bounding boxes to detect targets results in the detection boxes including many background areas and failing to reflect the size and aspect ratio of the target object. This not only increases the difficulty of the detection task but also leads to inaccurate representation of the target range.

[0066] Based on the structure of power towers and power lines, multiple power lines will perpendicularly pass through a power tower area, while power towers are not densely distributed in unmanned aerial images; only one power tower area is visible in the image. Therefore, this invention utilizes this feature to rotate the entire edge map according to the power line detection results. This can make the power tower area in the diagram horizontal, making subsequent power tower detection more accurate.

[0067] 4. Power tower detection module (execute step 4)

[0068] The power tower detection module takes as input the edge map obtained by the edge detection module. This module uses corner detection (such as Curvature Scale Space Corner Detection, CSS corner detection method) to obtain a corner distribution map, and then performs horizontal and vertical projections of the corners to apply threshold judgments to determine the top, bottom, left, and right boundaries. The basic principle is that the power tower has a complex, crisscrossing steel frame structure, and the density of edges and corners inside the tower is significantly higher than in the background area. Therefore, this characteristic can be used to distinguish between the tower area and the background area.

[0069] In fact, most existing methods directly use edge information to statistically analyze the density distribution of edge points for power tower detection. However, this method has some shortcomings. For example, the selection of high and low thresholds in the most commonly used Canny edge detection operator is not based on image characteristics, but rather on prior experience. If the high and low thresholds are set too high during the edge detection process, more image edge details will be lost, resulting in discontinuous detected edges. Conversely, if the high and low thresholds are set too low, false edges will appear.

[0070] Therefore, in this invention, for the internal structure of power towers, corner detection based on curvature scale space is further performed on the basis of edge detection, namely, Curvature Scale Space Corner Detection (CSS) algorithm. The algorithm steps are as follows: First, the image contour extracted by methods such as Canny edge detection operator is used to fill the gaps in the binarized edge contour; second, after filling, the curvature of each pixel on the contour is calculated at a large scale, and if it exceeds a threshold, it is determined as a candidate corner point; finally, each pixel in the candidate corner point set is tracked at a small scale to accurately locate the position of the corner point.

[0071] After obtaining the corner distribution map of the image, the corner distribution map is projected horizontally using pixel statistical projection to obtain a horizontal projection histogram. In this projection histogram, the projection height represents the edge density in the horizontal direction; locations with higher edge density have higher projection heights. Simultaneously, each projection value in the horizontal direction is replaced with its left and right adjacent pixels using median smoothing. The median of the projections is used to eliminate noise. This smoothing process mainly aims to remove isolated noise points in the image while preserving image details relatively well. This results in an image like... Figure 4As shown, a relatively smooth projection histogram is obtained. In this projection histogram, the height of the projected columns inside the power tower is much greater than that of the background area, and there are obvious peaks at the edges of the power tower. The width of the power line region can be used to estimate the proportion of the power tower in the edge map. Based on the proportion, the projection histogram is divided into multiple small regions, and the drop between each small region and its adjacent regions is calculated. The region with the largest drop contains the left and right boundaries of the power tower. Similarly, projecting in the vertical direction yields a vertical projection histogram, which determines the upper and lower boundaries of the power tower. That is, the corner distribution map is integrally projected in the vertical direction using pixel statistical projection to obtain a vertical projection histogram; then, median smoothing is used to replace each projection value in the vertical direction with the values ​​of the adjacent upper and lower nodes. The median of the projections is used to obtain a smoothed projection histogram. The proportion of power towers in the edge map is estimated based on the height of the power line region. According to the proportion, the vertical projection histogram is divided into multiple small regions. The drop between each small region and the adjacent region is calculated. The region with the largest drop has the upper and lower boundaries of the power tower.

[0072] 5. Related Correction Module (Execute Step 5)

[0073] After the aforementioned module completes the initial detection of power lines and power towers, it can obtain the power lines represented by parallel line groups and the power tower areas defined by rectangular frames. Next, the correlation correction module iteratively updates the detection results of power lines and power towers based on the spatial structural relationship between power towers and power lines (i.e., iteratively executes steps 3, 4, and 5) to correct the detection results of power lines and power towers.

[0074] Specifically, firstly, the rectangular bounding box representing the power tower detection result is mapped to the edge map, excluding the background areas on both sides of the power tower's rectangular bounding box area. Power lines are then re-detected within the rectangular bounding box area using the same method as the initial detection. Secondly, after obtaining the new power line detection results, the proportion of the power tower area in the image is estimated based on the left and right widths of the power line distribution area. This information is then used to update the parameters of the power tower detection module (i.e., the number of small region divisions in the power tower detection module), and the histogram projection method is used again to detect the power tower.

[0075] The degree of change in results during the iterative detection of power towers is evaluated using IoU (Intersection over Union), and the number of iterations is determined based on IoU. IoU is calculated by dividing the overlapping portion of two regions by the sum of the values ​​of the two regions, and it measures the degree of overlap between two power tower detection results. When the IoU exceeds a set threshold... If the two detection results are consistent, the current result is output as the final result, and the task ends; if the IoU is less than or equal to the threshold... If the number of iterations exceeds the given maximum number of iterations, the task is forcibly terminated, and the detection results of the last round are output.

[0076] The following describes one workflow of this detection algorithm. This process involves a small drone transmitting real-time footage, and the algorithm automatically detecting power towers and power lines.

[0077] First, the drone was started, its camera and lidar were activated, and it hovered over the vicinity of the power tower, capturing images at a resolution of [resolution missing]. The image is pixel-level. After receiving the image transmitted from the drone, the preprocessing module scales the image to the specified size according to the ratio. The resolution is adjusted to facilitate subsequent processing. After scaling, the RGB channels of the image are separated, and the red and blue channels are extracted to generate a grayscale image. Gaussian filtering is then applied to the grayscale image for noise reduction. The formula for the two-dimensional Gaussian function is as follows:

[0078] Two-dimensional Gaussian distribution:

[0079]

[0080] In the formula, The parameters are Gaussian distribution parameters, which can be calculated from the filter kernel size. A 3*3 template is used to filter the image.

[0081] The preprocessed grayscale image is input into the edge detection module, where the Canny algorithm is used for edge detection, identifying sets of pixels with drastic brightness changes. The Canny algorithm uses the Sobel operator.

[0082] Formula for calculating convolution kernel:

[0083]

[0084] A represents the original image.

[0085] The horizontal and vertical grayscale values ​​of each pixel in the image are combined using the following formula to calculate the grayscale value of that point:

[0086]

[0087] After edge detection, the edge map is then used for power line detection and power tower detection. Power line detection uses probabilistic Hough transform to detect line segments. By calculating the slope and distance between line segments, duplicate line segments are deleted and merged, and parallel line segments are categorized.

[0088] The algorithm flow for line segment detection and merging is as follows:

[0089] 1) Obtain the coordinates of the endpoints of the line segment through probabilistic Hough transform.

[0090] 2) Calculate the coordinates of the center point of the line segment, the slope of the line segment, and the intercept, and mark the status as UNUSED.

[0091] 3) Select the line segment containing the point with the smallest ordinate among the endpoints as the initial line segment. From this point to other line segments The geometric distance of the center point is .

[0092] 4) Traverse the line segments in the set whose state is UNUSED, and set a distance threshold. ,like < Then store the line segment into the line segment group. Set the state of the line segment to USED. If all line segments have been traversed, repeat step 2 until all line segments are in the USED state.

[0093] 5) Obtain line segment groups The coordinates of the endpoints of each line segment are used to fit the line segments using the least squares method, and the fitted straight line is then extended to run through the entire image.

[0094] 6) Compare the slopes of the fitted lines sequentially; if the slope difference is less than the threshold... Straight lines are grouped into a group of parallel lines, and the average slope is used to represent the group of parallel lines.

[0095] After classification, parallel line groups are clustered based on slope to eliminate interfering straight lines, ultimately detecting the electric field lines. The K-means++ algorithm was used for clustering, and the steps are as follows:

[0096] 1) Select K groups of parallel lines as the initial cluster centers { }, that is, the initial cluster center, where Indicates the first The slope of a group of parallel lines.

[0097] 2) Calculate the sample points represented by each group of parallel lines. Find the Euclidean distance to the K cluster centers, locate the cluster center closest to the given point, and assign it to the corresponding cluster. The Euclidean distance is calculated as follows:

[0098]

[0099] 3) After all points are assigned to clusters, all M sample points (parallel line groups) are divided into K clusters. Then, the centroid (average distance from the center) of each cluster is recalculated and designated as the new "cluster center." The calculation of the "cluster center" is as follows:

[0100]

[0101] 4) Repeat steps 2-3 until the "cluster center" no longer changes.

[0102] Theoretically, in drone footage, the number of segments representing power lines is the largest, and these segments have almost identical slopes. Therefore, through the clustering process described above, the cluster with the largest number of segments should be the set of segments corresponding to power lines. Next, select the cluster with the most parallel segments from all the clusters mentioned above, and calculate the average slope α of all segments in this cluster. This slope is the tilt angle of the power line in the current image. Subsequently, rotate the image by an angle α based on the slope α of the power line, making the power line perpendicular to the horizontal direction. Meanwhile, since in a real scene, the power line should be approximately perpendicular to the outer rectangle of the power tower, after the above rotation operation, the power tower area should appear as follows: Figure 3 The image shows a view of the power tower with its left and right sides parallel to the vertical direction.

[0103] Simultaneously, the edge map of the original image is also rotated accordingly to obtain a rotated and corrected edge map, which is then used for CSS corner detection. The corner detection process is as follows:

[0104] 1) For the edge contour extracted by the Canny edge detection operator, when the edge point is an endpoint, if there are other endpoints in the neighborhood of the endpoint, the non-edge point between the two endpoints is replaced with an edge point; if there are other edge lines in the neighborhood of the endpoint, they are marked as T-shaped nodes.

[0105] 2) For each edge contour, the curvature of each pixel on the contour is calculated using a large-scale Gaussian filter function.

[0106] 3) If the curvature of a pixel on the edge line is greater than a pre-set threshold and is a locally unique maximum value, and its curvature value is also greater than twice the minimum curvature value in its neighborhood, then this pixel is marked as a candidate corner point. The curvature is calculated as follows:

[0107] Based on arc length coefficient To define the edge curve:

[0108]

[0109] The extracted edge curves are smoothed and denoised using a one-dimensional Gaussian filter function to obtain smooth curves:

[0110]

[0111] in , This represents the convolution operation. The standard deviation is expressed as The one-dimensional Gaussian filter function. The curvature function of the curve can be obtained from the edge curve function:

[0112]

[0113]

[0114]

[0115]

[0116]

[0117] in They are about The first derivative and the second partial derivative.

[0118] 4) Since the large-scale filter used in 2) blurs the curve highly, the resulting set of candidate corner points is only roughly selected. Therefore, a small-scale Gaussian filter is needed to track each pixel in the candidate corner point set, accurately locate the corner point, and improve the accuracy of corner point location.

[0119] 5) For the obtained T-shaped node and the detected candidate corner point, if they are adjacent, delete one of them.

[0120] After obtaining a binarized corner map through corner detection, the corner map is integrally projected in both the horizontal and vertical directions to obtain projection histograms in the horizontal and vertical directions. Taking the vertical direction as an example, the number of corners in each column of the image is accumulated during projection, and the height of the bars represents the number of corners, which can intuitively represent the distribution of corners in the vertical direction.

[0121] After obtaining the integral projection histogram, median filtering is used to smooth and denoise the projection histogram. For example, a 9-pixel sliding window is used to traverse the projection map, and the height of each column of projection bars is saved to form a sequence. The heights of the projection bars in this sequence are sorted from largest to smallest, and the height of the projection bar in the middle position, i.e., the median, is saved and assigned to the pixel in the middle position of the sliding window. This results in a relatively smooth corner projection map.

[0122] The width of the power line distribution can be obtained from power line detection. , This can be approximated as the width of the power tower, combined with the image size after adjustment. After denoising, the projection image is divided horizontally and vertically into... A small area. The calculation method is as follows:

[0123]

[0124] by Taking the case as an example, the number of corner points in each column (row) of the first small area. Accumulation, among which Use the total number of corner points within the region This indicates the corner density of the region. Number of corners The calculation is as follows:

[0125]

[0126] get Then calculate the absolute value of the difference between each small region and its adjacent regions:

[0127]

[0128] Maximum difference There is a power tower on one side of the boundary between the two corresponding areas.

[0129] Similarly, projecting the vertical direction can determine the upper and lower boundaries of the power tower.

[0130] Simultaneously, correlation analysis is performed on several parallel line groups obtained from the corner map and line segment detection to remove irrelevant parallel line groups. This correlation analysis primarily considers that power lines are generally connected to certain support locations on power towers, and these support locations are typically corner points. Therefore, by determining whether the detected straight line passes through a corner point in the corner map, candidate lines that are unlikely to be power lines can be further eliminated. Specifically, for all parallel line groups, after extending them into straight lines spanning the entire image through least binarization, it is sequentially determined whether this straight line passes through any corner point in the corner map. Parallel lines that do not pass through any corner point are removed and are no longer considered potential power lines.

[0131] After the initial inspection of power lines and power towers is completed, the results are corrected through iterative inspection. The iterative process is as follows:

[0132] 1) Map the power tower detection results to the edge detection map.

[0133] 2) such as Figure 5 As shown, the area excluding both sides of the power tower area. Retest the power lines.

[0134] 3) Estimate the proportion of the power tower area in the image based on the left and right width of the power line distribution.

[0135] 4) Based on the results of 3), the projection method was used again to inspect the power tower.

[0136] 5) After the power tower is re-inspected, calculate the IoU value with the previous power tower inspection result. The IoU threshold is set to 0.7. After correction, compare with the previous inspection result. If the IoU is greater than 0.7, it means that the two inspection results are consistent, output the result, and end the task; if it is less than or equal to 0.7, it means that there is a deviation from the previous inspection result, continue iterating until the result is consistent or the limit of the number of iterations is exceeded. For example Figure 6 The IoU is calculated for the two detection regions shown below:

[0137] .

Claims

1. A method for detecting power towers and power lines to correct the correlation between power towers and power lines, characterized in that, Includes the following steps: Step 1: Take an image of the power tower and power lines from above, and perform grayscale preprocessing on the image; Step 2: Perform edge detection on the preprocessed image to obtain a binarized edge map; Step 3: Power line detection is performed using the edge map, which includes line segment detection, merging line segments, and parallel line group clustering to obtain power lines represented by parallel line groups. Step 4: Power tower detection is performed using the edge map to obtain the power tower area defined by a rectangular frame. The method is as follows: Corner detection is performed on the edge map using a corner detection method based on curvature scale space to obtain a corner distribution map; then, the corner distribution map is integrally projected in the horizontal and vertical directions using pixel statistical projection to obtain horizontal and vertical projection histograms; finally, each projection value in the horizontal direction is replaced with its left and right adjacent values ​​using median smoothing. The median of the projections is used to replace each vertical projection value with its adjacent values ​​above and below. The median of each projection is used to obtain a smooth projection histogram. The proportion of the power tower in the edge map is estimated by the width and height of the power line region. Based on the proportion, the projection histograms in the horizontal and vertical directions are divided into multiple small regions. The drop between each small region and the adjacent region is calculated. The regions with the largest drop are the left and right boundaries and the top and bottom boundaries of the power tower. Step 5: Correct the detection results of power lines and power towers through iterative detection. The method is as follows: Map the rectangle representing the power tower detection result to the edge map, exclude the background areas on both sides of the power tower rectangle, and re-detect the power lines in the area inside the power tower rectangle using the same method as the initial detection. After obtaining new power line detection results, estimate the proportion of the power tower area in the edge map based on the left and right width of the power line distribution area, and then update the power tower detection parameters, that is, update the number of small area divisions. Re-detect the power tower using the histogram projection method until the requirements are met or the number of iterations is reached. The degree of change in results during the iterative detection of power towers is evaluated using Interchange of Units (IoU), and the number of iterations is determined based on the IoU. The IoU is calculated by dividing the overlapping portion of two regions by the sum of the values ​​of the two regions, and is used to measure the degree of overlap between two power tower detection results. When the IoU exceeds a set threshold... If the two detection results are consistent, the current result is output as the final result, and the task ends; if the IoU is less than or equal to the threshold... If the number of iterations exceeds the given maximum number of iterations, the task is forcibly terminated, and the detection results of the last round are output.

2. The power tower and power line detection method for power tower and power line correlation correction according to claim 1, characterized in that, In step 1, each captured image contains only one power tower target and power lines; the preprocessing includes adjusting image resolution, image grayscale conversion, histogram equalization, and noise reduction; wherein, during the grayscale conversion process, only the red and blue channels are retained to generate a grayscale image.

3. The method for detecting power towers and power lines for power tower and power line correlation correction according to claim 1, characterized in that, In step 3, the probabilistic Hough transform is used to detect the two endpoints of the line segments in the edge image; then the coordinates of the center point of the line segment, the slope of the line segment, and the intercept are calculated. The line segments are grouped according to their geometric distance. After grouping, the least squares method is used to fit the line segments in each group to a straight line and extend them to penetrate the image. Finally, the straight lines with similar slopes are grouped into parallel line groups. Finally, the parallel line groups are clustered according to their slopes, and the straight line with the most counts in the clustering results is retained, while messy straight lines with slopes that differ greatly from the electric field lines are filtered out.

4. The power tower and power line detection method for power tower and power line correlation correction according to claim 3, characterized in that, In the electric field lines represented by the resulting set of parallel lines, the distance between the two outermost electric field lines is... Estimate the width of the power tower, and then estimate the area where the power tower is located based on its length-to-width ratio. The edge map is rotated as a whole according to the power line detection results. Make the power tower area in the diagram horizontal, where The angle between the clustered line group and the vertical direction.

5. The power tower and power line detection method for power tower and power line correlation correction according to claim 4, characterized in that, The corner detection method based on curvature scale space includes: first, using the image contour extracted by edge detection to fill the gaps in the binarized edge contour; second, after filling, calculating the curvature of each pixel on the contour at a large scale, and determining it as a candidate corner if it exceeds a threshold; finally, tracking each pixel in the candidate corner set at a small scale to accurately locate the position of the corner.

6. A power tower and power line detection system for power tower and power line correlation correction, comprising: The preprocessing module performs grayscale preprocessing on the images of power towers and power lines captured from above; The edge detection module performs edge detection on the preprocessed image to obtain a binarized edge map. The power line detection module uses the edge map to detect power lines, which includes line segment detection, merging line segments, and parallel line group clustering to obtain power lines represented by parallel line groups. The power tower detection module uses the edge map to detect power towers and obtains a rectangular frame defining the power tower area. The method is as follows: a corner detection method based on curvature scale space is used to detect corners in the edge map, obtaining a corner distribution map; then, the corner distribution map is integrally projected in the horizontal and vertical directions using pixel statistical projection, obtaining horizontal and vertical projection histograms; finally, median smoothing is used to replace each projection value in the horizontal direction with its left and right adjacent values. The median of the projections is used to replace each vertical projection value with its adjacent values ​​above and below. The median of each projection is used to obtain a smooth projection histogram. The proportion of the power tower in the edge map is estimated by the width and height of the power line region. Based on the proportion, the projection histograms in the horizontal and vertical directions are divided into multiple small regions. The drop between each small region and the adjacent region is calculated. The regions with the largest drop are the left and right boundaries and the top and bottom boundaries of the power tower. The correlation correction module maps the rectangle to the edge map, excludes the background areas on both sides of the power tower rectangle, and re-detects power lines in the area inside the power tower rectangle using the same method as the initial detection. After obtaining new power line detection results, the proportion of the power tower area in the edge map is estimated based on the left and right widths of the power line distribution area, and then the power tower detection parameters are updated, that is, the number of small area divisions is updated, and the histogram projection method is used again to detect the power tower until the requirements are met or the number of iterations is reached. The degree of change in results during the iterative detection of power towers is evaluated using Interchange of Units (IoU), and the number of iterations is determined based on the IoU. The IoU is calculated by dividing the overlapping portion of two regions by the sum of the values ​​of the two regions, and is used to measure the degree of overlap between two power tower detection results. When the IoU exceeds a set threshold... If the two detection results are consistent, the current result is output as the final result, and the task ends; if the IoU is less than or equal to the threshold... If the number of iterations exceeds the given maximum number of iterations, the task is forcibly terminated, and the detection results of the last round are output.

7. The power tower and power line detection system for power tower and power line correlation correction according to claim 6, characterized in that, The detection system is deployed on the UAV embedded platform, providing the UAV flight control program with the function of detecting and locating the positions of power towers and power lines; the images of the power towers and power lines are acquired by the UAV.

Images