Unmanned aerial vehicle power inspection real-time analysis method based on edge computing

Abstract

The application discloses a real-time analysis method for unmanned aerial vehicle power inspection based on edge computing and belongs to the technical field of unmanned aerial vehicle power inspection. The method comprises the following steps: acquiring real-time inspection images of tower regions collected by an unmanned aerial vehicle on an inspection route; extracting abnormal pixel blocks from the real-time inspection images and standard reference images of the tower regions to obtain an initial abnormal block set; screening the abnormal pixel blocks according to their edge curvature change rates to obtain a target abnormal block set; dividing out a to-be-confirmed hidden danger region according to the distribution density of the target abnormal block set in the real-time inspection images; calculating a deviation cumulative amount according to the angle sequence between the texture direction in the to-be-confirmed hidden danger region and the standard texture direction of the corresponding region in the standard reference images; marking a suspected hidden danger region when the deviation cumulative amount is greater than a preset threshold; and generating an inspection alarm signal according to the position of the suspected hidden danger region. The application realizes automatic identification and accurate positioning of tower hidden dangers.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) power line inspection technology, specifically to a real-time analysis method for UAV power line inspection based on edge computing. Background Technology

[0002] In power line inspections using drones, real-time image analysis of pole and tower areas using onboard edge computing devices can promptly detect potential hazards such as damaged or detached line components and foreign objects. Existing real-time analysis methods typically compare the inspected image pixel-by-pixel with historical standard images, extracting abnormal areas based on brightness or color differences. These methods are highly sensitive to changes in the shooting environment; natural differences in the appearance of pole and tower surfaces due to varying lighting angles and weather conditions often lead to the detection of numerous false abnormal pixel clusters. These false clusters lack regularity in shape and distribution; directly incorporating them into subsequent analysis consumes computational resources and generates a large number of false alarms.

[0003] In addition, real structural problems such as loose parts and crack propagation often involve surface deformation accompanied by subtle and non-uniform changes in texture direction. It is difficult to distinguish them from the normal texture undulations of the background area based solely on pixel-level differences.

[0004] How to separate credible anomalous targets from an initial region of discrepancies containing a large number of false alarms, and how to accurately determine whether a region of discrepancies is a real structural anomaly using texture structure information, are problems that need to be solved in the current real-time inspection image analysis of UAVs. Summary of the Invention

[0005] This paper presents a real-time analysis method for power line inspection using drones based on edge computing. After image acquisition is completed at the front end of the drone, it can effectively suppress false anomalies caused by changes in lighting and slight differences in viewing angle, and reliably confirm suspected hidden danger areas from the perspective of texture structure changes, thereby reducing false alarms and improving the real-time performance and accuracy of inspection analysis.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a real-time analysis method for power grid inspection based on edge computing. In view of the problems of noise interference in the identification of anomalies in tower area inspection images and insufficient accuracy in the determination of hidden danger areas in the existing technology, the method improves the reliability of inspection analysis by double verification of edge curvature screening and texture deviation accumulation.

[0007] The method includes: acquiring real-time inspection images of the pole tower area collected by the UAV along the inspection route;

[0008] Based on the real-time inspection image and the standard reference image of the tower area, abnormal pixel clusters are extracted from the real-time inspection image to obtain an initial abnormal cluster set.

[0009] Based on the edge curvature change rate of each abnormal pixel block in the initial abnormal block set, the initial abnormal block set is filtered to obtain the target abnormal block set;

[0010] Based on the distribution density of the target abnormal clusters in the real-time inspection image, the potential hazard area to be confirmed is segmented from the real-time inspection image;

[0011] The cumulative deviation of the area to be identified is calculated based on the angle sequence between the texture direction in the area to be identified and the standard texture direction in the corresponding area of ​​the standard reference image.

[0012] When the cumulative deviation exceeds a preset cumulative deviation threshold, the area to be identified as a potential hazard area is marked as a suspected hazard area.

[0013] An inspection alarm signal is generated based on the location of the suspected potential hazard area in the real-time inspection image.

[0014] As a preferred embodiment of the present invention, the process of extracting the initial abnormal cluster set includes: aligning the real-time inspection image with the standard reference image to obtain an aligned inspection image; calculating the brightness difference between each pixel in the aligned inspection image and the corresponding pixel in the standard reference image; marking pixels with brightness differences greater than a preset brightness difference threshold as difference pixels; performing eight-neighbor connectivity clustering on the difference pixels to obtain the abnormal pixel clusters; and aggregating the abnormal pixel clusters as the initial abnormal cluster set. This scheme extracts differences pixel by pixel based on image alignment and uses eight-neighbor connectivity clustering to aggregate isolated or discrete difference pixels into clusters, which can more completely capture the minute abnormal shapes of the tower area caused by foreign objects, damage, etc., while avoiding misjudging individual noise points as abnormalities.

[0015] As a preferred embodiment of the present invention, the process of obtaining a target abnormal block set from an initial abnormal block set includes: extracting the edge contour of each abnormal pixel block in the initial abnormal block set; calculating the curvature value of each edge point on the edge contour; using the absolute value of the difference between the curvature values ​​of adjacent edge points on the edge contour as the edge curvature change rate; retaining the abnormal pixel block when the edge curvature change rate of all edge points of the abnormal pixel block is less than a preset curvature change threshold; discarding the abnormal pixel block when the edge curvature change rate of any edge point of the abnormal pixel block is greater than or equal to the preset curvature change threshold; and using all retained abnormal pixel blocks as the target abnormal block set. By judging the global consistency of the edge curvature change rate, false abnormal blocks with irregular boundaries caused by lighting changes, shadows, or background textures can be effectively filtered out, while retaining target blocks with smooth contours and suspected real device defects, thus reducing the false alarm rate in subsequent processing.

[0016] Furthermore, when calculating the curvature value of each edge point on the edge contour, parametric spline fitting can be performed on the edge contour. The first and second derivatives at each sampling point on the fitted curve are calculated, and the curvature value at the sampling point is calculated based on the first and second derivatives, serving as the curvature value of the corresponding edge point. Spline fitting can smooth contour noise, making curvature calculation more stable and improving the accuracy of abnormal clumping screening.

[0017] As a preferred technical solution of the present invention, the process of segmenting the potential hazard area from the real-time inspection image includes: calculating the center point of each target abnormal cluster in the target abnormal cluster set; performing Delaunay triangulation on all the center points to obtain a triangulation mesh; calculating the area of ​​each triangle in the triangulation mesh; marking the area covered by triangles with an area less than a preset area threshold as a high-density distribution area; and extracting the smallest bounding rectangle containing the high-density distribution area from the real-time inspection image as the potential hazard area. By using Delaunay triangulation to construct spatial adjacency relationships based on the cluster center points and measuring the degree of local clustering by the triangle area, high-risk areas with tightly clustered abnormal clusters can be accurately located, avoiding redundant analysis of sparsely distributed clusters and improving the efficiency of hazard area location.

[0018] As a preferred technical solution of the present invention, the process of calculating the cumulative deviation of the area to be identified as a potential hazard includes: sampling multiple first texture sampling points within the area to be identified as a potential hazard according to a preset step size; calculating the texture direction angle at each of the first texture sampling points; sampling multiple second texture sampling points within the area corresponding to the area to be identified as a potential hazard in the standard reference image according to the preset step size; calculating the standard texture direction angle at each of the second texture sampling points; calculating the absolute value of the difference between the texture direction angle at the same sampling position and the standard texture direction angle, as the included angle value; arranging the included angle values ​​at all sampling positions according to the sampling order to obtain the included angle sequence; and summing all included angle values ​​in the included angle sequence to obtain the cumulative deviation. This method, by comparing the texture direction of the candidate hazard area and the standard reference area at the same position, accumulates the local direction difference into a global deviation, which can sensitively reflect the regular changes in texture caused by equipment structural deformation, component displacement, etc., and improve the sensitivity of hazard identification.

[0019] Furthermore, when calculating the texture orientation angle at each of the first texture sampling points, a local window of a preset size can be established centered on the first texture sampling point; the horizontal and vertical gradients of each pixel within the local window are calculated; the structure tensor matrix of the local window is calculated based on the horizontal and vertical gradients; the principal eigenvector of the structure tensor matrix is ​​extracted; and the orientation angle of the principal eigenvector is used as the texture orientation angle at the first texture sampling point. Principal orientation estimation based on the structure tensor is robust to changes in image contrast, stably extracts the dominant texture orientation, and ensures the reliability of the texture orientation angle calculation.

[0020] As a preferred embodiment of the present invention, the process of marking the area to be identified as a suspected hidden danger area includes: obtaining a first area value of the area to be identified; querying an area-threshold mapping table based on the first area value to obtain the preset cumulative deviation threshold corresponding to the first area value; and marking the area to be identified as a suspected hidden danger area when the cumulative deviation is greater than the preset cumulative deviation threshold. By setting an adaptive area threshold, the judgment criteria are correlated with the size of the hidden danger area, and different decision thresholds are used for candidate areas of different sizes, effectively balancing the accurate identification of significant hidden dangers in small areas and minor deformations in large areas.

[0021] As a preferred embodiment of the present invention, the process of generating an inspection alarm signal includes: obtaining the pixel coordinates of the suspected hazard area in the real-time inspection image; converting the pixel coordinates into actual spatial coordinates in the tower area based on the shooting parameters of the UAV when acquiring the real-time inspection image; generating the inspection alarm signal based on the actual spatial coordinates and the acquisition timestamp of the real-time inspection image; and sending the inspection alarm signal to the ground monitoring terminal. By converting the image coordinates back into actual spatial coordinates, the alarm information directly includes the actual location of the hazard on the tower, facilitating rapid location and handling by maintenance personnel and improving the efficiency of the inspection closed loop.

[0022] Compared with existing technologies, this invention filters out noise blocks with abrupt contour changes by screening for edge curvature consistency of differential pixel blocks, extracts the area of ​​potential hazards to be confirmed by the spatial distribution density of target blocks, and performs fine verification by using the cumulative amount of texture direction deviation. Under the condition of limited edge computing resources, it realizes multi-level anomaly analysis from coarse detection to fine judgment, effectively suppresses false alarms caused by changes in illumination, background interference and local noise, and enhances the ability to identify real hazards such as small deformations of tower structures and foreign object attachments. It provides a highly reliable real-time analysis method for autonomous inspection of power lines by UAVs.

[0023] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0024] Initial anomalous clusters are filtered based on the rate of change of edge curvature. The edge contour of each anomalous pixel cluster is extracted, and the curvature value of each edge point is calculated. The absolute value of the curvature difference between adjacent edge points is used as the rate of change of edge curvature. Clusters are retained if the rate of change of curvature for all edge points is less than a preset threshold; otherwise, they are discarded. False discrepancies caused by lighting variations or registration errors typically have irregular, sharply fluctuating edges with a large rate of change of curvature; while genuine structural defects such as cracks and missing parts have relatively smooth and continuous edge changes. This filtering operation can eliminate false alarm clusters caused by uneven lighting, shadows, and slight misalignments, resulting in a more reliable set of target anomalous clusters, reducing unnecessary calculations in subsequent steps, and lowering false detection output.

[0025] Hazard identification is achieved by utilizing the distribution density and cumulative texture orientation deviation of target anomalous clusters. Delaunay triangulation is performed on the center point of the target anomalous cluster, and triangular areas with a pre-defined threshold are designated as high-density distribution regions, thus identifying the areas to be identified as hazard zones. Based on this, the texture orientation angles of the corresponding regions in this area and the standard reference image are simultaneously sampled, constructing an angle sequence and summing it to obtain the cumulative deviation. The cumulative deviation of texture orientation reflects the non-uniform deformation, misalignment, or orientation disturbances introduced by anomalous objects on the tower surface structure. Combining cluster density-based localization with structural texture verification makes the segmentation and judgment of real structural anomalies more accurate, avoiding misjudgments caused by relying solely on cluster distribution density, and enabling stable identification of hazards even under varying lighting conditions and complex backgrounds. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0027] Figure 1 This is a flowchart of a real-time analysis method for drone power line inspection based on edge computing;

[0028] Figure 2 This is a flowchart for filtering abnormal clusters based on the rate of change of edge curvature.

[0029] Figure 3 This is a flowchart for calculating the cumulative deviation of areas with potential hazards to be identified;

[0030] Figure 4 This is a density distribution map of the rate of change of curvature at the edges of abnormal pixel clusters;

[0031] Figure 5 This is a statistical chart of the area distribution of Delaunay's triangle. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] See Figure 1 This invention provides a real-time analysis method for UAV power line inspection based on edge computing, including:

[0034] Acquire real-time inspection images of the pole and tower area collected by the drone along the inspection route;

[0035] Based on the real-time inspection images and the standard reference images of the tower area, abnormal pixel clusters are extracted from the real-time inspection images to obtain an initial set of abnormal clusters.

[0036] Based on the rate of change of edge curvature of each abnormal pixel block in the initial abnormal block set, the initial abnormal block set is filtered to obtain the target abnormal block set;

[0037] Based on the distribution density of the target abnormal clusters in the real-time inspection image, the area of ​​potential hazards to be confirmed is segmented from the real-time inspection image;

[0038] The cumulative deviation of the area to be identified is calculated based on the angle sequence between the texture direction of the area to be identified and the standard texture direction of the corresponding area in the standard reference image.

[0039] When the cumulative deviation exceeds the preset cumulative deviation threshold, the area to be identified as a potential hazard area will be marked as a suspected hazard area.

[0040] Based on the location of suspected potential hazard areas in the real-time inspection images, an inspection alarm signal is generated.

[0041] In the specific implementation, the real-time inspection image is aligned with a standard reference image to obtain an aligned inspection image. The alignment process employs a feature-point-based image registration method, extracting stable feature points from both the real-time inspection image and the standard reference image. Stable feature point extraction utilizes a scale-invariant feature transformation algorithm, detecting extreme points in the difference-of-Gaussian space in both the real-time inspection image and the standard reference image, and generating a feature description vector for each extreme point. By calculating the Euclidean distance between the feature description vectors of the real-time inspection image and the standard reference image, feature points are matched to obtain a set of matched feature point pairs. A random sampling consensus algorithm is used to remove mismatched point pairs from the set of matched feature point pairs, and the homography matrix is ​​estimated. The homography matrix is ​​a 3×3 matrix that describes the perspective mapping relationship from the real-time inspection image plane to the standard reference image plane. Based on the homography matrix, a perspective transformation is performed on the real-time inspection image, mapping the coordinates of each pixel in the real-time inspection image to the coordinate system of the standard reference image, generating the aligned inspection image. The pixel coordinates of the same spatial location in the aligned inspection image and the standard reference image are kept consistent with each other.

[0042] Calculate the brightness difference between each pixel in the aligned inspection image and its corresponding pixel in the standard reference image. The brightness difference is defined as the absolute value of the difference between the grayscale value of the pixel in the aligned inspection image and the grayscale value of its corresponding pixel in the standard reference image. When both the real-time inspection image and the standard reference image are color images, the color image is first converted to a grayscale image before the brightness difference is calculated. A preset brightness difference threshold is set, with a value of 30. This value is based on the fact that, under standard lighting conditions, statistical analysis shows that the grayscale fluctuation range caused by normal texture differences does not exceed 25 on the surface of tower metal components, insulator strings, and conductors. Setting the preset brightness difference threshold to 30 can distinguish between structural anomalies and normal texture differences. Pixels with brightness differences greater than the preset brightness difference threshold are marked as difference pixels.

[0043] Eight-neighbor connectivity clustering is performed on the dissimilar pixels to obtain anomalous pixel clusters. The process of eight-neighbor connectivity clustering is as follows: traverse all pixels in the aligned inspection image. When an unlabeled dissimilar pixel is detected, assign a new cluster label to the unlabeled dissimilar pixel. Using the unlabeled dissimilar pixel as the seed point, recursively search for pixels within the eight-neighbor range of the seed point. The eight-neighbor range refers to the eight adjacent pixels within a 3×3 neighborhood centered on the current pixel, excluding the center point. If another dissimilar pixel exists within the eight-neighbor range and has not yet been labeled, assign the same cluster label to the other dissimilar pixel and continue the recursive search using the other dissimilar pixel as the new seed point. Repeat the above neighborhood search and labeling process until all reachable dissimilar pixels from the initial seed point are assigned the same cluster label, completing the labeling of a connected region. Then, continue traversing the aligned inspection image, searching for the next unmarked difference pixel, and repeat the above process of assigning cluster labels and recursively searching until all difference pixels in the aligned inspection image are labeled. A set of interconnected difference pixels with the same cluster label is considered an anomalous pixel cluster.

[0044] All anomalous pixel clusters extracted from the aligned inspection image are collected to form an initial anomalous cluster set. Each anomalous pixel cluster in the initial anomalous cluster set consists of a group of adjacent pixels with a brightness difference greater than a preset brightness difference threshold.

[0045] In specific implementation, please refer to Figure 2The edge contour of each anomalous pixel cluster within the initial set of anomalous clusters is extracted. The edge contour extraction employs a morphological boundary tracking algorithm. For each anomalous pixel cluster, all pixels within the cluster are scanned, and the boundary pixels adjacent to the background pixels are located. Starting from any one of these boundary pixels, the algorithm iterates through the adjacent boundary pixels along the outer boundary of the anomalous pixel cluster in a clockwise or counterclockwise direction until it returns to the starting point, forming a closed pixel sequence that constitutes the outer boundary of the anomalous pixel cluster. This closed pixel sequence is the edge contour of the anomalous pixel cluster.

[0046] Calculate the curvature value at each edge point on the edge contour. The calculation method is as follows: perform parametric spline fitting on the edge contour to construct a continuous curve representation with arc length as the variable. Represent the two-dimensional image coordinates of each edge point on the edge contour as follows: subscript This is the index of the edge point in the edge contour sequence. It is calculated using the arc length of the edge contour. Using the parameters, cubic spline interpolation is performed along the cumulative arc length of the edge contour to obtain... Coordinates with respect to arc length spline function and Coordinates with respect to arc length spline function The spline function described above has continuous first and second derivatives across the entire arc length of the edge contour. On the fitted spline curve, for each edge point corresponding to an arc length position... ,calculate exist The first derivative at and second derivative ,as well as exist The first derivative at and second derivative Calculate the arc length position using the curvature formula. curvature value at The curvature value is used as the curvature value for the corresponding edge point. The curvature formula is:

[0047]

[0048] in, Indicates the position of the arc length on the edge contour. The curvature value at the edge point, This indicates the accumulation along the edge contour from the starting point to the th edge. The arc length of each edge point Representing spline functions In arc length The first derivative at that point, Representing spline functions In arc length The second derivative at that point, Representing spline functions In arc length The first derivative at that point, Representing spline functions In arc length The second derivative at that point, This indicates the operation of taking the absolute value.

[0049] The absolute value of the difference in curvature values ​​between two adjacent edge points on the edge profile is taken as the edge curvature change rate. For the adjacent edge points in the edge profile sequence... The edge point and the first The edge point, the first The curvature value of each edge point is , No. The curvature value of each edge point is The rate of change of edge curvature between the two adjacent edge points mentioned above The calculation method is as follows:

[0050]

[0051] in, Indicates the first The edge point and the first The rate of change of edge curvature between edge points Indicates the position of the arc length on the edge contour. The curvature value at the edge point, Indicates the position of the arc length on the edge contour. The curvature value of the edge point.

[0052] Traverse all edge points of each anomalous pixel cluster and calculate the rate of change of edge curvature between each pair of adjacent edge points on the edge contour of the anomalous pixel cluster. Set a preset curvature change threshold of 0.5. The basis for this value is that the contour edge of the tower component in the inspection image is usually composed of straight line segments and small curvature arc segments. The curvature of the normal component contour changes gently along the edge, and the rate of change of curvature between adjacent edge points is statistically concentrated in the range of 0 to 0.3. However, anomalous pixel clusters caused by noise, shadows, or unstructured interference have irregular jagged edges, and the rate of change of curvature between adjacent edge points usually exceeds 0.5. Therefore, setting the preset curvature change threshold to 0.5 can distinguish between structural anomalous clusters and interference clusters.

[0053] Determine whether any pair of adjacent edge points in the abnormal pixel cluster exhibits an edge curvature change rate greater than or equal to a preset curvature change threshold. If any pair of adjacent edge points exhibits an edge curvature change rate greater than or equal to 0.5, the abnormal pixel cluster is removed from the initial abnormal pixel cluster set. If the edge curvature change rates of all adjacent edge point pairs of the abnormal pixel cluster are less than the preset curvature change threshold of 0.5, the abnormal pixel cluster is retained.

[0054] All anomalous pixel clusters retained after the above screening are considered as the target anomalous cluster set. The anomalous pixel clusters in the target anomalous cluster set have smooth edge contours and continuously varying curvature, corresponding to structurally anomalous regions with regular geometric shapes in the tower area.

[0055] See Figure 4 In the figure, the horizontal axis represents the rate of change of edge curvature of the abnormal pixel clusters, ranging from 0 to 3, and the vertical axis represents the probability density of the corresponding rate of change of edge curvature. The dashed line in the figure represents the preset edge curvature change threshold of 0.5. The blue dashed curve is the density distribution of the rate of change of edge curvature of the retained target abnormal clusters, which is mainly concentrated in the range of 0 to 0.5, and reaches a peak in the range of 0.1 to 0.3, with a peak probability density of approximately 3.2, indicating that the rate of change of edge curvature of the target abnormal clusters is small and the change is gradual, and the edge contour is relatively smooth. The red solid curve is the density distribution of the rate of change of edge curvature of the removed abnormal clusters. Its probability density shows a slow upward trend in the range above 0.5, and there is continuous density fluctuation in the region above 0.5, indicating that the rate of change of edge curvature of the removed abnormal clusters is large, the edge contour is not smooth, and the curvature change is drastic.

[0056] In practical implementation, the center point of each target anomaly cluster in the target anomaly cluster set is calculated. For the first target anomaly cluster in the target anomaly cluster set... The target abnormal cluster, obtain the first The set of image coordinates of all pixels contained in a target anomaly block, denoted as the i-th... The total number of pixels within each target anomaly block is , No. Within the first target anomaly block The x-coordinate of each pixel is , No. Within the first target anomaly block The ordinate of each pixel is ,in This is the index of the pixel within the cluster. . No. The x-coordinate of the center point of the target anomalous cluster The value is obtained by calculating the arithmetic mean of the x-coordinates of all pixels within the block. The ordinate of the center point of each target anomalous cluster It is obtained by calculating the arithmetic mean of the ordinates of all pixels within the block. The calculation method is as follows:

[0057]

[0058] in, Indicates the first The x-coordinate of the center point of each target anomalous cluster Indicates the first The ordinate of the center point of each target anomalous cluster Indicates the first The total number of pixels contained in a target anomaly cluster. Indicates the first Within the first target anomaly block The x-coordinate of each pixel Indicates the first Within the first target anomaly block The ordinate of each pixel Indicates the index of the pixel within the block. This represents the index of the block in the target anomaly block set.

[0059] Delaunay triangulation is performed on the center points of all target anomaly clusters to obtain a triangulated mesh. A center point set is then created by combining the center points of all target anomaly clusters in the target anomaly cluster set, where each center point has two-dimensional coordinates. The Delaunay triangulation mesh is constructed using a point-by-point insertion method: an initial hypertriangle containing all center points is built, and center points from the center point set are inserted one by one into the current triangulation; for each inserted center point, the triangle containing that center point is located, and the triangle is split into three new triangles. The Delaunay property is restored through an edge flipping operation, i.e., checking whether the quadrilateral formed by any two adjacent triangles satisfies the empty circle criterion; if not, the diagonal is flipped; after all center points have been inserted, the triangles associated with the vertices of the initial hypertriangle are deleted, resulting in the Delaunay triangulation mesh generated from the center point set. The Delaunay triangulation mesh consists of multiple triangles, where the three vertices of each triangle are the center points of the target anomalous clique.

[0060] Calculate the area of ​​each triangle in the Delaunay triangulation mesh. For any triangle in the Delaunay triangulation mesh, obtain the coordinates of the three vertices of the triangle, denoted as . , and ,in Let be the coordinates of the first vertex of the triangle. Let be the coordinates of the second vertex of the triangle. Let be the coordinates of the third vertex of the triangle. The area of ​​the triangle. The calculation uses the vector cross product method, and the formula is:

[0061]

[0062] in, Represents the area of ​​the triangle. This represents the x-coordinate of the first vertex of the triangle. This represents the y-coordinate of the first vertex of the triangle. This represents the x-coordinate of the second vertex of the triangle. This represents the y-coordinate of the second vertex of the triangle. This represents the x-coordinate of the third vertex of the triangle. This represents the ordinate of the third vertex of the triangle. This indicates the operation of taking the absolute value.

[0063] A preset area threshold of 200 square pixels is set. This value is based on the following: In tower inspection areas, actual potential hazards typically correspond to a dense cluster of multiple abnormal clusters in the image, with small distances between the center points of adjacent clusters, resulting in a correspondingly small area for the Delaunay triangles formed by these center points. Based on the resolution of the tower inspection images and the typical size of potential hazards, the upper limit of the triangle area corresponding to a high density of abnormal clusters is set at 200 square pixels. The areas covered by triangles with areas smaller than the preset area threshold of 200 square pixels in the Delaunay triangulation mesh are marked as high-density distribution areas. High-density distribution areas consist of the image areas covered by all triangles that meet the area condition.

[0064] In the real-time inspection image, the smallest bounding rectangle encompassing all high-density distribution areas is selected as the area to be identified as a potential hazard. The minimum and maximum x-coordinates, as well as the minimum and maximum y-coordinates, of all pixels in the high-density distribution area within the real-time inspection image are determined. A smallest bounding rectangle is constructed with the minimum x-coordinate as the left boundary, the maximum x-coordinate as the right boundary, the minimum y-coordinate as the upper boundary, and the maximum y-coordinate as the lower boundary. The area defined by this smallest bounding rectangle in the real-time inspection image is the area to be identified as a potential hazard.

[0065] See Figure 5In the graph, the horizontal axis represents the area of ​​triangles in the Delaunay triangulation mesh, in square pixels, ranging from 0 to 1400 square pixels; the vertical axis represents the number of triangles within the corresponding area. The graph distinguishes between high-density and low-density triangles using different colors and fill styles. The green diagonal lines represent high-density triangles with an area less than a preset area threshold of 200 square pixels, while the red diagonal lines represent low-density triangles with an area greater than this threshold. The black dashed vertical lines clearly indicate the preset area threshold at the 200 square pixel area.

[0066] As can be observed in the image, the areas of high-density triangles are concentrated in the range of 0 to 200 square pixels, with the peak number occurring between 30 and 70 square pixels, exceeding 110 in some cases. This indicates that the center points of the target anomaly clusters form a densely clustered region within this range. As the triangle area gradually approaches 200 square pixels, the number of triangles decreases, suggesting that the spatial scale of the dense region is limited. Triangles exceeding 200 square pixels are classified as low-density triangles, with a more dispersed area distribution and a relatively smaller number. These triangles extend to 1400 square pixels, while their overall number remains low, indicating that the center points of the target anomaly clusters are widely spaced within this range, resulting in a lower cluster density.

[0067] In specific implementation, please refer to Figure 3 Within the area to be identified as a potential hazard, multiple first texture sampling points are obtained by sampling at equal intervals in both the row and column directions according to a preset step size. The preset step size is 8 pixels. This value is chosen because the size of the area to be identified as a potential hazard is typically in the thousands of pixels range. Sampling at 8-pixel intervals can obtain sufficiently dense texture direction sampling points while considering computational complexity, ensuring spatial representativeness for subsequent calculations of cumulative deviations. Starting from the top left corner of the area to be identified as a row-major region, the positions of the first texture sampling points are determined sequentially at 8-pixel intervals along the row direction and 8-pixel intervals along the column direction. The image coordinates of each first texture sampling point are then labeled as follows: ,in Indicates the first The row coordinates of the first texture sampling point Indicates the first The column coordinates of the first texture sampling point This is the index of the first texture sampling point.

[0068] Calculate the texture orientation angle at each first texture sampling point. A two-dimensional local window is established centered on the first texture sampling point. The preset size of the local window is 16 pixels × 16 pixels. This preset size is based on the fact that a 16-pixel × 16-pixel window can cover multiple cycles of the typical texture of the tower component, allowing the structure tensor matrix to effectively reflect the main direction of the texture within the window. The row coordinates contained within the local window are from... arrive The column coordinates are from arrive All pixels within the local window. For each pixel within the local window, the Sobel operator is used to calculate the horizontal and vertical gradients. The Sobel operator contains two 3×3 convolution kernels; the horizontal kernel is used to calculate the horizontal gradient. Vertical convolution kernels are used to calculate the vertical gradient. For the row coordinates within a local window, Column coordinates are The horizontal gradient value of a pixel is denoted as . The vertical gradient value is denoted as .

[0069] A structure tensor matrix is ​​constructed based on the horizontal and vertical gradients of all pixels within the local window. The structure tensor matrix is ​​a 2×2 symmetric matrix, and its expression is:

[0070]

[0071] in, Indicates the first The structure tensor matrix corresponding to the local window centered on the first texture sampling point; Indicates the first The preset size centered on the first texture sampling point is... Pixels A local window of a pixel; For local windows The row and column coordinates of the inner pixel; Represents a local window Inner coordinates are Column coordinates are The horizontal gradient value of the pixel; Represents a local window Inner coordinates are Column coordinates are The vertical gradient value of the pixel; Indicates a local window Sum all pixels within the range.

[0072] For the structure tensor matrix Eigenvalue decomposition is performed to obtain two eigenvalues ​​and their corresponding eigenvectors. The eigenvalue with the larger value is taken as the largest eigenvalue, and the eigenvector corresponding to the largest eigenvalue is taken as the principal eigenvector. The principal eigenvector is a two-dimensional unit vector, and its row direction component is denoted as . The column direction component is denoted as The direction angle of the principal eigenvector is taken as the first... The texture orientation angle at the first texture sampling point is calculated by taking the arctangent of the two components of the principal feature vector, denoted as . ,in Indicates the first Texture orientation angle at the first texture sampling point.

[0073] Within the region corresponding to the coordinates of the potential hazard area in the standard reference image, multiple second texture sampling points are obtained by sampling at equal intervals of 8 pixels, identical to the first texture sampling point. The sampling positions of the second texture sampling points correspond one-to-one with the sampling positions of the first texture sampling points in space. The second texture sampling point and the first Each first texture sampling point has the same row and column coordinates. Using the same method as calculating the texture orientation angle of the first texture sampling point, the standard texture orientation angle at each second texture sampling point is calculated: [The text then continues with a description of calculating the texture orientation angle at each second texture sampling point.] Centered on the second texture sampling point, a 16-pixel × 16-pixel local window is established. The horizontal and vertical gradients of each pixel within the window are calculated using the Sobel operator, and a structure tensor matrix is ​​constructed. Eigenvalue decomposition is performed on the structure tensor matrix, and the principal eigenvector corresponding to the largest eigenvalue is selected. The direction angle of the principal eigenvector is then used as the first... The standard texture orientation angle at the second texture sampling point is denoted as... .

[0074] For the same sampling location Calculate the first Texture orientation angle of the first texture sampling point With the The standard texture orientation angle of the second texture sampling point The absolute value of the difference is taken as the first... The included angle value at each sampling location is denoted as... ,in Indicates the first The included angle value at each sampling location.

[0075] Arrange the included angle values ​​at all sampling locations in row-major order to obtain the included angle sequence. Row-major sampling order refers to the order based on the row coordinates of the first texture sampling point. Arranged in ascending order, by column coordinates within the same row. Arrange the included angle values ​​in ascending order. The included angle sequence is denoted as... ,in This represents the total number of sampling locations.

[0076] The cumulative deviation is obtained by summing all the included angle values ​​in the included angle sequence, denoted as . ,in This indicates the cumulative deviation in the area of ​​potential hazards to be confirmed. Indicates the first angle in the angle sequence Angle values.

[0077] In practice, the first area value of the area to be identified as a potential hazard is obtained. This area is defined by the smallest bounding rectangle. The total number of pixels contained within the smallest bounding rectangle is counted, and this total number of pixels is used as the first area value, denoted as [value missing]. .

[0078] Based on the first area value The system queries a pre-defined area-threshold mapping table to obtain the pre-defined cumulative deviation threshold corresponding to the first area value. The area-threshold mapping table is established through offline statistical analysis. The establishment process involves collecting sample images of marked hazard areas in various inspection scenarios of the tower area, extracting the area value of the hazard area in each sample image, and determining the optimal cumulative deviation threshold for each area value through manual interpretation and cross-validation. Then, the area values ​​are grouped by interval, and the median of the optimal threshold for each sample within each area interval is calculated. This median is used as the pre-defined cumulative deviation threshold for the corresponding area interval. The area-threshold mapping table records the following: when the first area value is in the range (0, 500] square pixels, the corresponding preset cumulative deviation threshold is 120 degrees; when the first area value is in the range (500, 2000] square pixels, the corresponding preset cumulative deviation threshold is 200 degrees; when the first area value is in the range (2000, 8000] square pixels, the corresponding preset cumulative deviation threshold is 350 degrees; and when the first area value is greater than 8000 square pixels, the corresponding preset cumulative deviation threshold is 500 degrees. The above area range division is based on the following: in tower inspection images, 500 square pixels represent local anomalies of small components, 2000 square pixels represent range anomalies of medium-sized components, and 8000 square pixels represent anomalies of large structural areas. The larger the area, the greater the tolerance for cumulative texture deviation, therefore the threshold increases progressively.

[0079] When the cumulative deviation of the potential hazard area to be confirmed Greater than the preset cumulative deviation threshold obtained from the area-threshold mapping table When the hazard to be confirmed is discovered, the area is marked as a suspected hazard area; when the cumulative deviation is... Less than or equal to the preset cumulative deviation threshold At that time, areas identified as potential hazards are not marked.

[0080] For marked suspected hazard areas, obtain the pixel coordinates of these areas in the real-time inspection image. The pixel coordinates are obtained by extracting the row and column positions of the four vertices of the smallest bounding rectangle of the suspected hazard area in the real-time inspection image. The coordinates of the four vertices are denoted as follows: , , and ,in Represents the column coordinates of a pixel in the image. This represents the row coordinates of a pixel in the image.

[0081] Based on the shooting parameters of the drone when acquiring real-time inspection images, the pixel coordinates are converted into actual spatial coordinates within the tower area. The shooting parameters include the camera intrinsic parameter matrix. The position coordinates of the UAV in the world coordinate system The attitude angles of the UAV (including pitch angle) Roll angle and yaw angle ) and gimbal angle (gimbal pitch angle) and gimbal yaw angle Camera intrinsic parameter matrix The form is:

[0082]

[0083] in, Represents the camera intrinsic parameter matrix. This represents the camera's horizontal focal length, expressed in pixels. This represents the camera's vertical focal length, expressed in pixels. Represents the column coordinates of the optical center point in the image. Represents the row coordinates of the optical center point in the image.

[0084] The coordinate transformation process is achieved through an imaging geometry model: based on the UAV pose information and gimbal angle, a rotation matrix is ​​constructed from the world coordinate system to the camera coordinate system. Translation vector For a specific pixel coordinate in a suspected potential hazard area According to the imaging model, we have:

[0085]

[0086] in, , which is the scale factor, representing the depth value of a pixel in the camera coordinate system; This represents the pixel coordinates of a suspected hazardous area; For pixels The actual spatial coordinates of the tower area in the three-dimensional world coordinate system; The camera extrinsic parameter matrix is ​​3×4, derived from the rotation matrix. Translation vector It is pieced together; This is the camera intrinsic parameter matrix.

[0087] By solving the back projection transformation of the above imaging model, the vertex pixel coordinates of the suspected hidden danger area are converted into actual spatial coordinates in the world coordinate system. The actual spatial coordinates of the suspected potential hazard area are transformed from the center pixel coordinates of the suspected hazard area as the spatial location of the suspected hazard area.

[0088] By combining actual spatial coordinates and the acquisition timestamps of real-time inspection images, an inspection alarm signal is generated. The data structure of the inspection alarm signal includes: spatial location information of suspected hazard areas (in actual spatial coordinates). The data includes the image name, the discovery time (recorded as the acquisition timestamp of the real-time inspection image), and the corresponding real-time inspection image identifier (recorded as the file name or acquisition number of the real-time inspection image).

[0089] The inspection alarm signals are transmitted to the ground monitoring terminal via an airborne data link. The airborne data link is a wireless communication module configured on the UAV, which uses 4G or 5G cellular networks or private network radio communication to transmit the inspection alarm signals to the inspection management platform of the ground monitoring terminal in a structured message format.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A real-time analysis method for UAV power line inspection based on edge computing, characterized in that, include: Acquire real-time inspection images of the pole and tower area collected by the drone along the inspection route; Based on the real-time inspection image and the standard reference image of the tower area, abnormal pixel clusters are extracted from the real-time inspection image to obtain an initial set of abnormal clusters. Based on the edge curvature change rate of each abnormal pixel block in the initial abnormal block set, the initial abnormal block set is filtered to obtain the target abnormal block set; Based on the distribution density of the target abnormal clusters in the real-time inspection image, the potential hazard area to be confirmed is segmented from the real-time inspection image; The cumulative deviation of the area to be identified is calculated based on the angle sequence between the texture direction in the area to be identified and the standard texture direction in the corresponding area of ​​the standard reference image. When the cumulative deviation exceeds a preset cumulative deviation threshold, the area to be identified as a potential hazard area is marked as a suspected hazard area. Based on the location of the suspected potential hazard area in the real-time inspection image, an inspection alarm signal is generated.

2. The real-time analysis method for UAV power line inspection based on edge computing according to claim 1, characterized in that, The step involves extracting abnormal pixel clusters from the real-time inspection image based on the real-time inspection image and a standard reference image of the tower area to obtain an initial set of abnormal clusters, including: Align the real-time inspection image with the standard reference image to obtain an aligned inspection image; Calculate the brightness difference between each pixel in the aligned inspection image and the corresponding pixel in the standard reference image; Pixels whose brightness difference is greater than a preset brightness difference threshold are marked as differential pixels; Perform eight-neighbor connectivity clustering on the differential pixels to obtain the abnormal pixel clusters; The abnormal pixel clusters are collected to form the initial abnormal cluster set.

3. The real-time analysis method for UAV power line inspection based on edge computing according to claim 1, characterized in that, The step of filtering the initial set of abnormal pixels based on the edge curvature change rate of each abnormal pixel block in the initial set of abnormal blocks to obtain the target set of abnormal blocks includes: Extract the edge contour of each abnormal pixel block within the initial abnormal block set; Calculate the curvature value of each edge point on the edge profile; The absolute value of the difference in curvature values ​​between adjacent edge points on the edge contour is taken as the edge curvature change rate. When the rate of change of edge curvature of all edge points of the abnormal pixel cluster is less than a preset curvature change threshold, the abnormal pixel cluster is retained. When the rate of change of edge curvature at any edge point of the abnormal pixel cluster is greater than or equal to the preset curvature change threshold, the abnormal pixel cluster is removed. All retained abnormal pixel clusters are used as the target abnormal cluster set.

4. The real-time analysis method for UAV power line inspection based on edge computing according to claim 1, characterized in that, The step of segmenting the area to be identified as a potential hazard from the real-time inspection image based on the distribution density of the target abnormal clusters in the real-time inspection image includes: Calculate the center point of each target anomaly cluster in the target anomaly cluster set; Perform Delaunay triangulation on all the center points to obtain a triangulated mesh; Calculate the area of ​​each triangle in the triangulated mesh; The area covered by triangles with an area smaller than a preset area threshold is marked as a high-density distribution area; The smallest bounding rectangle containing the high-density distribution area is extracted from the real-time inspection image and used as the area of ​​potential hazard to be confirmed.

5. The real-time analysis method for UAV power line inspection based on edge computing according to claim 1, characterized in that, The step of calculating the cumulative deviation of the area to be identified as a potential hazard based on the angle sequence between the texture direction within the area to be identified and the standard texture direction of the corresponding area in the standard reference image includes: Multiple first texture sampling points are sampled within the area of ​​the potential hazard to be confirmed according to a preset step size; Calculate the texture orientation angle at each of the first texture sampling points; In the standard reference image, within the area corresponding to the potential hazard area to be confirmed, multiple second texture sampling points are sampled according to the preset step size; Calculate the standard texture orientation angle at each of the second texture sampling points; Calculate the absolute value of the difference between the texture orientation angle and the standard texture orientation angle at the same sampling position, and use it as the included angle value; Arrange the included angle values ​​at all sampling positions in the sampling order to obtain the included angle sequence; The cumulative deviation is obtained by summing all the included angle values ​​in the included angle sequence.

6. The real-time analysis method for UAV power line inspection based on edge computing according to claim 5, characterized in that, The calculation of the texture orientation angle at each of the first texture sampling points includes: A local window of a preset size is established with the first texture sampling point as the center; Calculate the horizontal and vertical gradients of each pixel within the local window; Calculate the structure tensor matrix of the local window based on the horizontal gradient and the vertical gradient; Extract the principal eigenvectors of the structure tensor matrix; The orientation angle of the main feature vector is taken as the texture orientation angle at the first texture sampling point.

7. The real-time analysis method for UAV power line inspection based on edge computing according to claim 1, characterized in that, When the accumulated deviation exceeds a preset accumulated deviation threshold, the area to be identified as a potential hazard area is marked as such, including: Obtain the first area value of the area to be identified as a potential hazard. Based on the first area value, query the area-threshold mapping table to obtain the preset cumulative deviation threshold corresponding to the first area value; When the cumulative deviation exceeds the preset cumulative deviation threshold, the area to be identified as a potential hazard area is marked as a suspected hazard area.

8. The real-time analysis method for UAV power line inspection based on edge computing according to claim 1, characterized in that, The step of generating an inspection alarm signal based on the location of the suspected hazard area in the real-time inspection image includes: Obtain the pixel coordinates of the suspected potential hazard area in the real-time inspection image; Based on the shooting parameters of the UAV when collecting the real-time inspection images, the pixel coordinates are converted into actual spatial coordinates in the tower area; The inspection alarm signal is generated based on the actual spatial coordinates and the acquisition timestamp of the real-time inspection image; The inspection alarm signal is sent to the ground monitoring terminal.

9. The real-time analysis method for UAV power line inspection based on edge computing according to claim 2, characterized in that, The step of performing eight-neighbor connectivity clustering on the differential pixels to obtain the abnormal pixel cluster includes: starting from any unlabeled differential pixel, recursively searching for other differential pixels within its eight-neighbor range and marking them as the same cluster, until all differential pixels are marked, and taking the set of pixels with the same cluster mark as the abnormal pixel cluster.

10. The real-time analysis method for UAV power line inspection based on edge computing according to claim 3, characterized in that, The calculation of the curvature value of each edge point on the edge contour includes: performing parametric spline fitting on the edge contour, calculating the first derivative and the second derivative at each sampling point on the fitted curve, and calculating the curvature value at the sampling point based on the first derivative and the second derivative, which is used as the curvature value of the corresponding edge point.

Images