Power line hidden danger rapid identification method fusing unmanned aerial vehicle inspection and three-dimensional modeling
By optimizing UAV pose and 3D modeling, and combining multi-view triangulation and the catenary formula, the reflection was removed by mirror transformation. This solved the problem of conductor modeling deviating from physical laws under complex image conditions, and achieved accurate identification of conductor shape and clearance calculation, thus improving the accuracy and efficiency of power line hazard identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI WANSU ELECTRIC POWER TRANSPORTATION INSPECTION TECH CO LTD
- Filing Date
- 2025-09-29
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies struggle to accurately construct a unified local three-dimensional spatial reference system for the poses of conductors, water surfaces, and UAVs under complex imaging conditions. They also fail to fully incorporate the physical and natural suspension characteristics of the catenary, resulting in significant discrepancies between the calculated clearance and the actual clearance, and hindering the rapid and accurate generation of hazard mitigation information.
The pixel coordinates of the rigid body are extracted by a local feature detection algorithm, the pose of the UAV and the three-dimensional coordinates of the rigid body are optimized, the water surface plane is fitted by multi-view triangulation and singular value decomposition, the height of the guide is calculated by the catenary formula, the reflection is stripped by mirror transformation, and the guide is calibrated by perspective projection function and bundle adjustment, so as to achieve accurate separation between the real guide and the reflection.
It improves the positioning accuracy to the centimeter level, establishes a stable spatial benchmark, accurately describes the water surface morphology, follows the natural suspension law of the conductor, significantly improves the accuracy and efficiency of hazard identification, and meets the needs of efficient operation and maintenance of power lines across water.
Smart Images
Figure CN121121571B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power line safety monitoring technology, specifically to a method for rapid identification of power line hazards that integrates drone inspection and 3D modeling. Background Technology
[0002] In the field of power system operation and maintenance, the inspection of power lines crossing rivers and other water-crossing sections is a key link in ensuring the safe and stable operation of the power grid. In such scenarios, the lines need to cross wide water areas, often accompanied by complex terrain, such as steep riverbanks, large water areas without continuous shoreline references, and a lack of effective ground observation points. Therefore, drone inspection has become an important technical means in this scenario. However, the images captured by drones are easily affected by water reflections, changes in lighting, such as backlighting leading to low contrast in conductor imaging, and differences in multi-view shooting, such as the deviation of image perspective at different flight altitudes and headings. These factors result in a high degree of overlap between the conductor image and the conductor's reflection on the water surface, and blurred boundaries.
[0003] Existing technologies struggle to accurately construct a unified local 3D spatial reference system for conductors, water surfaces, and UAV poses under complex imagery conditions affected by reflections. Furthermore, they fail to fully leverage the physical, natural suspension characteristics of catenaries to achieve efficient stripping of the actual conductor and rapid, accurate calculation of clearance. On one hand, the global geodetic coordinate system, when used for local 3D modeling of water-span sections, suffers from insufficient conversion accuracy and efficiency, leading to deviations in the spatial relationship description of elements such as conductors and water surfaces. On the other hand, image analysis lacks targeted methods for stripping conductor reflections on the water surface, easily mistaking reflections for actual conductors. Simultaneously, the modeling of conductor morphology does not fully adhere to the physical laws of catenaries, resulting in significant discrepancies between clearance calculations and reality. Ultimately, this hinders the rapid and accurate generation of hazard mitigation information, failing to meet the demands of efficient operation and maintenance of power lines across water-span sections. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention proposes a rapid identification method for power line hazards that integrates UAV inspection and 3D modeling. This method solves the problem of lacking specific means to remove the reflection of conductors on the water surface, which easily leads to the reflection being misjudged as the actual conductor.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A set of images captured by a drone during inspection is obtained, and the coarse pose of the drone is obtained. The pixel coordinates of the rigid body are extracted from the image set through a local feature detection algorithm and marked as rigid body pixel coordinates. The initial rigid body 3D coordinates are obtained. The initial rigid body 3D coordinates and the drone coarse pose are optimized by minimizing the sum of squares of the Euclidean distance between the rigid body pixel coordinates and the projected coordinates. The rigid body 3D coordinates and multi-view pose are obtained. The projected coordinates are the pixel coordinates of the initial rigid body 3D coordinates projected onto the images in the image set.
[0007] Based on multi-view pose, multi-view triangulation is performed on the image set to obtain a three-dimensional point set of the water-land intersection line. The singular value decomposition method is used to fit the three-dimensional point set of the water-land intersection line to obtain the water surface plane.
[0008] Obtain the rigid body three-dimensional coordinates of two rigid bodies connected by a power conductor, and label them as the first rigid body three-dimensional coordinates and the second rigid body three-dimensional coordinates, respectively. Construct a vertical plane based on the first rigid body three-dimensional coordinates and the second rigid body three-dimensional coordinates. Calculate the actual three-dimensional coordinates of the power conductor based on the catenary formula based on the vertical plane. Fit the three-dimensional curve of the power conductor based on the actual three-dimensional coordinates of the power conductor.
[0009] The three-dimensional curve of the power conductor is projected onto the image in the image set using a perspective projection function to obtain a two-dimensional projection curve. A candidate pixel set is obtained based on the two-dimensional projection curve. The three-dimensional curve of the power conductor is mirrored based on the water surface plane to obtain a reflected three-dimensional curve. The reflected three-dimensional curve is projected onto the image in the image set using a perspective projection function to obtain a two-dimensional mirror projection curve. The minimum Euclidean distance from each pixel in the candidate pixel set to the two-dimensional projection curve and the two-dimensional mirror projection curve is calculated to obtain the real curve residual and the mirror curve residual. The real conductor pixels are obtained based on the comparison results of the real curve residual and the mirror curve residual.
[0010] Furthermore, obtain the camera intrinsic parameters of the drone camera;
[0011] The image set includes three or more images taken by drones, each image containing power lines, water surfaces, and rigid bodies;
[0012] The coarse pose of the UAV includes an initial 3×3 rotation matrix and an initial 3×1 translation vector;
[0013] Construct a world coordinate system, which is a custom-defined local spatial coordinate system;
[0014] Obtain the spatial position of the rigid body in the world coordinate system and mark this spatial position as the initial rigid body three-dimensional coordinates.
[0015] Furthermore, an objective function to minimize the reprojection error is constructed. Based on the objective function, the coarse pose of the UAV and the initial rigid body 3D coordinates are optimized iteratively. The specific optimization method is as follows:
[0016] The optimized rotation matrix is obtained by optimizing the initial 3×3 rotation matrix of each image in the image set by minimizing the sum of the squares of the Euclidean distances between all rigid body pixel coordinates and their corresponding projected coordinates. The optimized translation vector is obtained by optimizing the initial 3×1 translation vector of each image. The optimized initial rigid body 3D coordinates are then used to mark the optimized UAV coarse pose as a multi-view pose. The multi-view pose includes the optimized rotation matrix and the optimized translation vector. The optimized initial rigid body 3D coordinates are then marked as rigid body 3D coordinates.
[0017] Furthermore, based on the multi-view pose, the region where the water surface and land meet in the image corresponding to the multi-view pose in the image set is triangulated from multiple perspectives to obtain the true three-dimensional coordinate set of points in the region where the water surface and land meet. This true three-dimensional coordinate set is marked as the three-dimensional point set of the water-land intersection line, and the number of points in the three-dimensional point set of the water-land intersection line is not less than five.
[0018] Furthermore, the water surface plane is obtained by fitting the three-dimensional point set of the water-land intersection line using the singular value decomposition method, as follows:
[0019] The equation of the water surface plane is n·X+d=0, where n is the unit normal vector of the water surface plane, X is the mean of the three-dimensional point set of the water-land intersection line, and d is the directed distance from the water surface plane to the origin of the world coordinate system.
[0020] The mean of the three-dimensional point set of the water-land intersection line is the mean obtained by summing all the points in the three-dimensional point set of the water-land intersection line and dividing by the total number of points.
[0021] Construct a matrix M, where each element is each point in the three-dimensional point set of the water-land intersection line minus the mean of the three-dimensional point set of the water-land intersection line. The right singular vector corresponding to the smallest singular value obtained after singular value decomposition of matrix M is the unit normal vector n of the water surface plane.
[0022] The directed distance d from the water surface plane to the origin of the world coordinate system is obtained by taking the negative of the dot product of the unit normal vector n of the water surface plane and the mean of the three-dimensional point set of the water-land intersection line.
[0023] Further, determine tower body A and tower body B connected to tower body A by power lines, obtain the rigid body three-dimensional coordinates of the rigid body of tower body A and the rigid body three-dimensional coordinates of the rigid body of tower body B, and mark the rigid body three-dimensional coordinates of the two tower bodies as the first rigid body three-dimensional coordinates and the second rigid body three-dimensional coordinates, respectively.
[0024] A straight line is determined by connecting the two points of the first rigid body's three-dimensional coordinates and the second rigid body's three-dimensional coordinates. A plane spanning the vertical is constructed with this straight line as the x-axis and the direction of gravity g as the y-axis.
[0025] Furthermore, within the vertical plane, the height z(s) of the conductor is calculated using the catenary formula with the parametric coordinates s along the chord direction from the first rigid body's three-dimensional coordinates to the second rigid body's three-dimensional coordinates. The specific calculation method is as follows:
[0026] z(s) = a·cosh(s / a - s0 / a) + c;
[0027] Where z(s) is the height of the power conductor at parameter coordinate s, the range of parameter coordinate s is [0, B], where B is the chord length between the first rigid body three-dimensional coordinate and the second rigid body three-dimensional coordinate, a is the shape scale parameter, s0 is the mid-span position parameter, s0 = B / 2, c is the vertical translation parameter, and cosh(·) is the hyperbolic cosine function.
[0028] Obtain the height z(s0) of the conductor at the parameter coordinate s=s0. Since s=s0, the height z(s0) of the conductor at the mid-span position parameter s0 is a+c. Substitute the three-dimensional coordinates of the first rigid body, the three-dimensional coordinates of the second rigid body, and the height z(s0) of the conductor at the parameter coordinate s0 into the catenary formula, and then solve them together to obtain the unique values of the shape scale parameter a and the vertical translation parameter c. Then map the parameter coordinates s and the height z(s) to the world coordinate system to obtain the three-dimensional coordinates of the actual power conductor. Fit the three-dimensional curve of the power conductor based on the three-dimensional coordinates of the actual power conductor.
[0029] Furthermore, the three-dimensional curve of the power conductor is combined with multi-view pose and camera intrinsic parameters and projected onto each image in the image set through a perspective projection function to obtain a two-dimensional projection curve. For each sampling point of the two-dimensional projection curve, the normal direction of the two-dimensional projection curve is calculated. Then, along the normal direction of each sampling point, the brightness gradient of the image pixel is extracted, and the pixel with the largest brightness gradient is retained. All the retained pixels are combined into a candidate pixel set.
[0030] The following is a mirror transformation of the three-dimensional curve of the power conductor based on the water surface plane:
[0031] Obtain the three-dimensional points of the three-dimensional curve of the power conductor. Subtract 2 × (the sum of the product of the unit normal vector n of the water surface and the three-dimensional point plus the directed distance d) × the unit normal vector n of the water surface from the three-dimensional point. The resulting point is the three-dimensional point of the reflection of the power conductor in the water. Fit all the three-dimensional points of the reflection to form the three-dimensional curve of the reflection.
[0032] By combining the reflection 3D curve with multi-view pose and camera intrinsic parameters, the reflection 3D curve is projected onto each image in the image set captured by the UAV inspection through a perspective projection function, resulting in a 2D mirror projection curve of the real power line curve in the image.
[0033] Furthermore, for each pixel in the candidate pixel set, the minimum Euclidean distance from the pixel to the two-dimensional projection curve is calculated to obtain the true curve residual, and the minimum Euclidean distance from the pixel to the two-dimensional mirror projection curve is calculated to obtain the mirror curve residual.
[0034] For each pixel in the candidate pixel set, compare the magnitude of the real curve residual and the mirror curve residual corresponding to each pixel, as follows:
[0035] If the real curve residual is less than the mirror curve residual, the pixel is determined to be a real conductor pixel; if the real curve residual is greater than or equal to the mirror curve residual, the pixel is determined to be a reflection pixel.
[0036] Retain the real wire pixels and combine all the real wire pixels into a set of real power wire pixels;
[0037] The minimum clearance value is calculated based on the pixel set of real power conductors.
[0038] Furthermore, the 3D curve of the power conductor is calibrated using bundle adjustment based on the actual power conductor pixel set, as follows:
[0039] By combining the real power conductor pixel set with multi-view pose and camera intrinsic parameters, each pixel in the real power conductor pixel set is back-projected into a spatial ray. The shape scale parameter a, mid-span position parameter s0, and vertical translation parameter c of the catenary formula are iteratively optimized using the Levenberg-Marquardt algorithm. Then, based on the optimized catenary formula, the sum of the distances from the points on the 3D curve of the power conductor to the corresponding spatial ray is minimized to obtain the optimized 3D curve of the power conductor. The optimized 3D curve of the power conductor is marked as the real 3D curve of the power conductor.
[0040] Calculate the minimum clearance between the power conductor and the water surface at the mid-span position parameter s0, as follows:
[0041] After substituting the mid-span position parameter s0 into the three-dimensional curve of the actual power conductor, the three-dimensional coordinates at the mid-span position parameter s0 are obtained. The equation of the water surface plane is n·X+d=0. The distance from the three-dimensional coordinates at the mid-span position parameter s0 to the water surface plane is calculated according to the formula for the distance from a point to a plane, and this distance is marked as the minimum clearance value.
[0042] Compared with existing technologies, it has the following advantages:
[0043] This proposed method for rapid identification of power line hazards, which integrates UAV inspection and 3D modeling, addresses the shortcomings of existing technologies in inspecting power lines across water, such as the susceptibility of pose optimization to reflection interference, difficulty in quantitatively describing water surface morphology, deviation of conductor modeling from physical laws, reflection misjudgment, and large deviations in clearance calculation. Through multi-stage technological innovation, this method achieves comprehensive improvement. Regarding the construction of a reliable coordinate framework, unlike existing technologies that rely on a global geodetic coordinate system leading to insufficient transformation accuracy or incorporate unstable factors such as mid-span reflections, this method uses only the rigid bodies of the towers on both banks as constraints to construct an objective function that minimizes reprojection errors. It iteratively optimizes the coarse pose of the UAV and the 3D coordinates of the rigid bodies, improving pose accuracy from meter-level to centimeter-level photogrammetric accuracy. Simultaneously, it establishes precise anchor points on both banks of the rigid bodies, effectively avoiding the contamination of the initial pose by mid-span reflections and providing a stable and unbiased spatial reference for subsequent stages.
[0044] In water surface plane fitting, existing technologies are difficult to accurately define water surface boundaries. This solution obtains the three-dimensional point set of the water-land intersection line by triangulation of the water-land boundary area based on multi-view pose, and then obtains the water surface plane by fitting it through singular value decomposition. This transforms the water surface from a visually blurred area into a geometric object that can be quantitatively calculated, providing a deterministic mathematical basis for reflection modeling based on reflection geometry and solving the problem that the water surface shape cannot be accurately described.
[0045] In constructing the three-dimensional curve of a real power conductor, this scheme uses the three-dimensional coordinates of rigid bodies on both banks to construct a vertical plane. The height of the conductor is calculated in the plane using the catenary formula, so that the shape of the conductor strictly follows the natural suspension law, thereby avoiding the interference of reflection on the conductor modeling from the root and laying the geometric benchmark for the real conductor.
[0046] In conductor reflection stripping, this solution first extracts a candidate pixel set, then performs a mirror transformation on the real conductor's three-dimensional curve based on the water surface reflection pattern to obtain the reflected three-dimensional curve. By comparing the residuals from the candidate pixels to the two-dimensional projection curve and the two-dimensional mirror projection curve, pixels with smaller residuals in the real curve are retained, achieving pure geometry-driven accurate reflection stripping. This completely solves the problem of reflection misjudgment, significantly improves the accuracy and efficiency of hazard identification, and meets the needs of efficient operation and maintenance of power lines spanning water. Attached Figure Description
[0047] Figure 1 This is a schematic diagram of the method flow of the present invention; Detailed Implementation
[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0049] Please see Figure 1 This application provides a method for rapid identification of potential hazards in power lines that integrates drone inspection and 3D modeling;
[0050] The method specifically includes the following steps:
[0051] Step 1: During the inspection of the water-span section of a cross-river power line, the 3D modeling of UAV images is prone to instability due to water reflections and the lack of solid anchor points in the middle span. Therefore, it is necessary to construct a reliable coordinate framework. First, obtain a set of images taken by the UAV during the inspection. The set of images contains more than three images covering different shooting angles of the water-span section. For example, for a 400-meter water-span section of a cross-river power line, the UAV takes off from near tower A on one side of the line and flies towards the middle span area in a direction perpendicular to the line. It takes one image every 50 meters, for a total of 8 images. These 8 images are taken from different spatial positions and angles, such as the side view near tower A, the front view of the middle span, and the side view near tower B on the opposite bank. Each image contains part of the power conductor, the water surface, and the rigid bodies of the towers on both banks.
[0052] Obtain the camera intrinsic parameters of the drone camera. The camera intrinsic parameters are 3×3 matrices that describe the camera imaging geometry and include calibration parameters such as focal length.
[0053] The coarse pose of the UAV is obtained from the UAV IMU (Inertial Measurement Unit). The coarse pose of the UAV includes an initial 3×3 rotation matrix (describing the camera's orientation when taking the image) and an initial 3×1 translation vector (describing the camera's spatial position in the world coordinate system when taking the image. The world coordinate system is a local spatial coordinate system that is customized to uniformly describe the three-dimensional spatial position of elements such as the cross-river power line span, UAV, tower, water surface, and power lines). Each initial 3×3 rotation matrix and initial 3×1 translation vector in the coarse pose of the UAV is uniquely associated with the corresponding image in the image set at the time of taking the image.
[0054] The pixel coordinates of multiple rigid bodies (such as the main pole and tower head of the tower) on both banks are extracted from the image set captured by the UAV inspection using a local feature detection algorithm. These coordinates are marked as rigid body pixel coordinates, and the initial three-dimensional coordinates of the rigid bodies (the spatial position of the rigid bodies in the world coordinate system) are obtained. Specifically, the data obtained based on the determined rigid bodies of the towers on both banks can ensure that the data covers the key anchoring areas on both banks of the water span.
[0055] Using only rigid body feature points on both sides as constraints, an objective function is constructed to minimize reprojection error. The coarse pose of the UAV and the initial rigid body 3D coordinates are optimized iteratively from multiple perspectives. The objective function aims to minimize the sum of squares of the Euclidean distances between all rigid body pixel coordinates and the projected coordinates. The projected coordinates are obtained by projecting the initial rigid body 3D coordinates onto the corresponding image using the camera intrinsics and the UAV coarse pose of the current image through the standard perspective projection function. By minimizing the sum of squares of the Euclidean distances between all rigid body pixel coordinates and their corresponding projected coordinates, the initial 3×3 rotation matrix of each image in the image set is simultaneously optimized to obtain the optimized rotation matrix, the initial 3×1 translation vector of each image to obtain the optimized translation vector, and the optimized rigid body 3D coordinates. The optimized UAV coarse pose is named the multi-view pose, which includes the optimized rotation matrix and the optimized translation vector.
[0056] Specifically, by constructing an objective function to minimize reprojection error, the coarse pose and rigid body 3D coordinates of the UAV are iteratively optimized, improving the coarse pose of the UAV from the coarse positioning level (meter level) to the photogrammetric level (centimeter level). This provides a stable and accurate camera pose reference for subsequent steps. The rigid body 3D coordinates serve as anchor points on both banks of the water span. After optimization, the rigid body 3D coordinates are more accurate, providing a reliable spatial reference for subsequent steps. Unlike conventional multi-view pose optimization, which is prone to incorporating unstable factors such as the reflection of the middle span, this step optimizes only based on the rigid bodies on both banks, specifically avoiding the contamination of the initial pose by the reflection of the middle span of the water span. This lays a stable and unbiased spatial reference for subsequent removal of reflection interference.
[0057] Step 2: Based on multi-view pose, perform multi-view triangulation on the region where the water surface meets the land in the corresponding image of the image set to obtain a set of real 3D coordinates of multiple points in the region where the water surface meets the land. Mark this set of real 3D coordinates as the 3D point set of the water-land intersection line. Specifically, in order to meet the minimum number of points required for plane fitting, the number of points in the 3D point set of the water-land intersection line is no less than 5, and each point is a 3D spatial coordinate. The triangulation accuracy is ensured by multi-view pose, so that the obtained 3D point set of the water-land intersection line can accurately reflect the spatial distribution of the water-land boundary and provide high-quality point cloud data for subsequent fitting of the water surface plane.
[0058] The water surface plane is obtained by fitting the three-dimensional point set of the water-land intersection line using the singular value decomposition method. The equation of the water surface plane is n·X + d = 0, where n is the unit normal vector of the water surface plane, X is the mean of the three-dimensional point set of the water-land intersection line, and d is the directed distance from the water surface plane to the origin of the world coordinate system. Specifically, the mean of the three-dimensional point set of the water-land intersection line is the mean obtained by summing all the points in the three-dimensional point set and dividing by the total number of points. A matrix M is constructed, which is obtained by subtracting the mean of the three-dimensional point set of the water-land intersection line from each point in the three-dimensional point set. Singular value decomposition (SVD) is performed on matrix M. After decomposition, the right singular vector corresponding to the smallest singular value is the unit normal vector n of the water surface plane. The value of d is calculated by taking the dot product of the unit normal vector n and the mean of the three-dimensional point set of the water-land intersection line and taking the negative, i.e., d = -n·X. The directed distance d from the water surface plane to the origin of the world coordinate system is thus determined.
[0059] Specifically, singular value decomposition can accurately fit the normal and position of the water surface plane, ensuring the geometric accuracy of the water surface plane. By using the unit normal vector and directed distance of the water surface plane, the water surface reflection law, which originally reflected the geometric relationship between the reflection and the real conductor, is transformed into a quantitatively calculable reflection geometry object, providing a deterministic mathematical basis for subsequent reflection stripping based on reflection geometry.
[0060] Step 3: Determine the same rigid body (e.g., the tower heads of the two towers) for tower A and tower B on the opposite bank connected to tower A by a power line. Obtain the rigid body 3D coordinates of tower A and tower B. Label these two rigid body 3D coordinates as the first rigid body 3D coordinate and the second rigid body 3D coordinate, respectively. Connect the first rigid body 3D coordinate and the second rigid body 3D coordinate to determine a straight line. Use this straight line as the x-axis and the gravity direction g as the y-axis (the gravity direction g is a vertically downward unit vector opposite to the vertically upward Z-axis of the world coordinate system). Construct a plane spanning the vertical plane. Specifically, the power line is suspended only within the plane spanning the vertical plane and has no horizontal offset. By clarifying the spatial reference, endpoint constraints, and planar constraints of the power line modeling, complete prerequisites are provided for the subsequent application of the catenary formula.
[0061] Within the vertical plane, the height z(s) of the conductor is calculated using the catenary formula, with the parametric coordinates s (ranging from the first rigid body's three-dimensional coordinates to the second rigid body's three-dimensional coordinates, where B is the chord length between the first and second rigid body's three-dimensional coordinates, i.e., the length of the line connecting the two points across the vertical plane's x-axis) as the reference coordinates along the chord direction. The specific calculation method is as follows:
[0062] z(s) = a·cosh(s / a - s0 / a) + c;
[0063] Where z(s) is the vertical height of the electric conductor at parameter coordinate s, a is the shape scale parameter, a can determine the curvature of the catenary. The larger a is, the smoother the curve, s0 is the mid-span position parameter, s0 can determine the horizontal position of the lowest point of the catenary. If the heights at both ends are the same, s0 = B / 2, c is the vertical translation parameter, c can determine the overall vertical position of the catenary, and cosh(·) is the hyperbolic cosine function, which is the core mathematical expression of the catenary, ensuring that the curve conforms to the natural suspension characteristics of being high at both ends and drooping in the middle.
[0064] Obtain the height z(s0) of the conductor at parameter coordinate s0. Specifically, the height z(s0) of the conductor at parameter coordinate s0 is the sag height of the power conductor relative to its two ends. It is derived from engineering design requirements or specifications, or it can be determined by the clearance requirements of on-site inspection. The height z(s0) of the conductor at parameter coordinate s0 is a+c. Substitute the three-dimensional coordinates of the first rigid body, the three-dimensional coordinates of the second rigid body, and the height z(s0) of the conductor at parameter coordinate s0 = a+c into the catenary formula, and then solve them together to obtain the unique values of the shape scale parameter a and the vertical translation parameter c. Then map the parameter coordinate s and the height z(s) to the world coordinate system to obtain the three-dimensional coordinates of the actual power conductor. Based on multiple three-dimensional coordinates of the actual power conductor, a three-dimensional curve of the power conductor is obtained. Specifically, the three-dimensional curve of the power conductor is a three-dimensional curve that conforms to the actual suspension shape of the power conductor, which serves as the benchmark for subsequent judgment of the actual conductor and its reflection.
[0065] Specifically, to address the problem that the lack of physical anchor points in the middle span of the water crossing makes the shape of the power conductors susceptible to interference from reflections, a method is proposed to construct a true geometric benchmark for the conductors based on the physical catenary. This method does not rely on unstable anchor points in the image, but rather establishes a true shape model in advance based on the physical laws of the power conductors themselves and fixed points on both banks, thus providing an unbiased geometric basis for reflection stripping and clearance calculation from the root.
[0066] Step 4: Combine the 3D curve of the power conductor with multi-view pose and camera intrinsic parameters, and project the 3D curve of the power conductor onto each image in the image set captured by the UAV inspection using a perspective projection function. This yields the 2D projection curve of the real power conductor in the image. For each sampling point of the 2D projection curve, calculate the normal direction of the 2D projection curve (the direction perpendicular to the tangent of the curve). Then, along the normal direction of each sampling point, extract the brightness gradient of the image pixels (the rate of change of brightness between adjacent pixels; the larger the gradient, the more obvious the edge). In each normal direction, retain the pixel with the largest brightness gradient (the power conductor or its reflection appears as a high-contrast edge in the image, where the gradient is largest). Summarize all the retained pixels to obtain the candidate pixel set. The candidate pixel set contains the 2D pixel coordinates of the retained pixels, and the number is hundreds to thousands based on the size of the mid-span region.
[0067] Based on the reflection geometry of the water surface, a mirror transformation is performed on the three-dimensional curve of the power conductor. Specifically, the three-dimensional points of the power conductor's three-dimensional curve are obtained. The product of 2 × (the product of the unit normal vector n of the water surface and the three-dimensional point plus the directed distance d from the water surface to the origin of the world coordinate system) × the unit normal vector n of the water surface is subtracted from the three-dimensional point. The resulting point is the three-dimensional point of the power conductor's reflection in the water. All the three-dimensional points of the reflection are fitted into a three-dimensional curve of the reflection. Specifically, the three-dimensional shape of the reflection is accurately constructed through the three-dimensional curve of the reflection, transforming the originally invisible interference of the power conductor's reflection into a clearly calculable geometric object, providing a clear comparison benchmark for subsequent discrimination.
[0068] By combining the reflection 3D curve with multi-view pose and camera intrinsic parameters, the reflection 3D curve is projected onto each image in the image set captured by the UAV inspection through the perspective projection function, and the two-dimensional mirror projection curve of the real power line curve in the image is obtained.
[0069] For each pixel in the candidate pixel set, calculate two projection residuals. The first residual is the true curve residual, which is the minimum Euclidean distance from the pixel to the two-dimensional projection curve.
[0070] The second residual is the mirror curve residual, which is the minimum Euclidean distance from the pixel to the two-dimensional mirror projection curve. Specifically, the three-dimensional curve is transformed into a two-dimensional pixel curve on the image through standard perspective projection, and the qualitative judgment of whether the pixel belongs to the real image or the reflection is transformed into a quantitative process of comparing residual values.
[0071] For each pixel in the candidate pixel set, compare the size of the real curve residual and the mirror curve residual, and retain the category with the smaller residual. Specifically: if the real curve residual < the mirror curve residual, the pixel is determined to be a real conductor pixel; if the real curve residual ≥ the mirror curve residual, the pixel is determined to be a reflection pixel. Only real conductor pixels are retained, and all real conductor pixels are combined into a real power conductor pixel set. The real power conductor pixel set includes real conductor pixels. The real power conductor pixel set is the set of pixels that contain only real power conductor observations after removing reflection pixels. Specifically, a smaller residual means that the pixel matches the projection of the corresponding curve better. Only when the residual of the real conductor is smaller is the pixel a real power conductor pixel. If the reflection residual is smaller, the pixel belongs to the reflection and will be excluded.
[0072] Specifically, to address the problem of easily confusing the reflection of power conductors across water with the real image, an innovative discrimination method is proposed that uses reflection geometry to generate a mirror curve and compares it with the residual. Pure geometry-driven reflection stripping is achieved, which solves the problem of poor anti-interference of traditional methods and provides a key guarantee for the accuracy of clearance calculation.
[0073] Step 5: The 3D curve of the power conductor is calibrated using bundle adjustment based on the real power conductor pixel set. Specifically, the real power conductor pixel set is combined with multi-view pose and camera intrinsic parameters. Each pixel in the real power conductor pixel set is back-projected into a spatial ray (a straight line from the camera optical center to the pixel). While maintaining the constraint that the 3D curve of the power conductor conforms to the physical form of a catenary (satisfying the form z(s) = a·cosh(s / a - s0 / a) + c), the catenary parameters a, s0, and c are iteratively optimized using the Levenberg-Marquardt algorithm. The optimized parameters are then substituted into the catenary formula to obtain the optimized catenary formula. Based on the optimized catenary formula... By minimizing the sum of distances from points on the 3D curve of the power conductor to the corresponding spatial rays, the 2D projection curve obtained by projecting the 3D curve of the power conductor is closer to the pixels in the real power conductor pixel set. The calibrated 3D curve of the power conductor is then marked as the real 3D curve of the power conductor. Specifically, calibrating the 3D curve of the power conductor based on the real power conductor pixel set makes up for the difference between the theoretical curve based on the physical laws of the catenary and the real conductor image affected by the environment (wind sway, temperature, etc.) and shooting (angle, distortion, etc.). This ensures that the conductor shape used for subsequent clearance calculations not only conforms to physical laws but also accurately matches the real scene of on-site inspection, thus guaranteeing the accuracy of clearance analysis and hazard identification.
[0074] To calculate the minimum clearance between the actual power conductor and the water surface at the mid-span position parameter s0, the following steps are taken: Substitute the mid-span position parameter s0 into the three-dimensional curve of the actual power conductor to obtain the three-dimensional coordinates at the mid-span position parameter s0. The equation of the water surface is n·X+d=0. Calculate the distance from the three-dimensional coordinates at the mid-span position parameter s0 to the water surface using the formula for the distance from a point to a plane, and mark this distance as the minimum clearance value.
[0075] Specifically, by combining the geometric benchmark of the water surface with the benchmark of the actual conductor shape, the key minimum clearance value is accurately calculated using the point-to-plane distance formula, providing a core quantitative basis for hazard identification. The smaller the clearance, the higher the hazard risk of the power line. This solution can effectively eliminate the interference of water reflections of power conductors when drones inspect power lines in cross-river sections, quickly separate interference factors and obtain the minimum clearance, thereby identifying safety hazards. It not only effectively solves the problem of reflection interference in water-crossing sections, but also significantly improves the efficiency and accuracy of power line hazard identification.
[0076] On the other hand, differences in the horizontal height of the towers on both sides and the connection points of the power conductors can cause the theoretical minimum clearance of the power line to not be located at the mid-span position parameter s0. Therefore, it is necessary to select multiple sampling points at equal intervals along the parameter range s∈[0,B] of the conductor, and for each sampling point s iFirst, the three-dimensional coordinates of the sampling point are obtained based on the three-dimensional curve of the actual power conductor. Then, the clearance between the power conductor and the water surface is calculated using the distance formula from the point to the plane, resulting in multiple clearance values. By comparing these multiple clearance values, the minimum clearance value can be determined. In this way, the existence of potential hazards can be identified by comparing the minimum clearance value with the minimum clearance required by the engineering design or specifications.
[0077] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.
Claims
1. A method for rapid identification of hidden dangers in power lines that integrates drone inspection and 3D modeling, characterized in that... include: A set of images captured by a drone during inspection is obtained, and the coarse pose of the drone is obtained. The pixel coordinates of the rigid body are extracted from the image set through a local feature detection algorithm and marked as rigid body pixel coordinates. The initial rigid body 3D coordinates are obtained. The initial rigid body 3D coordinates and the drone coarse pose are optimized by minimizing the sum of squares of the Euclidean distance between the rigid body pixel coordinates and the projected coordinates. The rigid body 3D coordinates and multi-view pose are obtained. The projected coordinates are the pixel coordinates of the initial rigid body 3D coordinates projected onto the images in the image set. Based on multi-view pose, multi-view triangulation is performed on the image set to obtain a three-dimensional point set of the water-land intersection line. The singular value decomposition method is used to fit the three-dimensional point set of the water-land intersection line to obtain the water surface plane. Obtain the rigid body three-dimensional coordinates of two rigid bodies connected by a power conductor, and label them as the first rigid body three-dimensional coordinates and the second rigid body three-dimensional coordinates, respectively. Construct a vertical plane based on the first rigid body three-dimensional coordinates and the second rigid body three-dimensional coordinates. Calculate the actual three-dimensional coordinates of the power conductor based on the catenary formula based on the vertical plane. Fit the three-dimensional curve of the power conductor based on the actual three-dimensional coordinates of the power conductor. The three-dimensional curve of the power conductor is projected onto the image in the image set through a perspective projection function to obtain a two-dimensional projection curve. A candidate pixel set is obtained based on the two-dimensional projection curve. The three-dimensional curve of the power conductor is mirrored based on the water surface plane to obtain a reflected three-dimensional curve. The reflected three-dimensional curve is projected onto the image in the image set through a perspective projection function to obtain a two-dimensional mirror projection curve. The minimum Euclidean distance from each pixel in the candidate pixel set to the two-dimensional projection curve and the two-dimensional mirror projection curve is calculated to obtain the real curve residual and the mirror curve residual. The real conductor pixels are obtained based on the comparison results of the real curve residual and the mirror curve residual. For each pixel in the candidate pixel set, the true curve residual is obtained by calculating the minimum Euclidean distance from the pixel to the two-dimensional projection curve, and the mirror curve residual is obtained by calculating the minimum Euclidean distance from the pixel to the two-dimensional mirror projection curve. For each pixel in the candidate pixel set, compare the magnitude of the real curve residual and the mirror curve residual corresponding to each pixel, as follows: If the real curve residual is less than the mirror curve residual, the pixel is determined to be a real conductor pixel; if the real curve residual is greater than or equal to the mirror curve residual, the pixel is determined to be a reflection pixel. Retain the real wire pixels and combine all the real wire pixels into a set of real power wire pixels; The 3D curve of the power conductor is calibrated using bundle adjustment based on a real set of power conductor pixels, as detailed below: By combining the real power conductor pixel set with multi-view pose and camera intrinsic parameters, each pixel in the real power conductor pixel set is back-projected into a spatial ray. The shape scale parameter a, mid-span position parameter s0, and vertical translation parameter c of the catenary formula are iteratively optimized using the Levenberg-Marquardt algorithm. Then, based on the optimized catenary formula, the sum of the distances from the points on the 3D curve of the power conductor to the corresponding spatial ray is minimized to obtain the optimized 3D curve of the power conductor. The optimized 3D curve of the power conductor is marked as the real 3D curve of the power conductor. Calculate the minimum clearance between the power conductor and the water surface at the mid-span position parameter s0, as follows: After substituting the mid-span position parameter s0 into the three-dimensional curve of the actual power conductor, the three-dimensional coordinates at the mid-span position parameter s0 are obtained. The equation of the water surface plane is n·X+d=0. The distance from the three-dimensional coordinates at the mid-span position parameter s0 to the water surface plane is calculated according to the formula for the distance from a point to a plane, and this distance is marked as the minimum clearance value.
2. The method for rapid identification of power line hazards by integrating UAV inspection and 3D modeling as described in claim 1, characterized in that, include: Obtain the camera's intrinsic parameters from the drone's camera; The image set includes three or more images taken by drones, each image containing power lines, water surfaces, and rigid bodies; The coarse pose of the UAV includes an initial 3×3 rotation matrix and an initial 3×1 translation vector; Construct a world coordinate system, which is a custom-defined local spatial coordinate system; Obtain the spatial position of the rigid body in the world coordinate system and mark this spatial position as the initial rigid body three-dimensional coordinates.
3. The method for rapid identification of power line hazards by integrating UAV inspection and 3D modeling as described in claim 2, characterized in that, include: A target function is constructed to minimize the reprojection error. Based on the target function, the coarse pose of the UAV and the initial rigid body 3D coordinates are optimized iteratively. The specific optimization method is as follows: The optimized rotation matrix is obtained by optimizing the initial 3×3 rotation matrix of each image in the image set by minimizing the sum of the squares of the Euclidean distances between all rigid body pixel coordinates and their corresponding projected coordinates. The optimized translation vector is obtained by optimizing the initial 3×1 translation vector of each image. The optimized initial rigid body 3D coordinates are then used to mark the optimized UAV coarse pose as a multi-view pose. The multi-view pose includes the optimized rotation matrix and the optimized translation vector. The optimized initial rigid body 3D coordinates are then marked as rigid body 3D coordinates.
4. The method for rapid identification of power line hazards by integrating UAV inspection and 3D modeling as described in claim 3, characterized in that, include: Based on multi-view pose, multi-view triangulation is performed on the region where the water surface and land meet in the image corresponding to the multi-view pose in the image set to obtain the true three-dimensional coordinate set of the points in the region where the water surface and land meet. This true three-dimensional coordinate set is marked as the three-dimensional point set of the water-land intersection line, and the number of points in the three-dimensional point set of the water-land intersection line is not less than five.
5. The method for rapid identification of power line hazards by integrating UAV inspection and 3D modeling as described in claim 4, characterized in that, include: The water surface plane is obtained by fitting the three-dimensional point set of the water-land intersection line using the singular value decomposition method, as follows: The equation of the water surface plane is n·X+d=0, where n is the unit normal vector of the water surface plane, X is the mean of the three-dimensional point set of the water-land intersection line, and d is the directed distance from the water surface plane to the origin of the world coordinate system. The mean of the three-dimensional point set of the water-land intersection line is the mean obtained by summing all the points in the three-dimensional point set of the water-land intersection line and dividing by the total number of points. Construct a matrix M, where each element is each point in the three-dimensional point set of the water-land intersection line minus the mean of the three-dimensional point set of the water-land intersection line. The right singular vector corresponding to the smallest singular value obtained after singular value decomposition of matrix M is the unit normal vector n of the water surface plane. The directed distance d from the water surface plane to the origin of the world coordinate system is obtained by taking the negative of the dot product of the unit normal vector n of the water surface plane and the mean of the three-dimensional point set of the water-land intersection line.
6. The method for rapid identification of power line hazards by integrating UAV inspection and 3D modeling as described in claim 5, characterized in that, include: Determine tower body A and tower body B connected to tower body A by power lines, obtain the rigid body three-dimensional coordinates of the rigid body of tower body A and the rigid body three-dimensional coordinates of the rigid body of tower body B, and label the rigid body three-dimensional coordinates of the two tower bodies as the first rigid body three-dimensional coordinates and the second rigid body three-dimensional coordinates, respectively. A straight line is determined by connecting the two points of the first rigid body's three-dimensional coordinates and the second rigid body's three-dimensional coordinates. A plane spanning the vertical is constructed with this straight line as the x-axis and the direction of gravity g as the y-axis.
7. The method for rapid identification of power line hazards by integrating UAV inspection and 3D modeling as described in claim 6, characterized in that, include: Within the vertical plane, the height z(s) of the conductor is calculated using the catenary formula, with the parametric coordinates s along the chord direction from the first rigid body's three-dimensional coordinates to the second rigid body's three-dimensional coordinates. The specific calculation method is as follows: z(s) = a·cosh(s / a - s0 / a) + c; Where z(s) is the height of the power conductor at parameter coordinate s, the range of parameter coordinate s is [0, B], where B is the chord length between the first rigid body three-dimensional coordinate and the second rigid body three-dimensional coordinate, a is the shape scale parameter, s0 is the mid-span position parameter, s0 = B / 2, c is the vertical translation parameter, and cosh(·) is the hyperbolic cosine function. Obtain the height z(s0) of the conductor at the parameter coordinate s=s0. Since s=s0, the height z(s0) of the conductor at the mid-span position parameter s0 is a+c. Substitute the three-dimensional coordinates of the first rigid body, the three-dimensional coordinates of the second rigid body, and the height z(s0) of the conductor at the parameter coordinate s0 into the catenary formula, and then solve them together to obtain the unique values of the shape scale parameter a and the vertical translation parameter c. Then map the parameter coordinates s and the height z(s) to the world coordinate system to obtain the three-dimensional coordinates of the actual power conductor. Fit the three-dimensional curve of the power conductor based on the three-dimensional coordinates of the actual power conductor.
8. The method for rapid identification of power line hazards by integrating UAV inspection and 3D modeling as described in claim 7, characterized in that, include: The three-dimensional curve of the power conductor is combined with multi-view pose and camera intrinsic parameters and projected onto each image in the image set through the perspective projection function to obtain a two-dimensional projection curve. For each sampling point of the two-dimensional projection curve, the normal direction of the two-dimensional projection curve is calculated. Then, along the normal direction of each sampling point, the brightness gradient of the image pixel is extracted and the pixel with the largest brightness gradient is retained. All the retained pixels are combined into a candidate pixel set. The following is a mirror transformation of the three-dimensional curve of the power conductor based on the water surface plane: Obtain the three-dimensional points of the three-dimensional curve of the power conductor. Subtract 2 × (the sum of the product of the unit normal vector n of the water surface and the three-dimensional point plus the directed distance d) × the unit normal vector n of the water surface from the three-dimensional point. The resulting point is the three-dimensional point of the reflection of the power conductor in the water. Fit all the three-dimensional points of the reflection to form the three-dimensional curve of the reflection. By combining the reflection 3D curve with multi-view pose and camera intrinsic parameters, the reflection 3D curve is projected onto each image in the image set captured by the UAV inspection through a perspective projection function, resulting in a 2D mirror projection curve of the real power line curve in the image.