Unmanned aerial vehicle route collision detection method based on automatic identification and measurement of power transmission line tower
By acquiring images of utility poles and constructing a pole-wire model before takeoff, the risk of collision when identifying small wires during UAV inspections is resolved, enabling efficient and safe route planning and early warning, while reducing computing and storage requirements.
Patent Information
- Application Number
- CN202511119685.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-11
- Publication Date
- 2025-11-25
AI Technical Summary
Existing drone inspection systems pose a risk of collision when identifying thin electrical wires, and real-time obstacle avoidance systems require significant computing resources and high-performance storage devices, affecting data accuracy and security.
By pre-capturing images of utility poles before takeoff, using target detection technology to identify the bounding boxes of the utility poles, calculating the spatial coordinates of the pole's apex and base, constructing a utility pole-power line model, planning a safe flight path, and using plane equations to determine whether the UAV's path crosses the power line, an early warning message is issued.
It improved the efficiency of flight path planning, reduced the probability of collisions, reduced the requirements for computing and storage devices, lowered hardware costs, and enhanced flight range.
Smart Images
Figure CN121010906A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of computer vision recognition, in particular to a UAV flight path collision detection method based on automatic identification and measurement of power transmission towers. BACKGROUND
[0002] In recent years, the development of State Grid construction has entered a stage of rapid development, and the safety inspection of power grids has attracted more and more attention. The growing power grid construction has made the traditional manual inspection mode unable to meet the strategic development needs of lean management, cost reduction and efficiency improvement of power grid inspection.
[0003] Currently, unmanned aerial vehicles are commonly used for tower inspection, but this method requires a flight hand to manually plan the flight point under the tower, which not only takes a long time for manual flight path planning, but also the real-time obstacle avoidance system of the unmanned aerial vehicle cannot identify small power lines, which can easily cause a collision accident. In addition, by using the real-time obstacle avoidance method, the amount of point cloud data used is very large, and processing such data requires a large amount of computing resources and high-performance storage devices. In addition, the accuracy of point cloud data is limited by factors such as sensor resolution and distance, which further affects the accuracy of the data in complex scenarios. In the complex inspection environment, there is a high risk of collision with power lines. SUMMARY
[0004] To overcome the existing technical problems, the present application provides a UAV flight path collision detection method based on automatic identification and measurement of power transmission towers, which determines the safe flight path in advance before takeoff, without real-time calculation in the air, and is more secure.
[0005] The present application adopts the following technical solutions.
[0006] The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers comprises the following steps:
[0007] Collecting multiple images of power poles in different environments;
[0008] Identifying the bounding box diagram of the power pole in each power pole image through target detection technology, calculating the two-dimensional coordinates of the power pole top point and the power pole bottom in each power pole image, and back-projecting the two-dimensional coordinates of the power pole top point and the power pole bottom to the space ray, and calculating the space coordinates of the power pole top point and the power pole bottom according to the space ray;
[0009] Extracting multiple key points on the power line of each power pole image, and calculating the final direction vector V final of the power line in the world coordinate system according to the key points;
[0010] According to the space coordinates of the two power poles and the power line direction vector V finalJudge whether two poles are connected by wires, if yes, construct the plane equation of the two poles, substitute the UAV planning path into the plane equation, judge whether the UAV crosses the plane where the wires are located and is located below the wires, if yes, issue a warning information.
[0011] As a further improvement of the present application, the specific steps of collecting multiple pole images under different environments are as follows: using a UAV to collect data under different lighting conditions and labeling.
[0012] As a further improvement of the present application, the specific steps of identifying the bounding box map of the pole in each pole image through target detection technology include: extracting feature maps of different levels of the pole image through a multi-scale convolution layer, fusing the feature maps of different levels to obtain a fused feature map, generating a candidate region by analyzing the fused feature map through an RPN network, mapping the candidate region to the fused feature map through ROI pooling and unifying the size of the candidate region, classifying the candidate region to determine whether it is a pole and optimizing the bounding box to accurately enclose the pole, thereby obtaining the bounding box map.
[0013] As a further improvement of the present application, before identifying the bounding box map of the pole in each pole image through target detection technology, the steps of constructing a Faster R-CNN model with target detection technology are further included.
[0014] The Faster R-CNN model is provided with a multi-task loss function L for improving detection accuracy, for each candidate region, the classifier will output the probability of the pole category, and calculate the cross-entropy loss function between it and the true label ′
[0015] where p i is the true label, p i is the probability predicted by the model, N cls is the number of samples.
[0016] For each candidate region, the smooth L1 loss function L reg ′
[0017] where t i is the true bounding box coordinate, t i is the predicted bounding box coordinate, N reg is the number of regression samples.
[0018] The multi-task loss function L = L cls + λL reg , where λ is an adjustable weight parameter that balances the classification loss and the regression loss.
[0019] As a further improvement of the present invention, the specific steps for calculating the two-dimensional coordinates of the pole's apex and base in each pole image include: calculating the two-dimensional coordinates of the pole's apex. Where x min and x max These are the x-coordinates and y-coordinates of the top left and top right corners of the boundary diagram. max It is the y-coordinate of the upper edge of the boundary diagram;
[0020] Calculate the two-dimensional coordinates of the base of the utility pole Where y min It is the y-coordinate of the lower edge of the boundary diagram;
[0021] The specific steps for back-projecting the two-dimensional coordinates of the top and bottom of the utility pole onto a space ray include:
[0022] Using the camera's intrinsic parameter matrix K and extrinsic parameter matrix [R|T], the two-dimensional coordinates of the pole's apex and base are back-projected onto a spatial ray. The expression of the spatial ray in the world coordinate system is: r (i) (s)=R i ·K -1 p (i) +T i Where i is the i-th viewpoint on the same utility pole, s is the distance factor from a point on the ray to the optical center of the camera, and K -1 p (i) It converts image coordinates into direction vectors in the camera coordinate system, R i It is the camera's rotation matrix, T i It is the translation vector of the camera, thus obtaining multiple spatial rays with their intersection points at the top of the utility pole and multiple spatial rays with their intersection points at the bottom of the utility pole.
[0023] As a further improvement of the present invention, the specific steps for calculating the spatial coordinates of the top and bottom of the utility pole based on spatial rays include:
[0024] There are three spatial rays intersecting at the top and bottom of the utility pole, respectively. Solving the equations yields the spatial coordinates P of the top of the utility pole. top3 (x p y p , z p ) and the spatial coordinates P of the bottom of the utility pole bottom3 (x p y p , z p );
[0025] Where t1, t2, t3, t4, t5, and t6 are parameters, d1, d2, d3, d4, d5, and d6 are direction vectors, and Q1, Q2, and Q3 are viewpoint coordinates.
[0026] As a further improvement of the present invention, the specific steps for extracting multiple key points on the wires in each utility pole image include: segmenting the wires in the utility pole image using Mask R-CNN instance segmentation technology, and extracting multiple key points on the central axis of the wires based on the segmentation results.
[0027] Calculate the final direction vector V of the wire in the world coordinate system based on the key points. final The specific steps include: using the camera's intrinsic parameter matrix K, back-projecting the image coordinates of two key points into direction vectors in the camera coordinate system, d1 = in and These are the homogeneous coordinates of the two key points;
[0028] Using the camera's extrinsic parameter matrix [R|T], the direction vector in the camera coordinate system is transformed to the direction vector in the world coordinate system, V. j =R·d j +T, j∈{1,2}, where R is the camera rotation matrix and T is the camera translation vector;
[0029] Calculate the vector difference between the two world coordinate systems to obtain the initial direction vector V = V2 - V1 of the power line in the image of the utility pole. Repeat this step to calculate the initial direction vector V of the power lines in all utility pole images corresponding to the same power line. Finally, calculate the final direction vector of the power line in the world coordinate system. Where i is the i-th viewpoint on the same wire.
[0030] As a further improvement to the present invention, based on the spatial coordinates of the two utility poles and the wire direction vector V... final The specific steps to determine whether two utility poles are connected by power lines include: calculating the direction vector V of the utility poles based on the spatial coordinates of the vertices of the two poles. 12 =P2-P1=(x2-x1, y2-y1, z2-z1), calculate the final direction vector V of the wire. final With respect to the direction vector V of the utility pole 12 The included angle θ between them If cosθ is close to 1 or close to -1, then the two utility poles are considered to be connected by a wire.
[0031] As a further improvement of the present invention, the plane equation formula is: n1X+n2Y+n3Z+n4=0;
[0032] The specific steps for constructing the planar equations of these two utility poles include: extracting the spatial coordinates P1(x1, y1, z1) and P2(x2, y2, z2) of the pole vertices corresponding to these two utility poles, and the spatial coordinates P3(x3, y3, z3) and P4(x4, y4, z4) of the pole bases; and calculating the vector difference between the two pole vertices. The vector difference between the vertex and the base of one of the utility poles The cross product of the differences of two vectors yields the plane normal vector n. Substitute the spatial coordinates P4(x4, y4, z4) into the plane equation to solve for the constant term n4 = -(n1x4 + n2y4 + n3z4). Substitute the calculated n1, n2, n3, and n4 into the plane equation to construct the plane equations for the two utility poles.
[0033] As a further improvement of the present invention, the specific steps for determining whether the UAV has crossed the plane containing the power line and is located below the power line by substituting the planned path of the UAV into the plane equation include: arbitrarily selecting two endpoints E1 = (x... e1 y e1 , z e1 ) and E2=(x e2 y e2 , z e2 Substituting into the plane equation, if f(E1)·f(E2)≦0, then it is assumed that the path will pass through the plane containing the wire. Calculate the direction vector from E1 to E2. Substitute the points into the plane equation to calculate the intersection point I. Calculate the sum of the areas of the triangles formed by connecting intersection point I with P1P2, intersection point I with P1P3, intersection point I with P3P4, and intersection point I with P2P4. If the sum of the triangle areas is equal to the area of the quadrilateral connected by P1, P2, P3, and P4, then the UAV's planned path is considered to pass through the plane containing the power line and be located below the power line.
[0034] The beneficial effects of this invention are as follows: This collision detection method allows for the pre-determining of safe flight paths before takeoff, eliminating the need for real-time in-flight calculations. This not only improves flight path planning efficiency but also avoids the manual adjustments required by pilots to each waypoint, shortening mission preparation time and increasing overall inspection efficiency. It effectively compensates for the blind spot limitations of existing UAV obstacle avoidance systems due to sensor resolution or distance constraints, significantly reducing the probability of collisions during flight. Furthermore, since it eliminates the need for real-time processing of large-scale point cloud data for obstacle avoidance judgments during flight, it reduces the demands on the computing power and storage capacity of airborne or ground equipment. This allows for efficient flight path analysis even on ordinary performance equipment, lowering hardware costs and technical barriers, and enhancing endurance. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 This is a flowchart illustrating the present invention. Detailed Implementation
[0037] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product.
[0038] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings. The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0039] A method for UAV flight path collision detection based on automatic identification and measurement of power transmission towers includes the following steps:
[0040] S10. Collect multiple images of utility poles under different environments;
[0041] The specific steps are as follows: Data is collected and labeled using a drone under different lighting conditions. These different lighting conditions can be sunny, cloudy, or rainy, and the images are taken from different viewpoints. It is important to note that in one embodiment of this invention, for the same power line and the same utility pole, there are three different viewpoint images of the utility pole. The drone image collection process involves capturing images of all utility poles and power lines, and then labeling them accordingly to facilitate subsequent determination of whether they belong to the same utility pole.
[0042] S20. Identify the bounding box diagram of the utility pole in each utility pole image using target detection technology, calculate the two-dimensional coordinates of the pole's top and bottom in each utility pole image, back-project the two-dimensional coordinates of the pole's top and bottom onto a spatial ray, and calculate the spatial coordinates of the pole's top and bottom based on the spatial ray.
[0043] Infrastructure inspection sites often have numerous and complexly intertwined power lines. During detailed inspections, the flight altitude is low, and conventional obstacle avoidance methods pose a high risk of collisions with power lines. By pre-collecting and analyzing images of power poles, the spatial coordinates of the pole's apex and base can be obtained. Given a known and standardized pole shape, a power pole-power line model can be established. This model can then be directly used when the drone plans its path, clearly identifying the risk of collisions. If the risk is high, the pilot can be alerted to further optimize the flight path and reduce the probability of a collision.
[0044] The specific steps for identifying the bounding box map of each utility pole image using object detection technology include: extracting feature maps at different levels of the utility pole image through multi-scale convolutional layers; fusing the feature maps at different levels to obtain a fused feature map; analyzing the fused feature map using an RPN network to generate candidate regions; mapping the candidate regions onto the fused feature map and unifying the size of the candidate regions through ROI pooling; classifying the candidate regions to determine whether they are utility poles and optimizing the bounding boxes to accurately surround the utility poles, thereby obtaining the bounding box map.
[0045] Using multi-scale convolution with residual connections to extract feature maps not only avoids the gradient problem, allowing the network to learn deeper features, but also captures details at different levels in the image. For example, large-scale convolution can extract the overall shape features of the utility pole, while small-scale convolution can extract the detailed features of the pole's edges, ensuring feature diversity and thus improving detection accuracy.
[0046] As a further improvement of the present invention, before identifying the bounding box map of each utility pole image using object detection technology, the method further includes the step of: constructing a Faster R-CNN model with object detection technology;
[0047] The Faster R-CNN model employs a multi-task loss function L to improve detection accuracy. For each candidate region, the classifier outputs the probability of the utility pole category and calculates the cross-entropy loss function between it and the true label. ′
[0048] Where p i It's a real label, p i It is the probability predicted by the model, N cls It is the number of samples;
[0049] For each candidate region, calculate the smoothed L1 loss function Li between the predicted boundary coordinates and the actual labeled coordinates. reg
[0050] Where t iTrue bounding box coordinates, t i ′ It predicts the bounding box coordinates, N reg It is the number of regression samples;
[0051] Multi-task loss function L = L cls +λL reg , where λ is an adjustable weight parameter that balances classification loss and regression loss.
[0052] The cross-entropy loss function calculates the cross-entropy loss with the ground truth label, thus optimizing the Faster R-CNN model and improving the accuracy of subsequent utility pole category recognition. The smoothed L1 loss function measures the positional and size differences between the model's predicted bounding box and the ground truth bounding box. Combining these two loss functions ensures the model not only correctly classifies the target but also accurately locates it. The λ weight parameter can be adjusted according to different use cases; for example, λ can be increased to sacrifice some classification accuracy for better localization, and vice versa.
[0053] As a further improvement of the present invention, the specific steps for calculating the two-dimensional coordinates of the pole's apex and base in each pole image include: calculating the two-dimensional coordinates of the pole's apex. Where x min and x max These are the x-coordinates and y-coordinates of the top left and top right corners of the boundary diagram. max It is the y-coordinate of the upper edge of the boundary diagram;
[0054] Calculate the two-dimensional coordinates of the base of the utility pole Where y min It is the y-coordinate of the lower edge of the boundary diagram;
[0055] The specific steps for back-projecting the two-dimensional coordinates of the top and bottom of the utility pole onto a space ray include:
[0056] Using the camera's intrinsic parameter matrix K and extrinsic parameter matrix [R|T], the two-dimensional coordinates of the pole's apex and base are back-projected onto a spatial ray. The expression of the spatial ray in the world coordinate system is: r (i) (s)=R i ·K -1 p (i) +T i Where i is the i-th viewpoint on the same utility pole, s is the distance factor from a point on the ray to the optical center of the camera, and K -1 p (i) It converts image coordinates into direction vectors in the camera coordinate system, R i It is the camera's rotation matrix, Ti It is the translation vector of the camera, thus obtaining multiple spatial rays with their intersection points at the top of the utility pole and multiple spatial rays with their intersection points at the bottom of the utility pole.
[0057] Calculating the pole's apex and base using the midpoint allows for determining the pole's center position, improving the accuracy of the planar equations and avoiding significant errors in the calculated apex and base from different viewpoints. For example, if the pole's apex is determined using the top-left vertex of the boundary map, the apex position captured from the left and right viewpoints will differ by the entire pole's diameter, resulting in a large error that affects the accuracy of subsequent spatial coordinate determination using spatial rays. A spatial ray refers to the two-dimensional coordinates of the pole's apex and base in the pole image and the corresponding spatial ray emitted from the camera's optical center. Regarding the specific data for the s-distance factor, since this invention subsequently solves for it using three intersecting rays, calculating the intersection point of each ray does not require the specific value of the s-distance factor. Furthermore, due to image acquisition and measurement errors, the three rays will not perfectly intersect at a single point, but the s-values of the three rays can be solved by simultaneously establishing three ray equations. The intersection point of the optimal solutions for the three rays can then be found using the least squares method, allowing for the calculation of the corresponding s-value and ultimately the spatial coordinates.
[0058] As another embodiment of the present invention, the specific steps for calculating the spatial coordinates of the top and bottom of the utility pole based on spatial rays include:
[0059] There are three spatial rays intersecting at the top and bottom of the utility pole, respectively. Solving the equations yields the spatial coordinates P of the top of the utility pole. top3 (x p y p , z p ) and the spatial coordinates P of the bottom of the utility pole bottom3 (x p y p , z p );
[0060] Where t1, t2, t3, t4, t5, and t6 are parameters, d1, d2, d3, d4, d5, and d6 are direction vectors, and Q1, Q2, and Q3 are viewpoint coordinates.
[0061] The viewpoint coordinates are already labeled when the drone captures the image of the utility pole, and the direction vector is obtained during the conversion between the camera's intrinsic and extrinsic parameter matrices, requiring no further calculation. Each of the t parameters is represented by its optimal solution to achieve the spatial coordinates P. top3 and P bottom3The optimal solution is the one where the coordinates calculated by each equation are basically the same. Of course, if the rays have a common intersection point, then they have a unique t parameter value and there is no need to calculate the optimal solution.
[0062] S30. Extract multiple key points on the power line in each power pole image, and calculate the final direction vector V of the power line in the world coordinate system based on the key points. final ;
[0063] As a further improvement of the present invention, the specific steps for extracting multiple key points on the wires in each utility pole image include: segmenting the wires in the utility pole image using Mask R-CNN instance segmentation technology, and extracting multiple key points on the central axis of the wires based on the segmentation results.
[0064] Calculate the final direction vector V of the wire in the world coordinate system based on the key points. final The specific steps include: using the camera's intrinsic parameter matrix K, back-projecting the image coordinates of two key points into direction vectors in the camera coordinate system. in and These are the homogeneous coordinates of the two key points;
[0065] Using the camera's extrinsic parameter matrix [R|T], the direction vector in the camera coordinate system is transformed to the direction vector in the world coordinate system, V. j =R·d j +T, j∈{1,2}, where R is the camera rotation matrix and T is the camera translation vector;
[0066] Calculate the vector difference between the two world coordinate systems to obtain the initial direction vector V = V2 - V1 of the power line in the image of the utility pole. Repeat this step to calculate the initial direction vector V of the power lines in all utility pole images corresponding to the same power line. Finally, calculate the final direction vector of the power line in the world coordinate system. Where i is the i-th viewpoint on the same wire.
[0067] Since the same power line on a utility pole is almost always at the same height, key points can be represented using homogeneous coordinates, reducing computational resource consumption. Because power lines are typically long and thin and can be affected by ambient light, occlusion, and other factors, accurate directional information is difficult to obtain from a single viewpoint. This invention effectively reduces deviations caused by visual errors or local occlusion by repeatedly calculating the preliminary direction vectors from multiple viewpoints along the same power line and taking the average.
[0068] S40. Based on the spatial coordinates of the two utility poles and the direction vector V of the power line... finalDetermine if two utility poles are connected by power lines. If so, construct the plane equations for the two utility poles, substitute the planned path of the drone into the plane equations, and determine if the drone passes through the plane containing the power lines and is located below the power lines. If so, issue a warning message.
[0069] By constructing a plane equation based on the spatial coordinates of the utility pole and the direction vector of the power line, the planar position of the power line can be accurately described. Predicting in advance whether the planned path of a drone will cross the plane containing the power line and lie beneath it enables automatic detection and early warning of the drone's flight path, significantly reducing preparation time for manual operation.
[0070] As a further improvement to the present invention, based on the spatial coordinates of the two utility poles and the wire direction vector V... final The specific steps to determine whether two utility poles are connected by power lines include: calculating the direction vector V of the utility poles based on the spatial coordinates of the vertices of the two poles. 12 =P2-P1=(x2-x1, y2-y1, z2-z1), calculate the final direction vector V of the wire. final With respect to the direction vector V of the utility pole 12 The included angle θ between them If cosθ is close to 1 or close to -1, then the two utility poles are considered to be connected by a wire.
[0071] The method simplifies the calculation process by calculating the spatial coordinate difference between the vertices of two utility poles to obtain their direction vectors. This is then combined with the final direction vector of the power line to calculate the angle between them. This method automatically identifies power line connections, reducing manual analysis time. When cosθ is close to 1 or close to -1, the two vectors are considered to be either completely identical or completely opposite. In the connection between utility poles and power lines, completely identical and completely opposite connections are essentially the same, therefore, it can be assumed that the two utility poles are connected by power lines. Specifically, an angle within 10° is considered close. However, if the wind is strong in some areas, the power lines may sway significantly during drone photography, in which case the angle can be set within 30%. The drone will mark the location data during photography, and the placement range of utility poles follows national standards, such as a typical pole spacing of 30 to 50 meters. If the location data difference between two utility poles exceeds 50 meters, there is no need to calculate the angle to determine whether the two poles are connected by power lines.
[0072] As a further improvement of the present invention, the plane equation formula is: n1X+n2Y+n3Z+n4=0;
[0073] The specific steps for constructing the planar equations of these two utility poles include: extracting the spatial coordinates P1(x1, y1, z1) and P2(x2, y2, z2) of the pole vertices corresponding to these two utility poles, and the spatial coordinates P3(x3, y3, z3) and P4(x4, y4, z4) of the pole bases; and calculating the vector difference between the two pole vertices. The vector difference between the vertex and the base of one of the utility poles The cross product of the differences of two vectors yields the plane normal vector n. Substitute the spatial coordinates P4(x4, y4, z4) into the plane equation to solve for the constant term n4 = -(n1x4 + n2y4 + n3z4). Substitute the calculated n1, n2, n3, and n4 into the plane equation to construct the plane equations for the two utility poles.
[0074] Based on the plane equation, it is easy to determine whether the planned path of a drone crosses the plane containing the power line, and provides technical support for subsequent confirmation that the drone passes under the power line. This method realizes automatic modeling and judgment of the power line position. Regardless of the region or terrain, it can calculate and construct the plane equation of the power pole by capturing images of the pole, and has strong compatibility.
[0075] As a further improvement of the present invention, the specific steps for determining whether the UAV has crossed the plane containing the power line and is located below the power line by substituting the planned path of the UAV into the plane equation include: arbitrarily selecting two endpoints E1 = (x... e1 y e1 , z e1 ) and E2=(x e2 y e2 , z e2 Substituting into the plane equation, if f(E1)·f(E2)≦0, then it is assumed that the path will pass through the plane containing the wire. Calculate the direction vector from E1 to E2. Substitute the points into the plane equation to calculate the intersection point I. Calculate the sum of the areas of the triangles formed by connecting intersection point I with P1P2, intersection point I with P1P3, intersection point I with P3P4, and intersection point I with P2P4. If the sum of the triangle areas is equal to the area of the quadrilateral connected by P1, P2, P3, and P4, then the UAV's planned path is considered to pass through the plane containing the power line and be located below the power line.
[0076] If f(E1)·f(E2) < 0, it means that endpoints E1 and E2 are located on opposite sides of the plane, and the path must pass through the plane. If it equals 0, it means that at least one point is on the plane, and the path is also considered to pass through the plane. Furthermore, by calculating the area of the triangles connecting intersection point I to each vertex of the plane, we can determine whether intersection point I is within a quadrilateral region. If intersection point I is within a quadrilateral region, the sum of the triangle areas must equal the area of the quadrilateral. If the areas are not equal, the path may be on the power line or bypass the two power poles.
[0077] This method enables real-time monitoring and early warning. Once a safety hazard is detected in the drone's path, the system immediately issues an alarm, reminding operators to optimize the planned path. Based on plane equations and geometric calculations, this method accurately determines the positional relationship between the drone's path and power lines, avoiding potential misjudgments that might arise from relying solely on visual or sensor data. It improves detection accuracy while reducing the burden on sensors and real-time computing. Compared to traditional multi-point measurement and complex data processing methods, this method automatically analyzes the potential risks of the drone's planned path after calculating the spatial coordinate differences and vector cross products of only a few key points to obtain the plane equation. This offers significant advantages over traditional manual path planning. Furthermore, this method can identify safety hazards before the drone takes off, improving inspection efficiency and reducing the risk of collisions. By optimizing the planned path warned by the early warning information, a safe path for the drone is obtained.
[0078] The entire process of this invention is automated, except for the operation of the drone to capture images of utility poles and the input of the planned path. After inputting the utility pole images, a boundary map is automatically output. After outputting the boundary map, the image coordinates are recorded and automatically converted into spatial coordinates. After obtaining the spatial coordinates, the plane equations are automatically established. After inputting the planned path, the two endpoints of each segment of the planned path are automatically selected and substituted into the calculation. If the calculated result of the path is that it will pass through the plane where the power line is located, the system will automatically calculate whether the next step of the path will pass under the power line. If not, the path segment is a safe path. If not, a warning message is issued so that the operator can optimize the path. The endpoint calculation steps are repeated until a complete safe path is obtained. This automated process can greatly reduce the workload and significantly reduce manpower costs in large-scale and complex inspection work. Furthermore, the calculation process of this invention has a small computational load, enabling effective flight path warning function under conditions of low computational cost and low hardware cost.
[0079] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0080] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A method for UAV flight path collision detection based on automatic identification and measurement of power transmission towers, characterized in that, Includes the following steps: Multiple utility pole images were captured in different environments; The bounding box of the utility pole in each utility pole image is identified by target detection technology. The two-dimensional coordinates of the pole's top and bottom in each utility pole image are calculated. The two-dimensional coordinates of the pole's top and bottom are back-projected onto a spatial ray. The spatial coordinates of the pole's top and bottom are then calculated based on the spatial ray. Multiple key points are extracted from the power line in each power pole image, and the final direction vector V of the power line in the world coordinate system is calculated based on the key points. final ; Based on the spatial coordinates of the two utility poles and the direction vector V of the power line final Determine if two utility poles are connected by power lines. If so, construct the plane equations for the two utility poles, substitute the planned path of the drone into the plane equations, and determine if the drone passes through the plane containing the power lines and is located below the power lines. If so, issue a warning message.
2. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 1, characterized in that, The specific steps for collecting multiple utility pole images in different environments are as follows: use a drone to collect and label data under different lighting conditions.
3. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 1, characterized in that, The specific steps for identifying the bounding box map of each utility pole image using object detection technology include: extracting feature maps at different levels of the utility pole image through multi-scale convolutional layers; fusing the feature maps at different levels to obtain a fused feature map; analyzing the fused feature map using an RPN network to generate candidate regions; mapping the candidate regions onto the fused feature map and unifying the size of the candidate regions through ROI pooling; classifying the candidate regions to determine whether they are utility poles and optimizing the bounding boxes to accurately surround the utility poles, thereby obtaining the bounding box map.
4. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 3, characterized in that, Before identifying the bounding box map of each utility pole image using object detection technology, the following step is included: constructing a Faster R-CNN model with object detection technology; The Faster R-CNN model employs a multi-task loss function L to improve detection accuracy. For each candidate region, the classifier outputs the probability of the utility pole category and calculates the cross-entropy loss function between it and the true label. ′ Where p i It's a real label, p i It is the probability predicted by the model, N cls It is the number of samples; For each candidate region, calculate the smoothed L1 loss function Li between the predicted boundary coordinates and the actual labeled coordinates. reg ′ Where t i True bounding box coordinates, t i It predicts the bounding box coordinates, N reg It is the number of regression samples; Multi-task loss function L = L cls +λL reg , where λ is an adjustable weight parameter that balances classification loss and regression loss.
5. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 1, characterized in that, The specific steps for calculating the two-dimensional coordinates of the pole's apex and base in each pole image include: calculating the two-dimensional coordinates of the pole's apex. Where x min and x max These are the x-coordinates and y-coordinates of the top left and top right corners of the boundary diagram. max It is the y-coordinate of the upper edge of the boundary diagram; Calculate the two-dimensional coordinates of the base of the utility pole Where y min It is the y-coordinate of the lower edge of the boundary diagram; The specific steps for back-projecting the two-dimensional coordinates of the top and bottom of the utility pole onto a space ray include: Using the camera's intrinsic parameter matrix K and extrinsic parameter matrix [R|T], the two-dimensional coordinates of the pole's apex and base are back-projected onto a spatial ray. The expression of the spatial ray in the world coordinate system is: r (i) (s)=R i ·K -1 p (i) +T i Where i is the i-th viewpoint on the same utility pole, s is the distance factor from a point on the ray to the optical center of the camera, and K -1 p (i) It converts image coordinates into direction vectors in the camera coordinate system, R i It is the camera's rotation matrix, T i It is the translation vector of the camera, thus obtaining multiple spatial rays with their intersection points at the top of the utility pole and multiple spatial rays with their intersection points at the bottom of the utility pole.
6. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 5, characterized in that, The specific steps for calculating the spatial coordinates of the top and bottom of a utility pole using spatial rays include: There are three spatial rays intersecting at the top and bottom of the utility pole, respectively. Solving the equations yields the spatial coordinates P of the top of the utility pole. top3 (x p y p , z p ) and the spatial coordinates P of the bottom of the utility pole bottom3 (x p y p , z p ); Where t1, t2, t3, t4, t5, and t6 are parameters, d1, d2, d3, d4, d5, and d6 are direction vectors, and Q1, Q2, and Q3 are viewpoint coordinates.
7. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 1, characterized in that, The specific steps for extracting multiple key points on the wires in each utility pole image include: segmenting the wires in the utility pole image using Mask R-CNN instance segmentation technology, and extracting multiple key points on the central axis of the wires based on the segmentation results; Calculate the final direction vector V of the wire in the world coordinate system based on the key points. final The specific steps include: using the camera's intrinsic parameter matrix K, back-projecting the image coordinates of two key points into direction vectors in the camera coordinate system. in and These are the homogeneous coordinates of the two key points; Using the camera's extrinsic parameter matrix [R|T], the direction vector in the camera coordinate system is transformed to the direction vector in the world coordinate system, V. j =R·d j +T, j∈{1,2}, where R is the camera rotation matrix and T is the camera translation vector; Calculate the vector difference between the two world coordinate systems to obtain the initial direction vector V = V2 - V1 of the power line in the image of the utility pole. Repeat this step to calculate the initial direction vector V of the power lines in all utility pole images corresponding to the same power line. Finally, calculate the final direction vector of the power line in the world coordinate system. Where i is the i-th viewpoint on the same wire.
8. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 1, characterized in that, Based on the spatial coordinates of the two utility poles and the direction vector V of the power line final The specific steps to determine whether two utility poles are connected by power lines include: calculating the direction vector V of the utility poles based on the spatial coordinates of the vertices of the two poles. 12 =P2-P1=(x2-x1, y2-y1, z2-z1), calculate the final direction vector V of the wire. final With respect to the direction vector V of the utility pole 12 The included angle θ between them If cosθ is close to 1 or close to -1, then the two utility poles are considered to be connected by a wire.
9. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 1, characterized in that, The equation of the plane is: n1X + n2Y + n3Z + n4 = 0; The specific steps for constructing the planar equations of these two utility poles include: extracting the spatial coordinates P1(x1, y1, z1) and P2(x2, y2, z2) of the pole vertices corresponding to these two utility poles, and the spatial coordinates P3(x3, y3, z3) and P4(x4, y4, z4) of the pole bases; and calculating the vector difference between the two pole vertices. The vector difference between the vertex and the base of one of the utility poles The cross product of the differences of two vectors yields the plane normal vector n. Substitute the spatial coordinates P4(x4, y4, z4) into the plane equation to solve for the constant term n4 = -(n1x4 + n2y4 + n3z4). Substitute the calculated n1, n2, n3, and n4 into the plane equation to construct the plane equations for the two utility poles.
10. The UAV flight path collision detection method based on automatic identification and measurement of power transmission towers according to claim 9, characterized in that, Substituting the drone's planned path into the plane equation, the specific steps to determine whether the drone crosses the plane containing the power line and is located below the power line include: arbitrarily selecting two endpoints of a certain planned path of the drone, E1 = (x... e1 y e1 , z e1 ) and E2=(x e2 y e2 , z e2 Substituting into the plane equation, if f(E1)·f(E2)≦0, then it is assumed that the path will pass through the plane containing the wire. Calculate the direction vector from E1 to E2. Substitute the points into the plane equation to calculate the intersection point I. Calculate the sum of the areas of the triangles formed by connecting intersection point I with P1P2, intersection point I with P1P3, intersection point I with P3P4, and intersection point I with P2P4. If the sum of the triangle areas is equal to the area of the quadrilateral connected by P1, P2, P3, and P4, then the UAV's planned path is considered to pass through the plane containing the power line and be located below the power line.