An unmanned aerial vehicle transmission line line-imitating flight control method, system and device

Abstract

The application discloses a kind of unmanned aerial vehicle transmission line imitative line flight control method, system and equipment, it is related to unmanned aerial vehicle autonomous inspection technical field, this method includes: obtaining transmission line inspection task, safety constraint and multi-source sensor data and carries out fusion positioning, determines the real-time pose information of unmanned aerial vehicle;Subsequently generate local point cloud map and carry out semantic classification to local point cloud map, obtain key target category and corresponding point cloud set;Afterwards, according to the point cloud set of key target category, key structure extraction is carried out, line reference structure and obstacle information are generated and global safety inspection route is planned;Finally, based on the dynamic obstacle information of real-time perception, local correction is carried out and unmanned aerial vehicle is controlled to carry out imitative line flight.The application improves the positioning robustness in complex environment by multi-sensor fusion positioning, improves the accuracy of line identification and obstacle distinction by semantic classification and key structure extraction, reduces the obstacle avoidance response time and reduces the manual takeover by local dynamic correction.

Description

Technical Field

[0001] This application relates to the field of autonomous inspection technology for unmanned aerial vehicles (UAVs), and in particular to a method, system, and equipment for UAV flight control of power transmission line imitation. Background Technology

[0002] With the continuous expansion of the scale of power transmission line operation and maintenance, especially in mountainous areas, crossing areas, and areas prone to tree obstructions, higher technical requirements are being placed on the automated line-following flight inspection of drones. When performing power transmission line inspection tasks, drones need to fly continuously between towers along the conductor or ground wire, and perceive the positions of conductors, towers, insulators, hardware, vegetation, buildings, and temporary obstacles in real time, generating smooth, continuous, and executable inspection tracks while ensuring a safe distance.

[0003] Currently, existing drone inspection methods mainly rely on manual remote control, preset waypoints, or single sensor-assisted positioning. Manual remote control is highly dependent on the operator's experience and skills, and struggles to maintain a stable alignment distance when the background of the conductor is complex, there are significant differences in tower elevation, or there is considerable wind disturbance. Preset waypoints are ill-suited to tree obstructions, temporary construction equipment, swaying overhead ground wires, and variations in tower structure. Single visual or single real-time kinematic (RTK) positioning methods are prone to instability or even loss of positioning under conditions of occlusion, weak texture, strong reflection, GNSS obstruction, or electromagnetic interference.

[0004] Although 3D lidar can provide high-precision spatial point cloud information within power transmission channels, it is still difficult to solve a series of problems such as extracting the slender structure of conductors, identifying key points of towers, distinguishing semantic obstacles, local dynamic obstacle avoidance, and ensuring flight control continuity if only simple point cloud mapping or obstacle avoidance based on distance thresholds is performed.

[0005] Therefore, how to provide a technical solution that can organically integrate three-dimensional environmental perception, semantic understanding, trajectory planning and flight control to enable UAVs to fly safely, continuously and autonomously along complex power transmission channels has become an urgent problem to be solved in this field. Summary of the Invention

[0006] The purpose of this application is to provide a method, system, and device for unmanned aerial vehicle (UAV) flight control for power transmission line simulation, which aims to improve the continuity, safety, and automation of UAV flight in complex power transmission channels.

[0007] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for controlling the flight of an unmanned aerial vehicle (UAV) along a power transmission line, comprising the following steps: The system acquires data on power transmission line inspection tasks, safety constraints, and multi-source sensor data, including 3D lidar point clouds, global navigation satellite system data, and inertial measurement data.

[0008] Based on the multi-source sensor data, the real-time pose information of the UAV is determined through fusion positioning.

[0009] A local point cloud map is generated based on the three-dimensional lidar point cloud and the real-time pose information.

[0010] Semantic classification is performed on the local point cloud map to obtain key target categories and corresponding point cloud sets; the key target categories include conductors, towers, crossarms, insulators, and obstacles.

[0011] Key structures are extracted from the point cloud set of the key target categories to generate route reference structures and obstacle information.

[0012] Based on the line reference structure, the safety constraints, and the obstacle information, a global safety inspection route is planned.

[0013] Based on real-time perceived dynamic obstacle information, the global safety inspection route is locally corrected and flight control commands are generated to control the UAV to perform line-following flight.

[0014] Optionally, based on the multi-source sensor data, fusion positioning is performed to determine the real-time pose information of the UAV, specifically including the following steps: Based on the inertial measurement data, state prediction is performed to obtain the predicted state information of the UAV.

[0015] The predicted state information is corrected by observing and updating the data from the Global Navigation Satellite System and / or the three-dimensional lidar point cloud to obtain the real-time pose information of the UAV.

[0016] Optionally, a local point cloud map is generated based on the three-dimensional lidar point cloud and the real-time pose information, specifically including the following steps: Based on the real-time pose information, the 3D LiDAR point cloud is transformed from the LiDAR coordinate system to the world coordinate system to obtain a world coordinate system point cloud. The real-time pose information is used to construct the pose transformation matrix from the LiDAR coordinate system to the world coordinate system; the pose transformation matrix includes rotation and translation components.

[0017] The local point cloud map is obtained by preprocessing the world coordinate system point cloud; the preprocessing includes at least one of time synchronization, distortion compensation, voxel downsampling, outlier removal, and ground filtering. The local point cloud map is a point cloud map within a spatial region centered on the current position of the UAV and with a predetermined sensing range as its radius.

[0018] Optionally, semantic classification is performed on the local point cloud map to obtain key target categories and corresponding point cloud sets, including the following steps: For each point in the local point cloud map, a multidimensional feature vector is constructed; the multidimensional feature vector includes at least one of the point's three-dimensional coordinates, reflection intensity, local normal vector, local curvature, and neighborhood point density.

[0019] The multidimensional feature vector is input into a pre-trained point cloud classification network, which outputs the probability that each point belongs to each key target category.

[0020] The category of the key target corresponding to the highest probability is used as the category label of each point, thus obtaining the point cloud set corresponding to each key target category.

[0021] Optionally, the line reference structure includes the spatial curves of each conductor, the line centerline, the center point of the tower bottom, the main direction of the crossarm, and the center point of the insulator; the obstacle information includes obstacle bounding boxes; key structures are extracted based on the point cloud set of the key target categories to generate the line reference structure and obstacle information, including the following steps: Cluster the point cloud set of the aforementioned conductor type to separate the point cloud clusters of each conductor.

[0022] Curve fitting is performed on the point cloud clusters of each conductor to obtain the spatial curve of each conductor.

[0023] The center line of the line is generated based on the spatial curves of each conductor.

[0024] The point cloud set of the aforementioned tower type is extracted to obtain the center point of the tower bottom.

[0025] Principal component analysis is performed on the point cloud set of the crossarm class to obtain the main direction of the crossarm.

[0026] The point cloud set of the insulator class is extracted to obtain the center point of the insulator.

[0027] Clustering and boundary extraction are performed on the point cloud set of the obstacle class to obtain the obstacle bounding box.

[0028] Optionally, based on the line reference structure, the safety constraints, and the obstacle information, a global safety inspection route is planned, including the following steps: Based on the safety distance parameter in the safety constraints, offsets are made in the lateral and height directions of the centerline of the line to generate a simulated reference trajectory.

[0029] A safety envelope set is generated based on the center point of the tower bottom, the main direction of the crossarm, the center point of the insulator, and the boundary frame of the obstacle.

[0030] The inspection space is discretized into a track node graph. Paths that satisfy the safety envelope set constraints are searched in the track node graph, and the searched paths are smoothed to obtain the global safety inspection route. The cost function of the search process includes the cost of candidate nodes deviating from the simulated reference trajectory.

[0031] Optionally, based on real-time perceived dynamic obstacle information, the global safety inspection route is locally corrected and flight control commands are generated to control the UAV to perform line-following flight, including the following steps: Based on real-time acquired 3D LiDAR point cloud data, dynamic obstacle information is identified.

[0032] When the distance between the dynamic obstacle information and the drone is less than a safe distance threshold, local obstacle avoidance is triggered.

[0033] Based on the real-time pose information of the UAV, the global safety inspection route, and the dynamic obstacle information, a local obstacle avoidance cost function is constructed.

[0034] Solve the local obstacle avoidance cost function to obtain local speed control commands, which are then used to locally correct the global safety inspection route.

[0035] The current reference point is determined based on the global safety inspection route, and the trajectory tracking error between the current position in the real-time pose information and the current reference point is calculated.

[0036] The local velocity control command and the trajectory tracking error are input into a pre-built model prediction controller, and the flight control command is generated based on the output of the model prediction controller and the safety constraints.

[0037] Optionally, after controlling the drone to perform line-following flight, the method further includes the following steps: Get the first route segment corresponding to the current tower and the second route segment corresponding to the next tower.

[0038] The intermediate transition point between the first route segment and the second route segment is determined based on the main direction of the crossarm.

[0039] Spline interpolation is performed based on the end point of the first route segment, the intermediate transition point, and the starting point of the second route segment to generate a transition route.

[0040] Control the drone to switch from the first route segment to the second route segment according to the transition route.

[0041] Secondly, this application provides an unmanned aerial vehicle (UAV) power transmission line simulation flight control system, including the following functional modules: The multi-source data acquisition module is used to acquire data from power transmission line inspection tasks, safety constraints, and multi-source sensor data; the multi-source sensor data includes three-dimensional lidar point clouds, global navigation satellite system data, and inertial measurement data.

[0042] The real-time pose determination module is used to perform fusion positioning based on the multi-source sensor data to determine the real-time pose information of the UAV.

[0043] The point cloud map generation module is used to generate a local point cloud map based on the three-dimensional lidar point cloud and the real-time pose information.

[0044] The point cloud semantic classification module is used to perform semantic classification on the local point cloud map to obtain key target categories and corresponding point cloud sets; the key target categories include conductors, towers, crossarms, insulators and obstacles.

[0045] The key structure extraction module is used to extract key structures based on the point cloud set of the key target categories, and generate route reference structures and obstacle information.

[0046] The global route planning module is used to plan a global safety inspection route based on the route reference structure, the safety constraints, and the obstacle information.

[0047] The local correction and control module is used to make local corrections to the global safety inspection route based on real-time perceived dynamic obstacle information and generate flight control commands to control the UAV to perform line-following flight.

[0048] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the UAV transmission line line-following flight control method described above.

[0049] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method, system, and device for unmanned aerial vehicle (UAV) flight control along a power transmission line. The method acquires multi-source sensor data, including 3D LiDAR point clouds, global navigation satellite system data, and inertial measurement data, based on the power transmission line inspection task and safety constraints. It then performs fusion positioning based on this data to determine the UAV's real-time pose information. This effectively addresses the problem of single-sensor positioning failure in obstructed or interfered environments, thereby improving positioning robustness and flight safety in complex environments. Furthermore, by generating a local point cloud map based on the 3D LiDAR point cloud and real-time pose information, and performing semantic classification on this local point cloud map to obtain key target categories such as conductors, towers, crossarms, insulators, and obstacles, along with their corresponding point cloud sets, the method can... The system accurately distinguishes between followable line components and obstacles that must be avoided, thereby improving the accuracy of conductor extraction and obstacle identification. Subsequently, key structure extraction is used to generate line reference structures and obstacle information. Based on the line reference structures, safety constraints, and obstacle information, a global safety inspection route is planned, enabling the simulated trajectory to adapt to different tower types and conductor sag variations, thus improving the continuity of the trajectory between towers and the consistency of the safety distance. Finally, by locally correcting the global safety inspection route based on real-time perceived dynamic obstacle information and generating flight control commands to control the UAV to perform line simulation flight, the response time of dynamic obstacle avoidance can be reduced and manual intervention can be minimized, thereby effectively improving the continuity, safety, and automation of UAV line simulation flight in complex transmission channels. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram illustrating the application environment of a UAV power transmission line simulation flight control method according to one embodiment of this application.

[0052] Figure 2 This is a flowchart illustrating a method for unmanned aerial vehicle (UAV) flight control for mimicking power transmission lines, provided as an embodiment of this application.

[0053] Figure 3 This is a flowchart illustrating step A4 of a UAV power transmission line simulation flight control method provided in an embodiment of this application.

[0054] Figure 4 This is a flowchart illustrating step A6 of a UAV power transmission line simulation flight control method provided in an embodiment of this application.

[0055] Figure 5 This is a schematic diagram illustrating the effect of key structure segmentation and automatic route generation in a UAV power transmission line simulation flight control method provided in an embodiment of this application.

[0056] Figure 6 This is a flowchart illustrating step A7 of a UAV power transmission line simulation flight control method provided in an embodiment of this application.

[0057] Figure 7 This is a flowchart illustrating step A8 of a UAV power transmission line simulation flight control method provided in an embodiment of this application.

[0058] Figure 8 This is a schematic diagram of the overall technical route of a drone-based power transmission line simulation flight control method provided in an embodiment of this application.

[0059] Figure 9 This is a schematic diagram of the functional modules of a drone power transmission line simulation flight control system provided in one embodiment of this application.

[0060] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] This embodiment uses Figure 1 Taking the example of an automated UAV line-tracking inspection scenario between power transmission line towers, this application provides a detailed explanation of the UAV line-tracking flight control method provided in this application. Figure 1 The UAV shown in part (a) is equipped with a 3D lidar, an RTK-GNSS receiver, an inertial measurement unit (IMU), an onboard computing unit, a flight control unit, and a communication link unit. Figure 1 The transmission line corridor shown in section (b) is used for pole-to-tower line simulation flight in accordance with the preset task area.

[0064] This application provides a method for unmanned aerial vehicle (UAV) flight control along a power transmission line. In one exemplary embodiment, such as... Figure 2 As shown, the method includes the following steps: A1. Acquire transmission line inspection tasks, safety constraints, and multi-source sensor data; the multi-source sensor data includes three-dimensional lidar point clouds, global navigation satellite system data, and inertial measurement data.

[0065] Specifically, the first step is to acquire the basic safety information and sensor data corresponding to the current inspection task. The inspection task includes tower number, tower location information, and the start and end points of the inspection section. The transmission line inspection task is used to determine the starting point, ending point, and flight direction of the line-following flight. The safety constraints include parameters such as conductor safety distance, tower safety distance, obstacle avoidance distance, maximum flight speed, maximum acceleration, maximum climb / descent speed, and inspection lateral deviation distance. These parameters can be obtained from maintenance task files, historical flight path files, or issued by the ground station.

[0066] In one exemplary embodiment, multi-source sensor data are acquired simultaneously, including 3D lidar point cloud, RTK-GNSS positioning data, and IMU inertial measurement data. The raw point cloud acquired by the lidar at time kk is represented as follows: in, Denotes the first in the lidar coordinate system One point, , , Here are the three-dimensional coordinates of the point. Reflection intensity, This is the timestamp for that point. This represents the total number of point clouds in the current frame.

[0067] The output of this step is: the original point cloud. and inspection safety constraint set .in, , For safe distance of conductors, For the safe distance of the pole, For safe distance from obstacles, For maximum flight speed, For maximum acceleration, and These are the minimum and maximum permitted flight altitudes, respectively.

[0068] A2. Based on the multi-source sensor data, perform fusion positioning to determine the real-time pose information of the UAV. Step A2 specifically includes the following steps: A21. Based on the inertial measurement data, state prediction is performed to obtain the predicted state information of the UAV.

[0069] In this step, a fusion framework based on Error-State Kalman Filter (ESKF) is employed to fuse RTK-GNSS, IMU, and LiDAR odometry information to obtain the UAV's real-time and continuous pose, velocity, and attitude states. The UAV state vector is uniformly represented as: in, This represents the position of the drone in the world coordinate system. For speed, For attitude quaternions, To achieve zero bias in the gyroscope, This is for zero bias of the accelerometer.

[0070] The measurement model of the IMU is: in, To measure angular velocity using a gyroscope, To measure acceleration with an accelerometer, For true angular velocity, For real acceleration, and These are the measurement noises of the gyroscope and accelerometer, respectively.

[0071] State prediction based on IMU: in, To predict the state, Input to IMU For time intervals, This is the state transition function.

[0072] State prediction equation The kinematic model based on the IMU is shown in the following equation: in, For attitude quaternions The corresponding rotation matrix, The gravity vector This represents quaternion multiplication.

[0073] During the update phase, location observations provided by RTK-GNSS are utilized. Relative pose observations provided by LiDAR odometry Through observation equations The predicted state is corrected. The observation equation can be expressed as: in, and These represent the observation noise of GNSS and LiDAR odometry, respectively.

[0074] When GNSS signals are good, the fusion positioning mainly relies on the absolute position constraints of RTK-GNSS to suppress long-term IMU drift. When GNSS signals are blocked or interfered with by multipath effects, the system automatically increases the weight of LiDAR odometry and relies on the relative pose constraints provided by point cloud matching to maintain the continuity of state estimation, thereby improving the system's positioning robustness in complex environments.

[0075] A22. Based on the Global Navigation Satellite System data and / or the 3D LiDAR point cloud, observation updates are performed to correct the predicted state information, thereby obtaining the real-time pose information of the UAV. State updates are performed using RTK-GNSS position observation and LiDAR odometry observation: in, For sensor observations, For observation models, This represents the filter gain. This filter gain can be calculated using methods such as extended Kalman filtering or factor graph optimization.

[0076] The output of this step That is, the drone in the first The real-time fusion status is used in subsequent steps for point cloud coordinate transformation and flight control.

[0077] A3. Generate a local point cloud map based on the 3D LiDAR point cloud and the real-time pose information. Step A3 specifically includes the following steps: A31. Based on the real-time pose information, the three-dimensional lidar point cloud is transformed from the lidar coordinate system to the world coordinate system to obtain the world coordinate system point cloud.

[0078] Based on the real-time pose information obtained in step A2 Point cloud in lidar coordinate system Transform to the world coordinate system. The coordinate transformation relationship is as follows: in, Points in the transformed world coordinate system. The external parameter matrix from the lidar coordinate system to the UAV system is... For the first The pose transformation matrix from the real-time unmanned aerial vehicle system to the world coordinate system, wherein the pose transformation matrix includes rotation and translation components, and can be derived from real-time pose information. The position and orientation information is used to construct the model.

[0079] A32. Preprocess the world coordinate system point cloud to obtain the local point cloud map; the preprocessing includes at least one of time synchronization, distortion compensation, voxel downsampling, outlier removal and ground filtering.

[0080] Time synchronization refers to aligning the timestamps of the 3D LiDAR point cloud with the timestamps of inertial measurement data or global navigation satellite system data from multi-source sensors to the same time reference; distortion compensation refers to correcting motion distortion in the 3D LiDAR point cloud to eliminate point cloud deformation caused by UAV movement during a single LiDAR frame scan; outlier removal refers to filtering out isolated noise points caused by sensor noise or environmental interference; ground filtering refers to separating and filtering ground points from the point cloud to reduce data interference for target identification in subsequent power transmission channels. Voxel downsampling refers to dividing the 3D space containing the world coordinate system point cloud into several cubic voxel units according to a preset side length. For each voxel unit containing at least one point, the spatial centroid of all points within that voxel unit is calculated as the representative point of that voxel unit, and all representative points replace the original point cloud to form a downsampled point cloud, thereby reducing the amount of point cloud data and suppressing random noise.

[0081] In this embodiment, voxel downsampling is performed as follows: the space is divided into sections with side lengths of... The voxel unit preserves the center of mass within each voxel: in, For the first Individual unit, This represents the number of points within the voxel. This is the center of mass of the voxel unit.

[0082] After preprocessing, a local point cloud map is output. This serves as input for subsequent point cloud classification and key structure extraction. A local point cloud map is a point cloud map within a spatial region centered on the UAV's current location and with a predetermined sensing range as its radius.

[0083] A4. Perform semantic classification on the local point cloud map to obtain key target categories and corresponding point cloud sets; the key target categories include conductors, towers, crossarms, insulators, and obstacles. Figure 3 The flowchart shown includes the following steps in step A4: A41. For each point in the local point cloud map, construct a multidimensional feature vector; the multidimensional feature vector includes at least one of the point's three-dimensional coordinates, reflection intensity, local normal vector, local curvature, and neighborhood point density.

[0084] After obtaining the preprocessed point cloud map Then, a pre-trained point cloud semantic segmentation network (such as PointNet) is used to classify the point cloud point by point. Each point in the input point cloud can be represented as: in, , , Let the coordinates of the point be the three-dimensional coordinates in the world coordinate system. The intensity is the reflection intensity.

[0085] To improve classification robustness, a multi-dimensional feature vector is constructed for each point: in, For the first Multidimensional feature vectors of points Equivalent to the three-dimensional coordinates of a point in the world coordinate system , For local normal vectors, For local curvature, denoted as the neighborhood point density.

[0086] A42. Input the multidimensional feature vector into a pre-trained point cloud classification network, and output the probability that each point belongs to each key target category through the point cloud classification network. Point Cloud Classification Network It consists of multiple multilayer perceptrons (MLPs) and symmetric pooling functions (max pooling). The network first performs independent high-dimensional mapping on the feature vector of each point, then aggregates the global features through max pooling layers, and finally concatenates the global and local features. The result is then passed through fully connected layers and a softmax function to output the feature vector of each point. Probability distribution of predefined categories: in, Total number of categories (in this embodiment) These correspond to conductors, towers, crossarms, insulators, vegetation, ground, and obstacles, respectively. The category with the highest probability is used as the final classification label for that point. The point cloud classification network outputs the probability of each point belonging to different categories, and takes the category with the highest probability as the label of that point: in, For point cloud classification networks, For network parameters, For the first Category labels for each point.

[0087] Network parameters The algorithm was pre-optimized on a large number of labeled power transmission channel point cloud datasets using backpropagation and stochastic gradient descent. The optimization objective was to minimize the cross-entropy loss function. in, For the first The true category label (one-hot encoded) of each point. For the network prediction of the first Each point belongs to the category The probability of.

[0088] The above method can assign semantic labels to each point in the local point cloud map, providing an accurate semantic information basis for subsequent key structure extraction and path planning.

[0089] A43. Using the category of the key target with the highest probability as the category label for each point, we obtain the point cloud set corresponding to each key target category. After classification, we obtain the classified point cloud set: in, For the point cloud set of the guide line, For tower point cloud aggregation, For the collection of horizontal point clouds, For insulator point cloud collection, For vegetation point cloud aggregation, For ground point cloud aggregation, A collection of point clouds representing buildings, tree barriers, temporary equipment, or other obstacles.

[0090] The output of this step is fed into step A5, which is used to extract key structural information of conductors, towers, crossarms, insulators, and obstacles.

[0091] A5. Extract key structures from the point cloud set of the key target categories to generate line reference structures and obstacle information. The line reference structures include the spatial curves of each conductor, the line centerline, the center point of the tower bottom, the main direction of the crossarm, and the center point of the insulator; the obstacle information includes obstacle bounding boxes. After obtaining the classified point clouds, this step extracts key structures for each category of point cloud. Step A5 includes the following steps: (a) Fitting of traverse space curves A51. Cluster the point cloud set of the aforementioned conductor type to separate the point cloud clusters for each conductor. The point cloud clusters for the conductors are defined according to the following formula. Perform Euclidean clustering to separate different conductors into multiple point cloud clusters: in, For the first A cluster of point clouds along a guideline.

[0092] A52. Perform curve fitting on the point cloud clusters of each conductor to obtain the spatial curve of each conductor. Perform curve fitting on the point cloud clusters of each conductor to obtain the spatial curve of the conductor: in, Here are the curve parameters along the direction of the conductor. For the first A conductor in parameters The three-dimensional position of the location.

[0093] The objective of wire fitting can be expressed as minimizing the fitting error: in, For point The corresponding parameters on the conductor parameter curve This represents the wire fitting error. In practice, it can be achieved using methods such as least squares, RANSAC, or spline fitting.

[0094] If the conductor can be approximated as a quadratic curve between adjacent towers, then its height direction can be expressed as: in, , , These are the fitting parameters for the conductor height curve.

[0095] A53. Generate the center line of the line based on the spatial curves of each conductor.

[0096] The line centerline is generated based on the conductor fitting results. If a single conductor is used as a reference, the line centerline is: If the centerline of the line corridor is constructed using multiple conductors, then: in, This represents the total number of conductors.

[0097] (ii) Extraction of the center of the tower base A54. Extract the point cloud set of the tower type to obtain the center point of the tower bottom.

[0098] Cloud Pointing from Tower In the middle, the point whose height is within the lowest range is selected as the bottom candidate point. : in, For point height, The minimum height for cloud point detection on the tower. This is the bottom height threshold.

[0099] The center point of the tower base is: in, Indicates the center point of the tower base. This represents the number of candidate points at the bottom.

[0100] (III) Extraction of crossarm direction A55. Perform principal component analysis on the point cloud set of the crossarm class to obtain the main direction of the crossarm.

[0101] For the horizontal pole cloud Calculate the covariance matrix: in, To support the center of mass of the horizontal axis, This represents the number of crossarm points. The eigenvector corresponding to the largest eigenvalue is used as the principal direction of the crossarm. The horizontal direction angle of the crossarm is: The tilt angle of the crossarm is: in, Indicates the azimuth angle of the crossarm in the horizontal plane. This indicates the angle of inclination of the crossarm relative to the horizontal plane.

[0102] (iv) Extraction of the insulator center point A56. Extract the point cloud set of the insulator class to obtain the insulator center point.

[0103] Point cloud of insulators Extract the center point: in, The center point of the insulator This represents the number of points in the insulator point cloud.

[0104] (v) Obstacle Boundary Extraction A57. Cluster and extract the boundary of the point cloud set of the obstacle class to obtain the obstacle bounding box.

[0105] Euclidean distance clustering was performed on the obstacle point cloud PobsPobs to obtain multiple independent obstacle point cloud clusters, and a 3D bounding box was constructed for each obstacle cluster. : The circuit reference structure output in this step includes: conductor curves. Centerline of the line Center of the base of the tower azimuth of the crossarm direction and tilt angle and the center point of the insulator The output obstacle information includes obstacle bounding boxes. The above results proceed to step A6, which is used to generate the safety corridor and the simulated line reference trajectory.

[0106] A6. Based on the aforementioned route reference structure, safety constraints, and obstacle information, plan a global safety inspection route. For example... Figure 4 The flowchart shown includes the following steps in step A6: A61. Based on the safety distance parameter in the safety constraints, offset the centerline of the line in the lateral and height directions to generate a simulated line reference trajectory.

[0107] The tangential direction of the line is calculated using the following formula. : After normalization, the unit tangent vector is obtained. : Construct the lateral normal direction in the horizontal plane according to the following formula. : in, It is a unit vector pointing vertically upwards. This represents the cross product operation.

[0108] The reference point for drone-based line tracing is obtained by offsetting the centerline of the line: in, For reference waypoints of the drone, This is the lateral safety offset distance. This is the height offset distance.

[0109] A62. Generate a safety envelope set based on the center point of the tower bottom, the main direction of the crossarm, the center point of the insulator, and the boundary frame of the obstacle.

[0110] A safety envelope is constructed based on the point clouds of towers, crossarms, insulators, and obstacles. For any target object (such as a tower or obstacle), its bounding box... Extended safety distance Then, the safety envelope is obtained: in, This represents the expansion of the bounding box in each direction in three-dimensional space. Distance. The set of safe envelopes is denoted as: Subsequently, the system discretizes the inter-tower inspection space into a three-dimensional track node map. Each candidate node qiqi must satisfy the following safety distance constraints: in, This represents the minimum distance from a point to a set of point clouds.

[0111] A63. Discretize the inspection space into a track node graph, search for paths that satisfy the safety envelope set constraints in the track node graph, and smooth the searched paths to obtain the global safety inspection route; wherein, the cost function of the search process includes the cost of candidate nodes deviating from the simulated line reference trajectory.

[0112] In this embodiment, the following is adopted: Dijkstra The algorithm, Algorithm A, or an improved Algorithm A, searches for safe routes. The total cost function for path search is: in, The total cost of the path. , , , These are the weighting coefficients. The distance between adjacent nodes. As a consequence of obstacle risk, For path smoothing cost, This refers to the cost of deviating from the reference trajectory. The cost of deviating from the reference trajectory is the candidate node. Deviating from the reference trajectory The measurement.

[0113] The specific risks and costs associated with obstacles are as follows: in, To prevent small positive numbers with a denominator of zero.

[0114] The specific costs of deviating from the reference trajectory are: After the search is completed, the system performs B-spline or polynomial smoothing on the path to obtain the global security inspection route. : in, For the first Global waypoints This represents the total number of waypoints.

[0115] Examples of key structure segmentation and automatic route generation are as follows: Figure 5 As shown, Figure 5 Part (a) shows the results of extracting key structural points and bounding boxes of the tower based on point cloud classification. Figure 5 Part (b) shows the process of planning and generating a global security inspection route based on key structures, photo points, and security constraints.

[0116] A7. Based on real-time perceived dynamic obstacle information, the global safety inspection route is locally corrected and flight control commands are generated to control the UAV to perform line-following flight. For example... Figure 6 The flowchart shown includes the following steps in step A7: A71. Identify dynamic obstacle information based on real-time acquired 3D LiDAR point cloud data.

[0117] During flight, newly emerging dynamic obstacles are detected using real-time point clouds. After removing known conductors, towers, and ground point clouds, Euclidean distance clustering is performed on the remaining point clouds to obtain a real-time obstacle set. : in, For the first A cluster of obstacle points.

[0118] For each obstacle point cloud cluster, calculate its centroid: in, For the first The centroid of an obstacle point cloud cluster.

[0119] A72. When the distance between the dynamic obstacle information and the UAV is less than the safe distance threshold, local obstacle avoidance is triggered.

[0120] Current location of the drone (from (obtained from) and obstacle substances Distance between for: when At that time, the system determines that local obstacle avoidance needs to be triggered.

[0121] A73. Based on the real-time pose information of the UAV, the global safety inspection route, and the dynamic obstacle information, construct a local obstacle avoidance cost function. Local obstacle avoidance cost function As shown in the following formula: in, The cost of moving towards the target direction, The cost of maintaining a safe distance from obstacles, For the price of speed, To control the smoothing cost, , , , These are the weights of each cost.

[0122] A74. Solve the local obstacle avoidance cost function to obtain local speed control commands, which are then used to locally correct the global safety inspection route. Select the control variable that satisfies the safety distance constraint and minimizes the total cost to generate local speed commands: in, , , This is a three-axis speed command. This is the yaw rate command. This local speed command is used to make real-time corrections to the global safety inspection route.

[0123] A75. Determine the current reference point based on the global safety inspection route, and calculate the trajectory tracking error between the current position in the real-time pose information and the current reference point.

[0124] According to the overall security inspection route Local speed command Real-time status of drones Calculate the trajectory tracking error. Let the current reference point be... The drone's current location is Then the trajectory tracking error is: Error along the route Lateral error and height error They are respectively: in, For reference track tangential direction, The lateral direction of the line. The vertical direction.

[0125] A76. Input the local velocity control command and the trajectory tracking error into the pre-built model prediction controller, and generate the flight control command based on the output of the model prediction controller and the safety constraints.

[0126] Flight control commands are generated using a combination of adaptive PID control and model predictive control (MPC). The adaptive PID control input is: in, , , These are the proportional, differential, and integral gains, respectively. The rate of change of error, This is the error integral term.

[0127] The optimization objective of MPC is: in, For the future Reference state for each prediction step size To control the input, To control the amount of input variation, , , This is the weight matrix. To predict the length of the time domain. express .

[0128] The constraints of MPC include: in, , , Each of the future The predicted step size includes position, velocity, and acceleration. For maximum flight speed, This is the maximum acceleration.

[0129] The combined outputs of MPC are used to generate the final flight control execution commands: in, , , For speed control commands, The desired heading angle. The flight control system based on... Adjust the drone's attitude and speed to ensure stable flight along the power line and maintain a safe distance from the power line, towers, and obstacles.

[0130] To achieve continuous line-following flight between towers, after controlling the UAV to perform line-following flight, the method further includes: A8. Smoothly connect the routes between adjacent towers to ensure continuous flight across towers and spans. For example... Figure 7 As shown, the specific steps include: A81. Obtain the first route segment corresponding to the current tower and the second route segment corresponding to the next tower.

[0131] A82. Determine the intermediate transition point between the first route segment and the second route segment based on the main direction of the crossarm.

[0132] A83. Spline interpolation is performed based on the end point of the first route segment, the intermediate transition point, and the starting point of the second route segment to generate a transition route. For two adjacent route segments, i.e., the first route segment... Second Route Section Construct a transition section: in, The first route segment The end point, For the second route segment The starting point, This is an intermediate transition point generated based on the main direction of the crossarm and the safety distance. This represents the spline interpolation function. To ensure a smooth transition path, the transition segment is constrained to satisfy the tangential continuity condition: in, Indicate route Regarding curve parameters The tangential direction. In this way, a continuous, safe, and smooth linear flight path is formed between different tower spans.

[0133] A84. Control the UAV to switch from the first route segment to the second route segment according to the transition route.

[0134] In another exemplary embodiment of this application, after step A8, the method further includes: A9. Save the inspection trajectory, point cloud map, obstacle avoidance record, control error and task results, and update the next inspection segment.

[0135] After completing the current traverse simulation flight between towers, record the actual flight trajectory, point cloud map, traverse fitting results, obstacle information, obstacle avoidance trigger records, fused positioning status, and flight control errors. The recorded content is uniformly represented as follows: in, To record content, In order to plan the route, For flight state sequence, It is a point cloud map sequence. For obstacle sequence, To control the sequence of instructions, Statistics on trajectory tracking errors.

[0136] If the current inspection task has not been completed, update the target flight segment based on the point cloud recognition result of the next tower or the next span, and return to the previous step A1 to continue execution; if the inspection task is completed, save all inspection data and end the task.

[0137] like Figure 8The schematic diagram of the overall technical roadmap shown in this embodiment illustrates that the method provided in this embodiment constructs a closed-loop line-following flight control method encompassing "perception and mapping - target recognition and feature extraction - path planning and obstacle avoidance - flight control and execution - inspection and data management, mission monitoring and anomaly handling." This transforms the traditional transmission line inspection method, which relies on manual remote control or fixed waypoints, into a seamless line-following flight process that can automatically generate and correct trajectories based on the real-time 3D environment. This process enables the UAV to maintain a safe distance, stably track the conductor direction, and achieve seamless connection between towers in complex transmission line scenarios. Compared with traditional manual remote control or fixed waypoint methods, this application can effectively improve line-following accuracy, increase obstacle avoidance response speed, reduce manual intervention, and significantly improve inspection efficiency.

[0138] Based on the same inventive concept, this application also provides a system for implementing the above-described UAV power transmission line simulation flight control method. The solution provided by this system is similar to the implementation described in the above method. In an exemplary embodiment, such as... Figure 9 As shown, a UAV power transmission line simulation flight control system is provided, including the following functional modules: The multi-source data acquisition module is used to acquire data from power transmission line inspection tasks, safety constraints, and multi-source sensor data; the multi-source sensor data includes three-dimensional lidar point clouds, global navigation satellite system data, and inertial measurement data.

[0139] The real-time pose determination module is used to perform fusion positioning based on the multi-source sensor data to determine the real-time pose information of the UAV.

[0140] The point cloud map generation module is used to generate a local point cloud map based on the three-dimensional lidar point cloud and the real-time pose information.

[0141] The point cloud semantic classification module is used to perform semantic classification on the local point cloud map to obtain key target categories and corresponding point cloud sets; the key target categories include conductors, towers, crossarms, insulators and obstacles.

[0142] The key structure extraction module is used to extract key structures based on the point cloud set of the key target categories, and generate route reference structures and obstacle information.

[0143] The global route planning module is used to plan a global safety inspection route based on the route reference structure, the safety constraints, and the obstacle information.

[0144] The local correction and control module is used to make local corrections to the global safety inspection route based on real-time perceived dynamic obstacle information and generate flight control commands to control the UAV to perform line-following flight.

[0145] As an optional implementation, the system further includes a route connection module, used to: obtain a first route segment corresponding to the current tower and a second route segment corresponding to the next tower; determine an intermediate transition point between the first route segment and the second route segment based on the main direction of the crossarm; perform spline interpolation based on the end point of the first route segment, the intermediate transition point, and the starting point of the second route segment to generate a transition route; and control the UAV to switch from the first route segment to the second route segment according to the transition route.

[0146] certainly, Figure 9 The architecture shown is merely exemplary; it can be omitted as needed when implementing different functionalities. Figure 9 One or at least two components of the system shown.

[0147] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 10 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data generated during the line-following flight control process. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it can implement the UAV power transmission line line-following flight control method provided in the previous embodiment.

[0148] Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0150] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0151] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations and be authorized by the owner of the corresponding device.

[0153] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0154] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for controlling the flight of an unmanned aerial vehicle (UAV) along a power transmission line, characterized in that, include: Acquire data on power transmission line inspection tasks, safety constraints, and multi-source sensor data; The multi-source sensor data includes three-dimensional lidar point clouds, global navigation satellite system data, and inertial measurement data; Based on the multi-source sensor data, the real-time pose information of the UAV is determined through fusion positioning. A local point cloud map is generated based on the three-dimensional lidar point cloud and the real-time pose information; Semantic classification is performed on the local point cloud map to obtain key target categories and corresponding point cloud sets; the key target categories include conductors, towers, crossarms, insulators, and obstacles; Key structures are extracted based on the point cloud set of the key target categories to generate route reference structures and obstacle information; Based on the line reference structure, the safety constraints, and the obstacle information, a global safety inspection route is planned. Based on real-time perceived dynamic obstacle information, the global safety inspection route is locally corrected and flight control commands are generated to control the UAV to perform line-following flight.

2. The UAV transmission line simulation flight control method according to claim 1, characterized in that, Based on the multi-source sensor data, a fusion positioning system is performed to determine the real-time pose information of the UAV, specifically including: Based on the inertial measurement data, state prediction is performed to obtain the predicted state information of the UAV; The predicted state information is corrected by observing and updating the data from the Global Navigation Satellite System and / or the three-dimensional lidar point cloud to obtain the real-time pose information of the UAV.

3. The UAV transmission line simulation flight control method according to claim 1, characterized in that, Based on the 3D LiDAR point cloud and the real-time pose information, a local point cloud map is generated, specifically including: Based on the real-time pose information, the 3D lidar point cloud is transformed from the lidar coordinate system to the world coordinate system to obtain a world coordinate system point cloud; the real-time pose information is used to construct the pose transformation matrix from the lidar coordinate system to the world coordinate system; the pose transformation matrix includes rotation and translation components. The point cloud in the world coordinate system is preprocessed to obtain the local point cloud map; the preprocessing includes at least one of time synchronization, distortion compensation, voxel downsampling, outlier removal and ground filtering; the local point cloud map is a point cloud map within a spatial region centered on the current position of the UAV and with a predetermined sensing range as its radius.

4. The UAV transmission line simulation flight control method according to claim 1, characterized in that, Semantic classification is performed on the local point cloud map to obtain key target categories and corresponding point cloud sets, including: For each point in the local point cloud map, a multidimensional feature vector is constructed; the multidimensional feature vector includes at least one of the point's three-dimensional coordinates, reflection intensity, local normal vector, local curvature, and neighborhood point density. The multidimensional feature vector is input into a pre-trained point cloud classification network, and the point cloud classification network outputs the probability that each point belongs to each key target category. The category of the key target corresponding to the highest probability is used as the category label of each point, thus obtaining the point cloud set corresponding to each key target category.

5. The UAV transmission line simulation flight control method according to claim 1, characterized in that, The line reference structure includes the spatial curves of each conductor, the line centerline, the center point of the tower bottom, the main direction of the crossarm, and the center point of the insulator; The obstacle information includes obstacle bounding boxes; Based on the point cloud set of the key target categories, key structures are extracted to generate route reference structures and obstacle information, including: Cluster the point cloud set of the aforementioned conductor type to separate the point cloud clusters of each conductor; Curve fitting is performed on the point cloud clusters of each conductor to obtain the spatial curve of each conductor; Generate the line centerline based on the spatial curves of each conductor; Extract the point cloud set of the tower type to obtain the center point of the tower bottom; Principal component analysis is performed on the point cloud set of the crossarm type to obtain the main direction of the crossarm; The point cloud set of the insulator class is extracted to obtain the center point of the insulator; Clustering and boundary extraction are performed on the point cloud set of the obstacle class to obtain the obstacle bounding box.

6. The UAV transmission line simulation flight control method according to claim 5, characterized in that, Based on the aforementioned line reference structure, safety constraints, and obstacle information, a global safety inspection route is planned, including: Based on the safety distance parameter in the safety constraints, offsets are made in the lateral and height directions of the line centerline to generate a simulated line reference trajectory; A safety envelope set is generated based on the center point of the tower bottom, the main direction of the crossarm, the center point of the insulator, and the boundary frame of the obstacle; The inspection space is discretized into a track node graph. Paths that satisfy the safety envelope set constraints are searched in the track node graph, and the searched paths are smoothed to obtain the global safety inspection route. The cost function of the search process includes the cost of candidate nodes deviating from the simulated reference trajectory.

7. The UAV transmission line line simulation flight control method according to claim 6, characterized in that, Based on real-time perceived dynamic obstacle information, the global safety inspection route is locally corrected and flight control commands are generated to control the UAV to perform line-following flight, including: Based on the real-time acquired 3D LiDAR point cloud, dynamic obstacle information is identified; When the distance between the dynamic obstacle information and the drone is less than a safe distance threshold, local obstacle avoidance is triggered. Based on the real-time pose information of the UAV, the global safety inspection route, and the dynamic obstacle information, a local obstacle avoidance cost function is constructed. Solve the local obstacle avoidance cost function to obtain local speed control commands, so as to make local corrections to the global safety inspection route; The current reference point is determined based on the global safety inspection route, and the trajectory tracking error between the current position in the real-time pose information and the current reference point is calculated. The local velocity control command and the trajectory tracking error are input into a pre-built model prediction controller, and the flight control command is generated based on the output of the model prediction controller and the safety constraints.

8. The UAV transmission line simulation flight control method according to claim 6, characterized in that, After controlling the drone to perform line-following flight, the method further includes: Get the first route segment corresponding to the current tower and the second route segment corresponding to the next tower; The intermediate transition point between the first route segment and the second route segment is determined according to the main direction of the crossarm. Spline interpolation is performed based on the end point of the first route segment, the intermediate transition point, and the starting point of the second route segment to generate a transition route; Control the drone to switch from the first route segment to the second route segment according to the transition route.

9. A UAV power transmission line simulation flight control system, characterized in that, include: The multi-source data acquisition module is used to acquire data from power transmission line inspection tasks, safety constraints, and multi-source sensor data. The multi-source sensor data includes three-dimensional lidar point clouds, global navigation satellite system data, and inertial measurement data; The real-time pose determination module is used to perform fusion positioning based on the multi-source sensor data to determine the real-time pose information of the UAV. The point cloud map generation module is used to generate a local point cloud map based on the three-dimensional lidar point cloud and the real-time pose information. The point cloud semantic classification module is used to perform semantic classification on the local point cloud map to obtain key target categories and corresponding point cloud sets; the key target categories include conductors, towers, crossarms, insulators, and obstacles. The key structure extraction module is used to extract key structures based on the point cloud set of the key target categories, and generate route reference structures and obstacle information; The global route planning module is used to plan a global safety inspection route based on the route reference structure, the safety constraints, and the obstacle information. The local correction and control module is used to make local corrections to the global safety inspection route based on real-time perceived dynamic obstacle information and generate flight control commands to control the UAV to perform line-following flight.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the UAV transmission line line-following flight control method according to any one of claims 1-8.

Images