Flight control instruction determination method and device, and electronic device

Abstract

The application discloses a flight control instruction determination method and device and electronic equipment. The method comprises the following steps: acquiring multi-modal sensor data; determining component instance information according to the multi-modal sensor data, wherein the component instance information comprises a three-dimensional space position of a power component instance and a component category, and the component category comprises a conductor and a tower; fitting a conductor three-dimensional center line space track according to the three-dimensional space position of the conductor instance in the component instance information, and taking the conductor three-dimensional center line space track as a reference path; determining deviation data corresponding to the unmanned aerial vehicle according to the reference path and a current pose of the unmanned aerial vehicle; inputting the deviation data into a target compound controller to generate a first flight control instruction, and controlling the unmanned aerial vehicle to fly according to the first flight control instruction. The application solves the technical problems of low tracking accuracy and unreliable safety distance in the related art when the unmanned aerial vehicle autonomously tracks and flies along an overhead line.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a method, apparatus, and electronic device for determining flight control commands. Background Technology

[0002] Current UAV-based 10kV line inspection technology has significant limitations. Mainstream solutions rely on preset GPS waypoints, essentially performing "path playback" rather than "autonomous tracking," lacking understanding and autonomous decision-making capabilities regarding line branches, corners, and other topological structures. Their obstacle avoidance functions are mostly simple static responses (stopping upon encountering an obstacle), struggling to handle dynamic obstacles and prone to deviating from the inspection corridor due to obstacle avoidance maneuvers. More importantly, the system lacks robustness: if the mission is interrupted due to signal loss, sensor failure, or emergency obstacle avoidance, the UAV cannot intelligently "resume execution," typically only passively returning to base or hovering awaiting manual intervention. This leads to a complete breakdown of the automated process, low inspection efficiency, and technical issues such as low tracking accuracy and unreliable safe distance maintenance when UAVs autonomously track overhead lines.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a method, apparatus, and electronic device for determining flight control commands, in order to at least solve the technical problems of low tracking accuracy and unreliable safe distance maintenance when UAVs autonomously track overhead lines.

[0005] According to one aspect of the present invention, a method for determining flight control commands is provided, comprising: acquiring multimodal sensor data, wherein the multimodal sensor data is collected by a sensor system mounted on a UAV, including spatial depth point cloud data; determining component instance information based on the multimodal sensor data, wherein the component instance information includes the three-dimensional spatial position and component category of a power component instance, the component category including: conductor, tower; fitting and generating a three-dimensional centerline spatial trajectory of the conductor based on the three-dimensional spatial position of the conductor instance in the component instance information, and using the three-dimensional centerline spatial trajectory of the conductor as a reference path; determining deviation data corresponding to the UAV based on the reference path and the current pose of the UAV, wherein the deviation data includes lateral deviation and altitude deviation. The deviations are: lateral deviation, which is the horizontal and vertical distance from the center of mass of the UAV to the nearest guideline reference point; altitude deviation, which is the preset vertical following altitude difference between the UAV and the guideline; and heading angle deviation, which is the angle difference between the tangent direction of the guideline at the reference point and the yaw angle of the UAV. The deviation data are input to the target composite controller to generate a first flight control command, which controls the UAV to fly according to the first flight control command. The first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the curvature of the guideline. The first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guideline, and to fly along the spatial extension direction of the guideline.

[0006] Optionally, after controlling the UAV to fly according to the first flight control command, the method further includes: determining whether the distance between the UAV and the target tower instance is less than a preset safe distance threshold based on the component instance information, and obtaining a first determination result; if the first determination result indicates that the distance is less than the preset safe distance threshold, triggering a tower processing mode; in the tower processing mode, determining the coordinates of the tower center point of the target tower instance, and constructing a local occupancy grid map of the target tower instance based on the spatial depth point cloud data; using a target trajectory algorithm, taking the tower center point coordinates as the trajectory reference point and the target variable as the optimization objective, generating a tower flight path, wherein the target variable includes trajectory length, trajectory execution time, and repulsive potential field between the trajectory point and the obstacle, and the obstacle is represented by the occupancy grid in the local occupancy grid map; determining a second flight control command based on the tower flight path, and controlling the UAV to fly according to the tower flight path.

[0007] Optionally, after controlling the UAV to fly according to the first flight control command, the method further includes: determining a second determination result based on the component instance information to see if a line branch point has been identified; if the second determination result indicates that a line branch point has been identified, triggering a branch decision mode; in the branch decision mode, determining a target branch line to be tracked from multiple candidate branch lines, wherein the method of determining the target branch line includes: determining the target branch line according to a preset tracking rule, wherein the preset tracking rule includes one of the following: selecting a candidate branch line with the largest cross-section, selecting a candidate branch line with the smallest deviation from the current heading angle of the UAV, selecting a candidate branch line that matches a preset mission target; and / or, inputting the conductor feature data of the multiple candidate branch lines into a decision model to determine the target branch line, wherein the decision model is a model trained based on historical inspection data; determining a third flight control command based on the target branch line, and controlling the UAV to fly according to the third flight control command.

[0008] Optionally, after controlling the UAV to fly according to the first flight control command, the method further includes: determining a target obstacle on the flight path; determining a target correction path corresponding to the target obstacle; determining a correction flight control command based on the target correction path, and controlling the UAV to fly according to the correction flight control command to control the UAV to bypass the target obstacle.

[0009] Optionally, determining the target correction path corresponding to the target obstacle includes: if the target obstacle includes a static obstacle, determining a target correction path for the static obstacle; and / or, if the target obstacle includes a dynamic obstacle, determining a predicted obstacle trajectory corresponding to the dynamic obstacle; determining the minimum predicted distance and collision time between the UAV and the dynamic obstacle based on the predicted obstacle trajectory; determining the collision risk probability based on the minimum predicted distance and the collision time; and when the collision risk probability exceeds a preset risk threshold, using a target path planning algorithm to generate a path around the dynamic obstacle. The target path correction algorithm includes a target artificial potential field function, which includes a gravitational potential field term, a static obstacle repulsion potential field term, a dynamic obstacle repulsion potential field term, and a path-maintaining term. The gravitational potential field term guides the UAV to move along the original path. The static obstacle repulsion potential field term maintains a safe distance between the UAV and the static obstacle. The dynamic obstacle repulsion potential field term enables the UAV to avoid the dynamic obstacle based on the relative speed of the dynamic obstacle and the collision time. The path-maintaining term constrains the deviation of the corrected path from the original path.

[0010] Optionally, after determining a third flight control command based on the target branch line and controlling the UAV to fly according to the third flight control command, the method further includes: generating traceable point information corresponding to the branch line point, wherein the traceable point information includes the branch line point coordinate data; storing the traceable point information in a traceable point stack so that after a task interruption, the traceable point information can be retrieved from the traceable point stack and the task continuation operation can be performed based on the traceable point information.

[0011] Optionally, after controlling the UAV to fly according to the first flight control command, the method further includes: when a mission interruption occurs, performing positioning based on the UAV's position coordinate data and attitude inertial sequence data to obtain interruption positioning data; controlling the UAV to fly to the mission interruption area based on the interruption positioning data; acquiring a current environmental image corresponding to the mission interruption area, matching the current environmental image with an image in a stored area to obtain target positioning data; determining a target backtracking point from multiple backtracking points included in the backtracking point stack based on the target positioning data; and controlling the UAV to fly to the target backtracking point.

[0012] According to one aspect of the present invention, a flight control command determination device is provided, comprising: an acquisition module for acquiring multimodal sensor data, wherein the multimodal sensor data is collected by a sensor system carried by an unmanned aerial vehicle (UAV) and includes spatial depth point cloud data; a first determination module for determining component instance information based on the multimodal sensor data, wherein the component instance information includes the three-dimensional spatial position and component category of a power component instance, the component category including: conductor, tower; a second determination module for fitting and generating a three-dimensional centerline spatial trajectory of the conductor based on the three-dimensional spatial position of the conductor instance in the component instance information, and using the three-dimensional centerline spatial trajectory of the conductor as a reference path; and a third determination module for determining deviation data corresponding to the UAV based on the reference path and the current pose of the UAV, wherein the deviation... The difference data includes lateral deviation, altitude deviation, and heading angle deviation. The lateral deviation is the horizontal and vertical distance from the center of mass of the UAV to the nearest guideline reference point. The altitude deviation is the preset vertical following altitude difference between the UAV and the guideline. The heading angle deviation is the angle difference between the tangent direction of the guideline at the reference point and the yaw angle of the UAV. The generation module is used to input the deviation data to the target composite controller, generate a first flight control command, and control the UAV to fly according to the first flight control command. The first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the curvature of the guideline. The first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guideline, and to fly along the spatial extension direction of the guideline.

[0013] According to one aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the flight control instruction determination method described in any one of the preceding embodiments.

[0014] According to one aspect of the present invention, a computer-readable storage medium is provided, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the flight control command determination method described above.

[0015] In this embodiment of the invention, multimodal sensor data is acquired, which is collected by a sensor system mounted on a UAV, including spatial depth point cloud data; based on the multimodal sensor data, component instance information is determined, wherein the component instance information includes the three-dimensional spatial position and component category of the power component instance, and the component category includes: conductor and tower; based on the three-dimensional spatial position of the conductor instance in the component instance information, a three-dimensional centerline spatial trajectory of the conductor is fitted and generated, and the three-dimensional centerline spatial trajectory of the conductor is used as a reference path; based on the reference path and the current pose of the UAV, deviation data corresponding to the UAV is determined, wherein the deviation data includes lateral deviation, altitude deviation, and heading angle. The deviations are defined as follows: lateral deviation is the horizontal and vertical distance from the UAV's center of mass to the nearest guideline reference point; altitude deviation is the preset vertical following altitude difference between the UAV and the guideline; and heading angle deviation is the angle difference between the tangent direction of the guideline at the reference point and the UAV's yaw angle. The deviation data are input to the target composite controller to generate a first flight control command, which controls the UAV to fly according to the first flight control command. The first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the guideline curvature. The first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guideline, and to fly along the spatial extension direction of the guideline. This invention employs a hybrid approach combining three-dimensional centerline trajectory guidance of a conductor with proportional-integral-derivative (PI-DE) feedforward control. It generates a three-dimensional centerline trajectory of the conductor as a reference path, determines deviation data based on the reference path and the UAV's current pose, and inputs the deviation data and feedforward control components based on the conductor curvature into the target composite controller to generate the first flight control command. This achieves precise control of the UAV's flight along the conductor's spatial extension direction and maintains a stable, preset safe distance from the conductor. This improves the autonomous tracking accuracy of UAVs over overhead lines and reliably ensures a safe distance between the UAV and the conductor, thus solving the technical problems of low tracking accuracy and unreliable safe distance maintenance during autonomous tracking of overhead lines by UAVs in related technologies. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0017] Figure 1 This is a flowchart of a flight control command determination method according to an embodiment of the present invention;

[0018] Figure 2 This is a system architecture diagram provided by an optional embodiment of the present invention;

[0019] Figure 3This is a flowchart of the hybrid behavior model state machine provided in an optional embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of the dynamic obstacle avoidance principle provided by an optional embodiment of the present invention;

[0021] Figure 5 This is a structural block diagram of a flight control command determination device according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Example 1

[0025] According to an embodiment of the present invention, an embodiment of a flight control command determination method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] Figure 1 This is a flowchart of a flight control command determination method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0027] Step S102: Acquire multimodal sensor data, wherein the multimodal sensor data is collected by the sensor system carried by the UAV, including: spatial depth point cloud data;

[0028] Among them, multimodal sensor data refers to the heterogeneous data set collected simultaneously by various types of sensors carried by UAVs in the same inspection mission. It can provide multi-dimensional and complementary panoramic perception information for power line inspection, such as spatial depth point cloud data providing three-dimensional geometric structure information.

[0029] Among them, spatial depth point cloud data refers to a set of three-dimensional spatial points generated by lidar by emitting laser beams and receiving reflected echoes. It can accurately measure the distance between drones and power lines, towers, and vegetation, providing centimeter-level spatial information for power line tracking and obstacle avoidance.

[0030] In this step, the sensor system onboard the UAV simultaneously collects multimodal sensor data, including spatial depth point cloud data, during the inspection flight. This step provides a precise 3D spatial data foundation for subsequent power component instance identification and conductor 3D spatial trajectory fitting, enabling the system to accurately perceive the spatial position information of conductors and towers in complex power line corridor environments.

[0031] Step S104: Based on the multimodal sensor data, determine the component instance information, wherein the component instance information includes the three-dimensional spatial location and component category of the power component instance, and the component category includes: conductor, tower;

[0032] Among them, component instance information refers to a structured data set containing individual attribute information of each power component, which is output after sensing and identifying multimodal sensor data. It can provide accurate tracking target information for subsequent conductor tracking and flight control, including the three-dimensional spatial location and component category of the power component instance.

[0033] Among them, power component instances refer to specific equipment individuals with unique spatial locations and individual attributes that are actually identified by perception algorithms in specific inspection scenarios. They can provide clear tracking targets for conductor tracking. For example, a certain tower instance has unique three-dimensional coordinates and number, and a certain conductor segment instance has a unique spatial direction.

[0034] Among them, the component category refers to the equipment type label to which the identified power component belongs. It can distinguish different types of power equipment and provide a target type basis for tracking and control. For example, the category label of the conductor indicates that the component is a conductor that can be tracked, and the category label of the tower indicates that the component is a node that requires special handling.

[0035] In this step, the multimodal sensor data acquired in step S102 is sensed and identified to determine the three-dimensional spatial location and component category of each power component instance, distinguishing between conductor instances and tower instances. Through this step, the system extracts the spatial location information of trackable conductor instances from the original sensor data, providing a clear tracking target for subsequent reference path generation. This enables the UAV to know which conductor to track, its position and direction in three-dimensional space, and when it will encounter a tower node ahead.

[0036] Step S106: Based on the three-dimensional spatial position of the conductor instance in the component instance information, fit and generate the three-dimensional centerline spatial trajectory of the conductor, and use the three-dimensional centerline spatial trajectory of the conductor as a reference path.

[0037] Among them, the three-dimensional centerline spatial trajectory of the traverse refers to the three-dimensional spatial curve that extends along the geometric centerline of the traverse after continuous curve fitting of each spatial point of the traverse instance. It can provide the UAV with a smooth and continuous spatial reference path, avoiding the problems of discontinuous and abrupt tracking trajectories caused by directly using discrete traverse sensing points.

[0038] The reference path refers to the ideal three-dimensional spatial path that the UAV expects to follow during autonomous tracking flight. It provides a continuous spatial reference benchmark for the UAV's flight control, enabling the flight control system to generate corresponding corrective control commands based on the deviation between the current position and the reference path.

[0039] In this step, based on the three-dimensional spatial position information of the traverse instance determined in step S104, three-dimensional spatial curve fitting is performed on the discrete spatial points constituting the traverse to generate a continuous and smooth spatial trajectory extending along the geometric center line of the traverse. This trajectory is then used as a reference path for the UAV's tracking flight. Through this step, the discrete traverse sensing points are transformed into continuous parametric spatial curves, providing a smooth and continuous tracking reference benchmark for the flight control system. This avoids the tracking trajectory jitter problem caused by the discrete and discontinuous traverse sensing points, enabling the UAV to fly smoothly along the traverse.

[0040] Step S108: Based on the reference path and the current pose of the UAV, determine the deviation data corresponding to the UAV. The deviation data includes lateral deviation, altitude deviation and heading angle deviation. The lateral deviation is the horizontal and vertical distance from the center of mass of the UAV to the nearest guide reference point. The altitude deviation is the preset vertical following altitude difference between the UAV and the guide. The heading angle deviation is the angle difference between the tangent direction of the guide at the reference point and the yaw angle of the UAV.

[0041] Among them, deviation data refers to the set of error data that represents the degree of deviation of the UAV's current spatial position and attitude from the reference path. It can provide specific error inputs to the flight controller, enabling it to generate corresponding corrective control quantities, including lateral deviation, altitude deviation, and heading angle deviation.

[0042] Among them, lateral deviation refers to the horizontal and vertical distance from the UAV's center of mass to the nearest guideline reference point. It can quantify the degree of deviation of the UAV in the lateral direction of the guideline and provide the controller with error basis for lateral correction.

[0043] Among them, altitude deviation refers to the difference between the current altitude of the drone and the preset vertical following altitude. It can quantify the degree of deviation of the drone in the vertical direction and provide the controller with an error basis for altitude correction.

[0044] Among them, the heading angle deviation, which is the angle difference between the tangent direction of the guide line at the reference point and the current yaw angle of the UAV, can quantify the degree of angular deviation between the UAV's nose direction and the direction of the guide line, and provide the controller with an error basis for heading correction.

[0045] In this step, the position and tangent direction of the nearest reference point on the reference path generated in step S106 are compared with the current three-dimensional spatial position and yaw angle of the UAV to calculate three types of error data: lateral deviation, altitude deviation, and heading angle deviation. This step quantifies the spatial position and attitude deviations of the UAV relative to the guide wire into specific error values, providing a clear error input for the subsequent proportional-integral-derivative (PID) feedforward composite controller. This allows the controller to generate corresponding corrective control components for different types of deviations, thereby achieving precise tracking control.

[0046] Step S110: Input the deviation data into the target composite controller to generate a first flight control command, and control the UAV to fly according to the first flight control command. The first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the curvature of the guide wire. The first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guide wire, and to fly along the spatial extension direction of the guide wire.

[0047] Among them, the target composite controller refers to a flight controller that integrates four control components: proportional control, integral control, derivative control, and feedforward control based on conductor curvature. It can combine the advantages of multiple control strategies and simultaneously achieve rapid response to tracking errors, steady-state elimination, overshoot suppression, and early compensation for curve changes.

[0048] The first flight control command refers to the attitude control command generated by the target composite controller to control the UAV to fly autonomously along the guide wire. It can directly drive the flight control actuator of the UAV to adjust the flight attitude, so that the UAV maintains a preset safe distance from the guide wire and flies along the direction of the guide wire.

[0049] The proportional control component refers to the controller output component that is proportional to the current deviation. It can respond instantly to the current tracking error. The larger the deviation, the larger the proportional control component, enabling the UAV to quickly approach the reference path.

[0050] The integral control component refers to the controller output component that is proportional to the cumulative amount of deviation over time. It can eliminate steady-state tracking errors caused by constant disturbances such as wind force and load imbalance, ensuring that the UAV will not deviate from the reference path during long-term flight.

[0051] Among them, the differential control component refers to the controller output component that is proportional to the rate of change of the deviation. It can predict the trend of deviation change and perform damping correction in advance, suppress overshoot and oscillation, and make the tracking process smoother and more stable.

[0052] Among them, the feedforward control component based on the curvature of the guide wire refers to the control component pre-calculated based on the curvature of the reference path at the current reference point. It can apply steering control in advance when the UAV is about to enter the curve, compensate for the tracking error caused by the turn, and avoid deviation from the curve due to the lag in the controller response.

[0053] In this step, the three types of deviation data calculated in step S108 are input to the target composite controller. The proportional control component responds instantly to the current deviation, driving the UAV to quickly return to its position. The integral control component accumulates historical deviations to eliminate steady-state errors. The derivative control component performs damping correction based on the deviation change trend to suppress oscillations. The feedforward control component based on the curvature of the conductor compensates for the tracking error caused by turning in advance based on the curvature of the conductor bend. The four control components are superimposed to generate the first flight control command, controlling the UAV to fly along the conductor. Through this step, the target composite controller combines the advantages of feedback control and feedforward control. The proportional-integral-derivative feedback control loop can effectively cope with random disturbances and ensure tracking steady-state accuracy. The feedforward control component based on the curvature of the conductor uses the prior geometric information of the conductor's direction to generate a turning action in advance at the bend, overcoming the problem of curve tracking deviation caused by response lag in pure feedback control. This allows the UAV to maintain a precise safe distance and following altitude when flying on straight sections, and smoothly navigate curves without deviating from the conductor when flying on curved sections, achieving high-precision and high-reliability autonomous tracking flight of overhead lines.

[0054] Through steps S102-S110 above, multimodal sensor data is acquired. This multimodal sensor data is collected by the sensor system mounted on the UAV and includes spatial depth point cloud data. Based on the multimodal sensor data, component instance information is determined. This component instance information includes the three-dimensional spatial location and component category of the power component instance, with the component category including: conductor and tower. Based on the three-dimensional spatial location of the conductor instance in the component instance information, a three-dimensional centerline spatial trajectory of the conductor is fitted and generated, and this trajectory is used as a reference path. Based on the reference path and the current pose of the UAV, deviation data corresponding to the UAV is determined. This deviation data includes lateral deviation, altitude deviation, and... The heading angle deviation is calculated as follows: the lateral deviation is the horizontal and vertical distance from the UAV's center of mass to the nearest guideline reference point; the altitude deviation is the preset vertical following altitude difference between the UAV and the guideline; and the heading angle deviation is the angle difference between the tangent direction of the guideline at the reference point and the UAV's yaw angle. The deviation data is input to the target composite controller to generate a first flight control command, and the UAV is controlled to fly according to the first flight control command. The first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the curvature of the guideline. The first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guideline, and to fly along the spatial extension direction of the guideline. This invention employs a hybrid approach combining three-dimensional centerline trajectory guidance of a conductor with proportional-integral-derivative (PI-DE) feedforward control. It generates a three-dimensional centerline trajectory of the conductor as a reference path, determines deviation data based on the reference path and the UAV's current pose, and inputs the deviation data and feedforward control components based on the conductor curvature into the target composite controller to generate the first flight control command. This achieves precise control of the UAV's flight along the conductor's spatial extension direction and maintains a stable, preset safe distance from the conductor. This improves the autonomous tracking accuracy of UAVs over overhead lines and reliably ensures a safe distance between the UAV and the conductor, thus solving the technical problems of low tracking accuracy and unreliable safe distance maintenance during autonomous tracking of overhead lines by UAVs in related technologies.

[0055] As an optional embodiment, after controlling the UAV to fly according to the first flight control command, the method further includes: determining whether the distance between the UAV and the target tower instance is less than a preset safe distance threshold based on component instance information, and obtaining a first determination result; if the first determination result is that the distance is less than the preset safe distance threshold, triggering a tower processing mode; in the tower processing mode, determining the coordinates of the tower center point of the target tower instance, and constructing a local occupancy grid map of the target tower instance based on spatial depth point cloud data; using a target trajectory algorithm, taking the tower center point coordinates as the trajectory reference point and the target variable as the optimization objective, generating a tower flight path, wherein the target variable includes trajectory length, trajectory execution time, and the repulsive potential field between the trajectory point and the obstacle, and the obstacle is represented by the occupancy grid in the local occupancy grid map; determining a second flight control command based on the tower flight path, and controlling the UAV to fly according to the tower flight path.

[0056] Among them, the target tower instance refers to a specific tower instance located in front of the UAV and gradually approaching it during the main line tracking flight, which is identified by the perception module. It can serve as the judgment object for triggering the tower processing mode and the target reference for bypassing and passing through.

[0057] Among them, the preset safe distance threshold refers to the minimum safe space distance between the drone and the pole, which can be used as a quantitative criterion for determining whether the pole processing mode needs to be triggered, ensuring that the drone has enough time to switch flight modes before approaching the pole.

[0058] Among them, the pole handling mode refers to the flight control mode that is triggered when the drone approaches the pole, which is specifically used to handle obstacle avoidance and bypassing of the pole node. It enables the drone to switch from the normal state of following the guide line to the state of bypassing the pole as a reference target, ensuring that the drone safely passes through the pole area, such as bypassing or passing through.

[0059] Among them, the center point coordinates of the tower refers to the geometric center position coordinates of the tower instance in three-dimensional space. It can provide a spatial reference point for planning the bypass or traverse trajectory, enabling the trajectory planning algorithm to generate a smooth path around this point, bypassing one side or traversing from a specific direction.

[0060] Among them, the local occupancy grid map refers to a three-dimensional grid map constructed based on spatial depth point cloud data within a local spatial range around the tower, in which the spatial occupancy status is represented by grid cells. It can accurately depict the spatial occupancy area of ​​the tower body and its auxiliary equipment, providing high-precision collision constraint information for trajectory planning.

[0061] The trajectory reference point refers to the target coordinate point that serves as a spatial reference for path generation in the trajectory planning algorithm, providing spatial guidance for the destination direction of path generation.

[0062] Among them, the target trajectory algorithm refers to the trajectory planning algorithm that generates a safe flight path with the trajectory length, trajectory execution time and the repulsive potential field between the trajectory point and the obstacle as the optimization target. It can generate a flight trajectory with short time and smooth path while meeting the requirements of safe obstacle avoidance.

[0063] Among them, the repulsive potential field refers to the virtual repulsive force field calculated based on the spatial distance between the trajectory point and the obstacle. It can make the planned flight trajectory stay away from the area occupied by the obstacle. The closer the distance, the greater the repulsive force, thereby ensuring flight safety.

[0064] Among them, the tower flight path refers to the three-dimensional spatial flight trajectory generated by the tower node processing mode to fly around or pass through the tower, which can guide the UAV to safely and smoothly fly around one side of the tower or pass through the tower area from the direction of the conductor.

[0065] The second flight control command refers to the flight control command generated in the tower processing mode to control the UAV to fly along the tower's flight path. It can drive the UAV to switch from the main line tracking state to the tower detour state and resume main line tracking after passing the tower.

[0066] In this embodiment, the distance to the tower ahead is continuously monitored during the mainline tracking flight. When the distance is less than a preset safe distance threshold, the tower processing mode is automatically triggered. The coordinates of the tower's center point are determined, and a local occupancy grid map is constructed. A target trajectory algorithm is used to generate a bypass or passage path with the tower's center point as a reference point. A second flight control command is output to control the UAV to perform the bypass action. In this way, the system can automatically switch from the wire-following mode to the tower processing mode when the UAV approaches the tower. The local occupancy grid map accurately depicts the three-dimensional spatial occupancy area of ​​the tower and its auxiliary equipment. The trajectory optimization algorithm generates a safe and smooth bypass path based on the collision constraints of the occupancy grid map and the spatial reference of the tower's center point. This solves the problem of the UAV losing the tracking target due to wire interruption or sudden change in direction at the tower, ensuring the continuous and safe flight of the UAV throughout the entire line corridor.

[0067] As an optional embodiment, after controlling the UAV to fly according to the first flight control command, the method further includes: determining a second determination result based on component instance information to see if a line branch point has been identified; if the second determination result is that a line branch point has been identified, triggering a branch decision mode; in the branch decision mode, determining a target branch line to be tracked from multiple candidate branch lines, wherein the method of determining the target branch line includes: determining the target branch line according to a preset tracking rule, wherein the preset tracking rule includes one of the following: selecting the candidate branch line with the largest cross-section, selecting the candidate branch line with the smallest deviation from the current heading angle of the UAV, selecting the candidate branch line that matches the preset mission target; and / or, inputting the line feature data of multiple candidate branch lines into a decision model to determine the target branch line, wherein the decision model is a model trained based on historical inspection data; determining a third flight control command based on the target branch line, and controlling the UAV to fly according to the third flight control command.

[0068] Among them, the line branch point refers to the topological node location where multiple conductors converge or fork in an overhead line. It usually appears at T-junction poles or branch poles and can serve as a spatial discrimination basis for triggering the branch decision mode.

[0069] Among them, the branch decision mode refers to the decision mode triggered when the UAV identifies a branch point of the line, which is used to select the tracking target from multiple candidate branch conductors. It enables the UAV to automatically determine which branch line to continue tracking when it encounters a line fork, and realize autonomous traversal of the entire line.

[0070] Among them, candidate branch conductors refer to multiple traceable conductors that branch off at the line branching point, which can serve as the selection range for branching decisions, such as the main line conductor and the branch line conductor at the T-junction.

[0071] Among them, the target branch traverse refers to the traverse to be tracked selected from multiple candidate branch traverses through preset rules or learning models, which can provide the UAV with a clear direction for subsequent flight tracking.

[0072] Among them, the preset tracking rules refer to the branch selection judgment logic pre-set based on power grid topology conventions and inspection experience, which can quickly make reasonable branch selection decisions without training data, such as prioritizing the tracking of the main conductor with a thicker cross section.

[0073] Among them, the decision model refers to the branch selection model trained based on historical inspection data, which can learn the statistical characteristics and inspection preferences of the main line and provide data-driven intelligent judgment for branch decision-making.

[0074] The third flight control command refers to the flight control command generated in the branch decision mode to control the UAV to fly along the target branch line. It can drive the UAV to turn to the selected branch line at the branch point and continue to follow the main line.

[0075] In this embodiment, when the perception module identifies a branch point, it automatically triggers a branch decision mode. The target branch conductor is determined from candidate branch conductors using preset tracking rules or a decision model, and a third flight control command is generated to control the UAV to fly along the selected branch. In this way, the system can autonomously make tracking decisions when facing branch lines. The preset rule method utilizes power grid topology experience to quickly determine the direction of the main line, making correct decisions in most scenarios without requiring extensive training data. The decision model method learns the characteristics of the main line through historical inspection data, enabling the identification of more complex line direction patterns. These two methods complement each other, giving the UAV the ability to autonomously select paths in branch scenarios and avoiding the limitations of manual intervention or preset flight path methods.

[0076] As an optional embodiment, after controlling the UAV to fly according to the first flight control command, the method further includes: determining the target obstacle on the flight path; determining the target correction path corresponding to the target obstacle; determining the correction flight control command according to the target correction path, and controlling the UAV to fly according to the correction flight control command, so as to control the UAV to bypass the target obstacle.

[0077] Among them, target obstacles refer to objects detected on the flight path that may affect flight safety and can trigger obstacle avoidance procedures to correct the path.

[0078] Among them, the target correction path refers to the corrected flight path generated to deviate from the original tracking path in order to avoid the target obstacle. It can minimize the deviation from the original tracking path while ensuring safe obstacle avoidance.

[0079] Among them, the corrected flight control command refers to the flight control command generated based on the target corrected path to control the UAV to perform obstacle avoidance maneuvers, which can drive the UAV to fly along the corrected path and bypass obstacles.

[0080] In this embodiment, obstacles on the flight path are detected in real time during the mainline tracking flight. A target correction path is generated for each detected obstacle, and corrective flight control commands are output to guide the UAV to bypass the obstacle. In this way, the system can autonomously cope with various obstacles in complex route corridor environments, ensuring that the UAV's mainline tracking flight is unaffected by obstacles such as trees and intersections within the route corridor, thus improving the safety of autonomous tracking flight and its adaptability to environmental changes.

[0081] As an optional embodiment, determining the target correction path corresponding to the target obstacle includes: when the target obstacle includes a static obstacle, determining the target correction path for the static obstacle; and / or, when the target obstacle includes a dynamic obstacle, determining the obstacle prediction trajectory corresponding to the dynamic obstacle; determining the minimum predicted distance and collision time between the UAV and the dynamic obstacle based on the obstacle prediction trajectory; determining the collision risk probability based on the minimum predicted distance and collision time; when the collision risk probability exceeds a preset risk threshold, generating a target correction path that bypasses the dynamic obstacle using a target path planning algorithm, wherein the target path planning algorithm includes a target artificial potential field function, which includes a gravitational potential field term, a static obstacle repulsion potential field term, a dynamic obstacle repulsion potential field term, and a path-keeping term. The gravitational potential field term is used to guide the UAV to move along the original path, the static obstacle repulsion potential field term is used to maintain a safe distance between the UAV and the static obstacle, the dynamic obstacle repulsion potential field term is used to enable the UAV to avoid the dynamic obstacle based on the relative speed and collision time of the dynamic obstacle, and the path-keeping term is used to constrain the deviation of the correction path from the original path.

[0082] Static obstacles refer to obstacles whose positions remain fixed on the flight path, and can be directly used to generate detour paths using path replanning algorithms.

[0083] Dynamic obstacles refer to obstacles that have the ability to move along the flight path. It is necessary to predict their movement trajectory and assess the collision risk before an effective obstacle avoidance path can be generated.

[0084] Among them, obstacle prediction trajectory refers to the prediction result of the movement trajectory of dynamic obstacles over a period of time in the future, which can provide information on the movement trend of obstacles for collision risk assessment.

[0085] Among them, the minimum predicted distance refers to the minimum spatial distance between the drone and the dynamic obstacle within the prediction period, which can serve as a direct indicator for quantifying collision risk.

[0086] The collision time refers to the estimated time it would take for the drone to collide with a dynamic obstacle if it continues in its current state of motion, and it reflects the urgency of the collision risk.

[0087] Among them, the collision risk probability refers to the quantitative index of collision possibility calculated by combining the minimum predicted distance and the collision time, which can be used as the decision threshold for whether to trigger reactive obstacle avoidance.

[0088] Among them, the preset risk threshold refers to the pre-set critical value of the collision risk probability. When the value is exceeded, obstacle avoidance maneuvers are triggered, which can balance flight safety and obstacle avoidance maneuver frequency.

[0089] Among them, the target artificial potential field function refers to the total potential field function that introduces dynamic obstacle rejection and path preservation terms on the basis of the traditional artificial potential field method. It can generate the optimal obstacle avoidance path by integrating multiple objectives such as gravity guidance, static obstacle avoidance, dynamic obstacle avoidance and path preservation.

[0090] Among them, the gravitational potential field term refers to the virtual gravitational field generated by the original tracking path, which can guide the corrected path to return to the original tracking direction after obstacle avoidance.

[0091] Among them, the static obstacle repulsive potential field term refers to the virtual repulsive force field generated by the static obstacle, which can make the trajectory points in the corrected path move away from the area occupied by the static obstacle.

[0092] Among them, the dynamic obstacle repulsion potential field term refers to the dynamic virtual repulsion force field generated by the dynamic obstacle, whose intensity is related to the relative speed and collision time, which can enable the corrected path to move away from the dynamic obstacle in advance near the predicted collision point.

[0093] Among them, the path-keeping term refers to the potential field term that constrains the modified path to be as close as possible to the original tracking path, which enables the obstacle avoidance path to quickly return to the original route after the obstacle is bypassed, reducing the impact of obstacle avoidance maneuvers on normal tracking.

[0094] In this embodiment, obstacles are differentiated into static and dynamic types and processed separately. For static obstacles, a corrected path is directly generated. For dynamic obstacles, their trajectory is first predicted, and the collision risk probability is calculated based on the predicted trajectory. When the risk exceeds a preset threshold, a corrected path is generated using a target artificial potential field function. In this way, the system can adopt differentiated obstacle avoidance strategies for different types of obstacles. For static obstacles that do not require prediction, the path is directly replanned to improve response speed. For dynamic obstacles, trajectory prediction and collision risk assessment enable early avoidance, avoiding the lag problem of reacting only after the obstacle is close. In the target artificial potential field function, the gravitational potential field term ensures the ability to return to the original flight path after obstacle avoidance; the static repulsive potential field term maintains a safe distance from fixed obstacles; the dynamic repulsive potential field term is related to relative velocity and collision time, enabling the UAV to move away from the obstacle's trajectory near the predicted collision point in advance; and the path-keeping term constrains the obstacle avoidance path deviation to be minimized. The synergistic effect of these four potential fields minimizes the impact of the corrected path on the original tracking path while ensuring safety.

[0095] As an optional embodiment, after determining the third flight control command based on the target branch line and controlling the UAV to fly according to the third flight control command, the method further includes: generating traceable point information corresponding to the line branch point, wherein the traceable point information includes the branch point coordinate data of the line branch point; storing the traceable point information into a traceable point stack so that after a mission interruption, the traceable point information can be retrieved from the traceable point stack and the mission continuation operation can be performed based on the traceable point information.

[0096] Among them, the traceable point information refers to the set of key information recorded by the UAV at the branch point for resuming tracking after the mission is interrupted. It can provide the UAV with the spatial coordinate positioning basis for returning to the branch point after the mission is interrupted.

[0097] Among them, the branch point coordinate data refers to the three-dimensional spatial coordinate information of the branch points of the line, which can provide accurate spatial navigation targets for the subsequent execution of tasks.

[0098] Among them, the backtrackable point stack refers to a data stack structure that stores backtrackable point information in the order of recording time. It can quickly locate the most recent decision branch point by popping the most recently recorded backtrackable point information after the task is interrupted, so as to achieve accurate task backtracking.

[0099] Among them, the task continuation operation refers to the operation of resuming the unfinished inspection task based on the traceable point information after the task is interrupted, which enables the UAV to continue to perform the inspection from the interruption point or the nearest decision point.

[0100] In this embodiment, after a decision is made at a branch point, backtrackable point information containing the branch point's coordinates is automatically generated and stored in a backtrackable point stack. This allows the system to retrieve information from the stack and resume the task if the task is interrupted. In this way, the system can proactively record backtrackable point information at each branch decision, building a complete inspection decision history. When the task is interrupted due to obstacle avoidance, signal interruption, or insufficient power, there is no need to restart the inspection or rely on manual redeployment of the flight path. The system can directly pop the most recent backtrackable point information from the backtrackable point stack and autonomously navigate back to the nearest branch decision point to continue the unfinished inspection task, ensuring the continuity and integrity of the inspection task.

[0101] As an optional embodiment, after controlling the UAV to fly according to the first flight control command, the method further includes: when a mission interruption occurs, performing positioning based on the UAV's position coordinate data and attitude inertial sequence data to obtain interruption positioning data; controlling the UAV to fly to the mission interruption area based on the interruption positioning data; acquiring the current environmental image corresponding to the mission interruption area, matching the current environmental image with the image in the storage area to obtain target positioning data; determining the target backtracking point from multiple backtracking points included in the backtracking point stack based on the target positioning data; and controlling the UAV to fly to the target backtracking point.

[0102] Among them, mission interruption refers to an event in which the UAV's tracking flight is suspended or interrupted due to various reasons during the inspection flight, which can trigger the mission continuation process.

[0103] Interruption positioning data refers to the estimated current position of the UAV obtained by positioning based on position coordinate data and attitude inertial sequence data after the mission is interrupted. It can provide coarse-grained spatial navigation information for the UAV to fly to the mission interruption area.

[0104] The mission interruption area refers to the approximate spatial range where the UAV was located when the mission interruption occurred, which can serve as the initial search area for visual relocalization.

[0105] Among them, the stored area image refers to the visual image frame data of the explored area that is pre-stored during flight, which can be used as a matching reference image for visual relocalization.

[0106] Among them, target positioning data refers to the precise spatial positioning data obtained by matching the current environmental image with the image in the stored area, which can provide a precise spatial location reference for the selection of traceable points.

[0107] Among them, the target backtracking point refers to the specific backtracking point that needs to be returned from the backtracking point stack, which can provide a clear flight target for the continued execution of the mission.

[0108] In this embodiment, when the mission is interrupted, coarse positioning is first performed using position coordinate data and attitude inertial sequence data to guide the UAV to the mission interruption area. Visual repositioning is then achieved by matching features between the acquired environmental image and the stored area image. Based on the repositioning data, the target backtracking point is determined from the backtracking point stack, and the UAV is controlled to fly to that point to continue the inspection mission. In this way, the system can autonomously complete a two-stage positioning process from coarse to precise positioning after a mission interruption. Coarse positioning uses GPS and inertial measurement unit data to quickly guide the UAV back to the interruption area, while visual repositioning improves positioning accuracy to the centimeter level through image feature matching, ensuring precise positioning of backtracking points. This two-stage positioning strategy balances the efficiency and accuracy of mission recovery, ultimately achieving a seamless transition from mission interruption to mission resumption.

[0109] Based on the above embodiments and optional embodiments, an optional implementation method is provided, which is described in detail below.

[0110] Among related technologies, current UAV-based 10kV line inspection technology has significant limitations. Mainstream solutions rely on preset GPS waypoints, essentially performing "path playback" rather than "autonomous tracking," lacking understanding and autonomous decision-making capabilities regarding line branches, corners, and other topological structures. Their obstacle avoidance functions are mostly simple static reactions (stopping upon encountering an obstacle), struggling to handle dynamic obstacles and prone to deviating from the inspection corridor due to obstacle avoidance maneuvers. More importantly, the system lacks robustness: if the mission is interrupted due to signal loss, sensor failure, or emergency obstacle avoidance, the UAV cannot intelligently "resume execution from the breakpoint," typically only passively returning to base or hovering awaiting manual intervention, resulting in a complete interruption of the automated process and low inspection efficiency. Furthermore, existing methods lack a sophisticated, safe, smooth, and continuously locking strategy for traversing complex tower areas. These shortcomings mean that current technology remains highly dependent on human intervention, failing to meet the demands of large-scale, fully automated, and highly reliable intelligent inspections.

[0111] In view of this, an optional embodiment of the present invention provides a method for determining flight control commands, which can also be called a method for autonomous tracking and dynamic obstacle avoidance decision control of 10kV overhead lines, belonging to the field of power line inspection and UAV autonomous control technology. The present invention achieves stable autonomous tracking, dynamic obstacle avoidance, and interruption recovery of 10kV overhead lines by constructing a hierarchical intelligent control system.

[0112] Figure 2 This is a system architecture diagram provided by an optional embodiment of the present invention. Figure 3 This is a flowchart of the hybrid behavior model state machine provided in an optional embodiment of the present invention. Figure 4 This is a schematic diagram of the dynamic obstacle avoidance principle provided by an optional embodiment of the present invention, such as... Figure 2 As shown in 3, 4, etc. Figure 2 The overall system architecture of the method described in this invention is demonstrated. The system adopts a layered design, including a perception layer, a data processing layer, a decision control layer, and an execution layer. The perception layer acquires environmental information through multiple sensors such as LiDAR, visual cameras, and GNSS / IMU; the data processing layer performs point cloud processing, visual recognition, and multi-sensor fusion; the decision control layer includes three core modules: a hybrid behavior model, a dynamic obstacle avoidance module, and a memory map management module; the execution layer converts decision commands into flight control signals. A closed-loop control is formed between the layers, enabling intelligent inspection of 10kV overhead lines. Figure 3 The state transition logic of the hybrid behavior model is demonstrated. The system begins with initialization, entering search mode to locate the line; upon detecting the conductor, it switches to mainline tracking mode; during inspection, it intelligently switches between tower handling mode, branch decision mode, and mainline tracking mode based on environmental changes; in case of an emergency, it enters safety mode; and the system terminates upon mission completion. The state machine design ensures the UAV's adaptive behavior in different scenarios. Figure 4The system demonstrates its dynamic obstacle avoidance workflow. First, the perception module acquires environmental information and identifies obstacles through cluster analysis. The prediction module uses a motion model to predict trajectories and estimate uncertainties. The decision-making module assesses collision risk and determines thresholds. When the risk exceeds the threshold, the planning module initiates path optimization and trajectory generation using an artificial potential field method, ultimately outputting control commands. This process enables proactive prediction and intelligent avoidance of dynamic obstacles.

[0113] The specific steps of this invention are as follows:

[0114] (1) Tracking strategy based on hybrid behavior model:

[0115] To address the complexity of 10kV overhead line structures (such as branches, turns, and elevation changes), a hybrid behavior model was designed to enable UAVs to smoothly switch between different scenarios.

[0116] Mainline Tracking Mode: This is the basic flight mode for the UAV. The system uses the three-dimensional spatial vector of the guide line obtained by the front-end sensing as the reference path. The Flight Controller (FCU) adopts a proportional-integral-derivative and feedforward composite control (PID feedforward composite control) algorithm. By calculating the lateral deviation, altitude deviation, and heading angle deviation of the guide line relative to the UAV, it generates attitude control commands to ensure that the UAV always maintains a preset safe lateral distance and overhead following altitude from the guide line.

[0117] In this mode, the three-dimensional centerline vector of the conductor is calculated in real time by the sensing module. The reference path represents the three-dimensional centerline of the conductor in a 10kV overhead line. In "mainline tracking mode," the UAV uses this curve as the preset flight reference path, meaning it is expected to fly along this curve while maintaining a safe distance from the conductor. Here, Pline(s): the three-dimensional coordinate vector (column vector form) of a point on the conductor at an arc length s from the starting point. x(s), y(s), z(s): represent the north (or east), east (or north), and elevation coordinates of this point, respectively. They are functions of the arc length parameter s (similar to the above, where the three-dimensional spatial trajectory of the conductor's centerline is fitted based on the three-dimensional spatial position of the conductor instance in the component instance information, and this trajectory is used as the reference path).

[0118] Define the current pose of the drone as ,in This is the yaw angle.

[0119] The controller calculates the following error quantities:

[0120] lateral deviation : The horizontal and vertical distance from the UAV's center of mass to the nearest guideline reference point.

[0121] Height deviation The preset vertical following altitude difference between the drone and the guide wire.

[0122] Heading angle deviation ,in This represents the tangent direction at the reference point of the conductor.

[0123] The flight controller uses PID feedforward composite control to generate attitude commands:

[0124]

[0125] in, , It is a diagonal gain matrix. The feedforward control quantity based on the curvature of the guide wire is used to compensate for the tracking error caused by turning in advance. e represents the comprehensive tracking error of the UAV relative to the reference path (the three-dimensional center line of the guide wire) (which is composed of proportional control component, integral control component, derivative control component and feedforward control component based on the curvature of the guide wire, as mentioned above).

[0126] Tower Node Processing Mode: When an approaching tower is detected, the system triggers the node processing routine. The UAV will temporarily use the tower's center as a virtual waypoint and execute a smooth circumnavigation or passage trajectory. This process deeply integrates LiDAR point cloud data for real-time motion planning, ensuring an absolutely safe distance from the tower and its auxiliary equipment (such as crossarms and guy wires).

[0127] The specific processing mode is as follows: when the perception module identifies the target pole ahead, and the distance (distance between the drone and the target pole instance) is... When the (safety threshold) is reached, the system switches to this mode.

[0128] Center of the tower The virtual waypoints (same as the tower center point coordinates mentioned above) are combined with a real-time grid map of the tower's local occupancy, constructed using LiDAR point clouds. (Similar to the above-mentioned construction of a local occupancy grid map of the target tower instance based on spatial depth point cloud data), using the Temporal Elastic Band (TEB) or Model Predictive Control (MPC) algorithm (same as the target trajectory algorithm above), a smooth bypass / crossing trajectory is generated online through optimization. (Same as the above-mentioned tower flight path):

[0129]

[0130] in, For the trajectory length, For time, The repulsive potential field between the trajectory point and the obstacle. These are weighting coefficients to ensure trajectory safety and efficiency.

[0131] Branch decision and path selection: When a branch point (such as a T-junction or branching pole) is identified, the system enters the decision state.

[0132] Its decision-making logic is based on a model that combines a rule-based expert system with lightweight reinforcement learning:

[0133] Rule base: Prioritizes following power grid topology conventions, such as "following conductors with thicker cross-sections," "continuing along the existing direction," or "selecting specific branches based on preset task objectives." For example:

[0134] If a conductor with a thicker cross-section exists, then select it as the tracking target;

[0135] ELSEIF (There exists a guideline with the smallest deviation from the current heading angle) THEN selects it as the tracking target;

[0136] ELSEIF (Preset task objective is a specific branch) THEN selects the corresponding branch based on visual / RFID tags.

[0137] Learner: By learning from historical inspection data, the model can identify more likely trunk line features.

[0138] Based on the component instance information mentioned above, a second determination result is made regarding whether a line branch point has been identified. If the second determination result indicates that a line branch point has been identified, a branch decision mode is triggered. In the branch decision mode, a target branch line to be tracked is determined from multiple candidate branch lines. The method for determining the target branch line includes: determining the target branch line according to preset tracking rules, wherein the preset tracking rules include one of the following: selecting the candidate branch line with the largest cross-section, selecting the candidate branch line with the smallest deviation from the current heading angle of the UAV, or selecting the candidate branch line that matches the preset mission objective; and / or, inputting the conductor feature data of multiple candidate branch lines into the decision model to determine the target branch line, wherein the decision model is a model trained based on historical inspection data. Based on the target branch line, a third flight control command is determined, and the UAV is controlled to fly according to the third flight control command.

[0139] After making a decision, the system binds the high-precision GNSS coordinates, laser SLAM pose, and visual features of the starting point (branching point) of the selected branch and stores them in the stack, recording them as a traceable point.

[0140] (2) Multimodal perception-driven active obstacle avoidance and safety assurance:

[0141] The corridor environment contains numerous unstructured obstacles (such as trees, crossings, and temporary construction equipment), requiring the UAV to have dynamic obstacle avoidance capabilities. The process involves: identifying target obstacles along the flight path; determining the target correction path corresponding to the target obstacle; and based on the target correction path, determining the corrected flight control commands and controlling the UAV to fly according to these commands to avoid the target obstacle.

[0142] The system features dynamic obstacle trajectory prediction and reactive obstacle avoidance. It performs real-time clustering of laser point cloud and visual data to identify potential obstacles. For static obstacles, an improved A / B algorithm is employed. Alternatively, a fast random tree algorithm can be used for local path replanning; for dynamic obstacles (such as birds), linear Kalman filtering or nonlinear particle filtering is used to predict their trajectory and calculate the collision risk. Once the risk exceeds a threshold, the UAV will initiate reactive obstacle avoidance, generating a collision-free path with the smallest deviation from the original tracking path based on the artificial potential field method or vector field histogram method.

[0143] Specifically, this is implemented as intelligent avoidance of dynamic obstacles (such as birds) in the environment by the drone. The system first performs real-time clustering analysis on laser point cloud and visual data to distinguish between static and dynamic obstacles. For dynamic obstacles, an Interactive Multi-Model (IMM) filtering algorithm is used to predict their future trajectory. This algorithm integrates multiple motion models, such as uniform velocity and coordinated turning, and obtains the optimal predicted state through model probability weighting.

[0144]

[0145] in For the first The estimated state of the model, Let Xk be its time-varying probability. Xk is the final state estimation vector at time k after weighted fusion of probabilities from each model. It typically contains information such as the position, velocity, and acceleration of the dynamic obstacle, used for subsequent collision risk assessment and obstacle avoidance decisions. j is the model index, taking values ​​in the range 1, 2, ..., M. M is the total number of motion models used (e.g., uniform velocity model, uniform acceleration model, coordinated turning model, etc.). Let Mj be the state estimate of the j-th model at time k after independent filtering. Different models obtain estimates through their respective Kalman or particle filters, reflecting the target motion state under the assumptions of that model. Mj is the posterior probability of the j-th model at time k, and satisfies... This probability changes dynamically depending on how well the model matches the observed data at that moment. Models with a high degree of matching have higher weights, while those with a low degree of matching have lower weights. Probability updates are typically calculated based on the likelihood function.

[0146] Based on the predicted trajectory, calculate the minimum predicted distance between the drone and the obstacle. With collision time And assess the probability of collision risk. When the risk exceeds a set threshold, reactive obstacle avoidance is triggered.

[0147] The obstacle avoidance path planning employs an improved artificial potential field method, introducing dynamic repulsion and path-maintaining terms into the traditional potential field. The total potential field function is:

[0148]

[0149] The target artificial potential field function includes a gravitational potential field term, a static obstacle repulsion potential field term, a dynamic obstacle repulsion potential field term, a path-preserving term, and a dynamic repulsion term. Related to the relative velocity to the obstacle and the predicted collision time, this forces the drone to avoid it in advance; path-keeping item This ensures that the obstacle avoidance path deviates from the original tracking path to a minimum. This is achieved by solving for the negative gradient of the potential field. Generate control commands to drive the drone to fly along a safe and smooth new path, and autonomously return to track status after the risk is eliminated.

[0150] Similar to the above, when the target obstacle includes dynamic obstacles, the predicted trajectory of the dynamic obstacle is determined; based on the predicted obstacle trajectory, the minimum predicted distance and collision time between the UAV and the dynamic obstacle are determined; based on the minimum predicted distance and collision time, the collision risk probability is determined; when the collision risk probability exceeds a preset risk threshold, a target path planning algorithm is used to generate a target correction path to bypass the dynamic obstacle. The target path planning algorithm includes a target artificial potential field function, which includes a gravitational potential field term, a static obstacle repulsion potential field term, a dynamic obstacle repulsion potential field term, and a path-keeping term. The gravitational potential field term guides the UAV to move along the original path, the static obstacle repulsion potential field term maintains a safe distance between the UAV and the static obstacle, the dynamic obstacle repulsion potential field term enables the UAV to avoid the dynamic obstacle based on the relative speed and collision time of the dynamic obstacle, and the path-keeping term constrains the deviation of the correction path from the original path.

[0151] Multi-sensor redundancy safety monitoring: High-frequency attitude data of the aircraft provided by the IMU and low-frequency global pose data provided by visual-laser SLAM are deeply fused through complementary filters or Kalman filters to form an attitude-position estimation system. When the system detects GPS signal loss, visual tracking failure, or distance from the guide wire exceeding the safety tolerance, it immediately triggers a safety fault protection mode, such as hovering, reversing along the original path, or ascending to a safe altitude, and waits for remote commands or autonomously attempts to resume tracking.

[0152] (3) Autonomous repositioning and task continuation based on memory maps:

[0153] When the drone automatically identifies a 10kV line branch point or encounters mission interruption issues due to obstacle avoidance, signal interruption, or power alarm, it can automatically achieve a high level of mission-level autonomy.

[0154] Construction of the topological memory map: During flight, the system not only records the original point cloud map, but also simultaneously constructs a sparse topological semantic map. This map is stored in a graph structure, with nodes representing poles and branch points, edges representing tracked line segments, and includes multimodal sensing feature descriptors for key locations.

[0155] Global localization and relocalization: When the mission is interrupted and needs to be resumed, the UAV first uses GNSS and IMU for coarse localization. After reaching the approximate area, it acquires the current environment image through the activated visual sensor, and compares it with keyframes in the topological memory map through image retrieval and feature matching technology to achieve visual relocalization with centimeter-level accuracy.

[0156] As described above, when a mission interruption occurs, the system performs positioning based on the UAV's position coordinates and attitude inertial sequence data to obtain interruption positioning data; based on the interruption positioning data, the system controls the UAV to fly to the mission interruption area; it acquires the current environmental image corresponding to the mission interruption area, matches the current environmental image with the image in the stored area to obtain target positioning data; based on the target positioning data, it determines the target backtracking point from multiple backtracking points included in the backtracking point stack; and it controls the UAV to fly to the target backtracking point.

[0157] Task backtracking and continuation: After successful relocation, the UAV autonomously navigates to the last recorded backtrackable point (such as a branch point). Subsequently, the system pops the context information of that point from the stack, reinitializes the tracker, and seamlessly switches to "mainline tracking mode" or "branch decision mode" to continue executing the unfinished inspection task until the entire route topology traversal is completed.

[0158] Similar to the above, generate traceable point information corresponding to the line branch points, wherein the traceable point information includes the branch point coordinate data of the line branch points; store the traceable point information into the traceable point stack so that after the task is interrupted, retrieve the traceable point information from the traceable point stack and perform the task continuation operation based on the traceable point information.

[0159] The "Autonomous Tracking and Decision Control" module of this system utilizes a hybrid behavior model, an active obstacle avoidance mechanism, and topological memory. Figure 3 The core technology enables the construction of an autonomous aerial agent capable of understanding power grid topology semantics, coping with complex corridor environments, and recovering from outages. This not only ensures the integrity and continuity of the 10kV overhead line automatic tracking mission but also greatly enhances the robustness and practical value of the entire system in real-world, complex scenarios.

[0160] The above optional implementation methods can achieve at least the following beneficial effects:

[0161] 1) Achieved true autonomous line recognition and continuous operation:

[0162] Through a "hybrid behavior model," drones can intelligently distinguish and handle different scenarios such as mainline tracking, tower crossing, and branch line decision-making, much like experienced line inspectors. They no longer need to pre-record waypoints; instead, they use the real-time perceived conductor itself as a dynamic reference path, achieving "what you see is what you fly." At branch points, a decision-making model combining rules and learning can autonomously select the correct path, ensuring long-distance, uninterrupted continuous inspection capabilities on complex power lines, greatly reducing the need for manpower.

[0163] 2) Safe passage capability in dynamic environments has been built:

[0164] This invention, through multimodal perception fusion and trajectory prediction, enables the system not only to avoid static obstacles but also to predict the movement trends of dynamic targets such as birds. The improved artificial potential field method innovatively introduces a guiding potential along a guide wire, building upon traditional repulsive and target attractive forces. This ensures that the UAV can navigate around obstacles with minimal deviation and automatically return to the tracking path. This achieves an optimal balance between "safe obstacle avoidance" and "stable tracking," significantly improving the reliability and safety of operations in unstructured corridors such as tree-covered areas and crossing zones.

[0165] 3) It endows the system with task-level self-healing and continued execution capabilities:

[0166] This invention establishes a complete task context memory by constructing a topological semantic map and backtracking stack online with multimodal features. When a task is interrupted due to signal loss, power alarm, or manual pause, the UAV can accurately "find" its position in the power grid topology using visual relocation technology and autonomously navigate to the last decision point, seamlessly continuing the unfinished inspection task. This capability transforms the inspection process from a fragile "one-shot" link into a robust closed loop capable of handling complex real-world interference, greatly improving the practicality of fully automated inspection.

[0167] 4. Improved the safety and efficiency of critical node operations:

[0168] For high-risk areas like power poles and towers, this invention provides a refined and adaptive solution. By fusing precise geometric information from laser point clouds and employing an online trajectory optimization algorithm, it can plan an optimized passage trajectory for the UAV in the power pole and tower area that is absolutely safe (precisely avoiding all crossarms and guy wires), smooth in motion (reducing sudden stops and turns), and prioritizes efficiency (shortest path). This replaces the original dangerous manual tower passage or inefficient large-scale detours, significantly improving the overall smoothness and efficiency of inspection operations while ensuring the safety of equipment and the UAV itself.

[0169] In summary, the optional embodiments of this invention create an aerial intelligent inspection system capable of understanding the environment, making autonomous decisions, responding safely, and recovering from interruptions. It not only addresses the core pain points of existing technologies, such as low automation, poor environmental adaptability, and insufficient robustness, but also lays a solid technical foundation for realizing unmanned, routine, and intelligent operation of power grid inspections, possessing extremely high engineering application value and promising prospects for industrial promotion.

[0170] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0172] Example 2

[0173] According to an embodiment of the present invention, an apparatus for implementing the above-described flight control command determination method is also provided. Figure 5 This is a structural block diagram of a flight control command determination device according to an embodiment of the present invention, such as... Figure 5 As shown, the device includes: an acquisition module 502, a first determination module 504, a second determination module 506, a third determination module 508, and a generation module 510. The device will be described in detail below.

[0174] The acquisition module 502 is used to acquire multimodal sensor data, which is collected by the sensor system carried by the UAV and includes spatial depth point cloud data. The first determination module 504, connected to the acquisition module 502, is used to determine component instance information based on the multimodal sensor data. The component instance information includes the three-dimensional spatial position and component category of the power component instance, including conductors and towers. The second determination module 506, connected to the first determination module 504, is used to fit and generate a three-dimensional centerline spatial trajectory of the conductor based on the three-dimensional spatial position of the conductor instance in the component instance information, and uses the three-dimensional centerline spatial trajectory of the conductor as a reference path. The third determination module 508, connected to the second determination module 506, is used to determine the position of the UAV corresponding to the reference path and the current pose of the UAV. Deviation data, including lateral deviation, altitude deviation, and heading angle deviation, wherein the lateral deviation is the horizontal and vertical distance from the UAV's center of mass to the nearest guideline reference point, the altitude deviation is the preset vertical following altitude difference between the UAV and the guideline, and the heading angle deviation is the angle difference between the tangent direction of the guideline at the reference point and the UAV's yaw angle; the generation module 510, connected to the aforementioned third determining module 508, is used to input the deviation data to the target composite controller, generate a first flight control command, and control the UAV to fly according to the first flight control command, wherein the first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the curvature of the guideline, and the first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guideline, and to fly along the spatial extension direction of the guideline.

[0175] It should be noted here that the above-mentioned acquisition module 502, first determination module 504, second determination module 506, third determination module 508 and generation module 510 correspond to steps S102 to S110 in the flight control command determination method. The multiple modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiment 1.

[0176] Example 3

[0177] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to execute instructions to implement the flight control instruction determination method of any of the above embodiments.

[0178] Example 4

[0179] According to another aspect of the present invention, a computer-readable storage medium is also provided, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the flight control command determination method described above.

[0180] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0181] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0185] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0186] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A flight control command determination method characterized by comprising: include: Acquire multimodal sensor data, wherein the multimodal sensor data is collected by a sensor system carried by the UAV, including: spatial depth point cloud data; Based on the multimodal sensor data, component instance information is determined, wherein the component instance information includes the three-dimensional spatial location and component category of the power component instance, and the component category includes: conductor, tower; Based on the three-dimensional spatial position of the conductor instance in the component instance information, a three-dimensional centerline spatial trajectory of the conductor is fitted and generated, and the three-dimensional centerline spatial trajectory of the conductor is used as a reference path; Based on the reference path and the current pose of the UAV, the deviation data corresponding to the UAV is determined. The deviation data includes lateral deviation, altitude deviation and heading angle deviation. The lateral deviation is the horizontal and vertical distance from the center of mass of the UAV to the nearest guide reference point. The altitude deviation is the preset vertical following altitude difference between the UAV and the guide. The heading angle deviation is the angle difference between the tangent direction of the guide at the reference point and the yaw angle of the UAV. The deviation data is input to the target composite controller to generate a first flight control command, and the UAV is controlled to fly according to the first flight control command. The first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the curvature of the guide wire. The first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guide wire, and to fly along the spatial extension direction of the guide wire.

2. The method of claim 1, wherein, After controlling the drone to fly according to the first flight control command, the system further includes: Based on the component instance information, determine whether the distance between the drone and the target tower instance is less than a preset safe distance threshold, and obtain a first determination result; If the first determination result is that the distance is less than the preset safe distance threshold, the tower processing mode is triggered. In the tower processing mode, the coordinates of the tower center point of the target tower instance are determined, and a local occupancy grid map of the target tower instance is constructed based on the spatial depth point cloud data. A target trajectory algorithm is adopted, with the center point coordinates of the tower as the trajectory reference point and the target variable as the optimization objective, to generate the tower flight path. The target variable includes the trajectory length, trajectory execution time, and repulsive potential field between the trajectory point and the obstacle. The obstacle is represented by the occupied grid in the local occupied grid map. Based on the pole's flight path, a second flight control command is determined, and the UAV is controlled to fly according to the pole's flight path.

3. The method of claim 1, wherein, After controlling the drone to fly according to the first flight control command, the system further includes: Based on the component instance information, a second determination result is made as to whether a line branch point has been identified; If the second determination result is that a branch point of the line has been identified, the branch decision mode is triggered; In the branch decision mode, a target branch line to be tracked is determined from multiple candidate branch lines. The method for determining the target branch line includes: determining the target branch line according to a preset tracking rule, wherein the preset tracking rule includes one of the following: selecting the candidate branch line with the largest cross-section, selecting the candidate branch line with the smallest deviation from the current heading angle of the UAV, selecting the candidate branch line that matches the preset mission target; and / or, inputting the wire feature data of the multiple candidate branch lines into a decision model to determine the target branch line, wherein the decision model is a model trained based on historical inspection data. Based on the target branch guideline, a third flight control command is determined, and the UAV is controlled to fly according to the third flight control command.

4. The method of claim 1, wherein, After controlling the drone to fly according to the first flight control command, the system further includes: Identify target obstacles along the flight path; Determine the target correction path corresponding to the target obstacle; Based on the target correction path, a corrected flight control command is determined, and the UAV is controlled to fly according to the corrected flight control command in order to control the UAV to bypass the target obstacle.

5. The method of claim 4, wherein, Determining the target correction path corresponding to the target obstacle includes: In the case where the target obstacle includes a static obstacle, determine a target correction path relative to the static obstacle, and / or, When the target obstacle includes a dynamic obstacle, determine the predicted obstacle trajectory corresponding to the dynamic obstacle; Based on the predicted obstacle trajectory, determine the minimum predicted distance and collision time between the UAV and the dynamic obstacle; The collision risk probability is determined based on the minimum predicted distance and the collision time. When the collision risk probability exceeds a preset risk threshold, a target path planning algorithm is used to generate a target corrected path to bypass the dynamic obstacle. The target path planning algorithm includes a target artificial potential field function, which includes a gravitational potential field term, a static obstacle repulsion potential field term, a dynamic obstacle repulsion potential field term, and a path-keeping term. The gravitational potential field term guides the UAV to move along the original path. The static obstacle repulsion potential field term maintains a safe distance between the UAV and the static obstacle. The dynamic obstacle repulsion potential field term enables the UAV to avoid the dynamic obstacle based on the relative speed of the dynamic obstacle and the collision time. The path-keeping term constrains the deviation of the corrected path from the original path.

6. The method of claim 3, wherein, Based on the target branch guideline, after determining the third flight control command and controlling the UAV to fly according to the third flight control command, the process further includes: Generate traceable point information corresponding to the line branch point, wherein the traceable point information includes the branch point coordinate data of the line branch point; The backtrackable point information is stored in the backtrackable point stack so that after a task interruption, the backtrackable point information is retrieved from the backtrackable point stack and the task continuation operation is performed based on the backtrackable point information.

7. The method according to any one of claims 1 to 6, characterized in that, After controlling the drone to fly according to the first flight control command, the system further includes: In the event of a mission interruption, the location is determined based on the UAV's position coordinate data and attitude inertial sequence data to obtain interruption location data; Based on the interruption location data, control the UAV to fly to the mission interruption area; Obtain the current environment image corresponding to the task interruption area, and match the current environment image with the image in the storage area to obtain target positioning data; Based on the target location data, the target backtracking point is determined from multiple backtracking points included in the backtracking point stack; Control the drone to fly to the target backtracking point.

8. A flight control command determination device, characterized in that, include: The acquisition module is used to acquire multimodal sensor data, wherein the multimodal sensor data is collected by the sensor system carried by the UAV, including spatial depth point cloud data; The first determining module is used to determine component instance information based on the multimodal sensor data, wherein the component instance information includes the three-dimensional spatial location and component category of the power component instance, and the component category includes: conductor, tower; The second determining module is used to fit and generate a three-dimensional centerline spatial trajectory of the conductor based on the three-dimensional spatial position of the conductor instance in the component instance information, and use the three-dimensional centerline spatial trajectory of the conductor as a reference path. The third determining module is used to determine the deviation data corresponding to the UAV based on the reference path and the current pose of the UAV. The deviation data includes lateral deviation, altitude deviation and heading angle deviation. The lateral deviation is the horizontal and vertical distance from the center of mass of the UAV to the nearest guide reference point. The altitude deviation is the preset vertical following altitude difference between the UAV and the guide. The heading angle deviation is the angle difference between the tangent direction of the guide at the reference point and the yaw angle of the UAV. The generation module is used to input the deviation data to the target composite controller, generate a first flight control command, and control the UAV to fly according to the first flight control command. The first flight control command consists of a proportional control component, an integral control component, a derivative control component, and a feedforward control component based on the curvature of the guide wire. The first flight control command is used to control the UAV to maintain a preset safe lateral distance and an upward following altitude from the guide wire, and to fly along the spatial extension direction of the guide wire.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the flight control instruction determination method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the flight control instruction determination method as described in any one of claims 1 to 7.

Images