A motion planning method for smooth flight of a power inspection unmanned aerial vehicle

Abstract

The application provides a motion planning method for smooth flight of a power inspection unmanned aerial vehicle, which can provide smooth flight instructions for the unmanned aerial vehicle, improve inspection efficiency, improve global planning capability of the unmanned aerial vehicle, efficiently avoid electromagnetic field interference areas and obstacle areas, and the like. t = [l t ,d t ,g t ]; wherein l t is laser radar data; d t is obstacle kinematics information including a current position and a speed of the obstacle; and g t is a sub-target position; S2, a deep reinforcement learning network is trained using a proximal optimization strategy to obtain a control strategy, then an observation value is input into the control strategy network trained by reinforcement learning to obtain an action of flight of the unmanned aerial vehicle, and a corresponding reward function is designed according to the flight action; S3, whether the unmanned aerial vehicle flies to an inspection target position is judged, if not, the step S1 is repeated, and if yes, the method is ended.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, specifically to a motion planning method for smooth flight of a power line inspection UAV. Background Technology

[0002] Power line inspection is crucial for the safe operation of power systems, primarily involving the inspection of overhead lines, power poles, and related electrical equipment. However, in many mountainous regions of my country, numerous power poles are installed in complex terrains such as ravines and valleys, resulting in high-voltage transmission lines traversing diverse and challenging terrains. In such circumstances, manual inspection presents several drawbacks, including significant safety hazards, inconsistent accuracy in equipment inspections, and low efficiency.

[0003] With the development of drone technology, rotary-wing drones, due to their agility, are becoming an inevitable trend in power line inspection, replacing manual inspections. However, in the aforementioned power line inspection scenarios, drones not only need to avoid obstacles in the environment such as trees, buildings, and mountains, but also need to avoid dangerous areas and no-fly zones caused by the electromagnetic environment around power transmission lines. In addition, drones are often required to fly as smoothly as possible to ensure that onboard sensors can more accurately perceive the condition of power equipment.

[0004] Motion planning methods for drones ensure their safe and rapid flight to designated power pole locations while maintaining a safe distance from high-voltage transmission lines, making it a key technology for drone-based power line inspection. Existing motion planning methods typically use Rapid Random Tree (RRT)-based path search, followed by optimization to obtain a smooth trajectory. However, RRT-based methods only provide asymptotically optimal paths, leading to unnecessary drone travel and impacting inspection efficiency. While 3D graph search methods (such as A* and its improved versions) can find the shortest path, the computational burden is high due to the need to rasterize the power line inspection scene. Furthermore, for different power line inspection environments, professionals often need to readjust the drone motion planning algorithm parameters to suit the target environment. Conversely, reinforcement learning-based methods promise to provide end-to-end planning strategies. The method involves inputting the inspection task and environmental map of the UAV into the UAV, and then using reinforcement learning to train a control strategy that directly outputs the corresponding speed to control the UAV to perform the inspection task, thereby inspecting overhead lines, towers, and related power equipment. However, current reinforcement learning methods still have the following problems: 1) The reward function of reinforcement learning directly affects the quality of the trained control strategy, and in reality, it is often difficult to design a suitable reward function; 2) Due to local planning, the UAV may deadlock when there is a lack of a global guidance path; 3) When the power inspection scenario used for training differs greatly from the actual power inspection scenario, it is often necessary to train the control strategy to avoid electromagnetic interference areas and obstacle areas. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a motion planning method for smooth flight of a power line inspection drone. This method can provide the drone with stable flight commands, improve inspection efficiency, enhance the drone's global planning capabilities, and efficiently avoid electromagnetic interference areas and obstacle areas.

[0006] This invention adopts the following technical solution: a motion planning method for smooth flight of a power line inspection drone, characterized by comprising the following steps:

[0007] S1. Based on the environmental conditions of the power grid inspection using the UAV's sensors, the UAV obtains partial observations from the environment at time t. t =[l t d t g t ];

[0008] Among them, l t For lidar data; d t For obstacle kinematic information, including the obstacle's current position and velocity; g t Location of the sub-target;

[0009] S2. A deep reinforcement learning network is trained using a proximal optimization strategy to obtain a control policy. The observations are then input into the control policy network trained by reinforcement learning to obtain the drone's flight actions. And design corresponding reward functions based on flight maneuvers.

[0010]

[0011] in, This represents the forward velocity of the UAV in the coordinate system at time t;

[0012] This represents the yaw angle rotational speed in the UAV coordinate system at time t;

[0013] This represents the pitch angle and rotational speed in the UAV coordinate system at time t;

[0014] This represents the roll angle rotational speed in the UAV coordinate system at time t;

[0015] It signifies a reward for progress made toward a goal;

[0016] This indicates a punishment for passively approaching or colliding with an obstacle;

[0017] This indicates a rapid change in the direction of the penalty yaw angle;

[0018] This indicates a rapid change in the direction of the pitch angle as a penalty.

[0019] This indicates a rapid change in the direction of the roll angle as a penalty.

[0020] This indicates a reward for actively avoiding obstacles and moving towards the sub-target;

[0021] S3. Determine if the UAV has flown to the inspection target location. If not, repeat step S1; if yes, end. Further, before the observed values ​​are input into the control policy network trained through reinforcement learning, the formula... The observed value o t Perform maximum absolute scaling, where, and These represent the maximum and minimum ranging range values ​​of the lidar sensor, respectively.

[0022] Furthermore, reward value The value of is given by formula (2).

[0023]

[0024] Where, r goal This indicates that the distance between the current position of the drone and the final inspection target position is within a set distance g. m Reward value within the specified time;

[0025] r path This indicates the reward value for the drone during its approach to the inspection target location, even before it has reached the target location.

[0026] g m This indicates the threshold distance at which the drone approaches the inspection target.

[0027] t max This represents the maximum time it takes for the drone to fly from its current position at time t to the location of the inspection target.

[0028] This indicates the target position of the UAV at time t;

[0029] Furthermore, penalty value The value of is given by formula (3).

[0030]

[0031] Where, r collision This represents the penalty value when a drone collides with other objects in the inspection environment or when a drone flies into a no-fly zone caused by electromagnetic interference.

[0032] r obstacle This represents the penalty value when a drone gets too close to other objects in the environment or to a no-fly zone.

[0033] d r This indicates the distance at which a drone is judged to collide with an obstacle boundary or no-fly zone boundary in the environment;

[0034] d m This represents another distance value between the drone and the boundary of an obstacle or no-fly zone.

[0035] Indicates the center of an obstacle or no-fly zone;

[0036] Furthermore, penalty value The values ​​of are all given by formula (4).

[0037]

[0038] Where, r rotationThis indicates the penalty value for excessively rapid changes in the drone's yaw, pitch, or roll angles.

[0039] ω m Indicates the maximum permissible rate of angular change;

[0040] General The penalty value for any one of them;

[0041] General Any one of them;

[0042] Furthermore, reward value The value of is given by formula (5).

[0043]

[0044] Where, r angle This represents the reward value for speed flight obtained by solving the speed obstacle tracking mechanism for drones;

[0045] θ m This is the maximum permissible heading deviation;

[0046] The velocity vector v of the UAV at time t represents the actual flight velocity of the UAV. t With the desired collision avoidance velocity vector The angle between them.

[0047] The beneficial effects of this invention are that by inputting the observed values ​​into the control strategy trained using deep reinforcement learning, the changes in the yaw angle, pitch angle, and roll angle of the UAV are obtained. Through the setting of the corresponding reward function, the UAV can fly more stably during the inspection of power transmission lines, thereby ensuring that the perception data of the UAV's onboard sensors on the power transmission lines is more accurate and avoiding the situation where the inspection data has large noise due to flight vibration. In the power inspection scenario, it can more efficiently avoid electromagnetic interference areas and obstacle areas, and further improve the global planning capability of the UAV, which has good economic value. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating the present invention. Detailed Implementation

[0049] like Figure 1 As shown, the present invention provides a motion planning method for smooth flight of a power line inspection drone, comprising the following steps:

[0050] S1. Based on the environmental conditions of the power grid inspection using the UAV's sensors, the UAV obtains partial observations from the environment at time t.t =[l t d t g t ];

[0051] Among them, l t For LiDAR data;

[0052] d t This provides obstacle kinematic information, including the obstacle's current position and velocity; the velocity of a static obstacle is 0.

[0053] g t The sub-target locations are defined as follows: Considering that the UAV can traverse non-convex power inspection environments, this invention uses multiple sub-target points selected along the collision avoidance path to the final inspection target to guide the UAV flight.

[0054] And o t All data is represented in the drone's coordinate system;

[0055] Specifically, lidar data l t The combination of minimum pooling and average pooling is used to obtain the minimum and average values ​​of the sensing data obtained from the 10 laser scans of the UAV (which is actually the minimum and average values ​​of the sensing data obtained from the LiDAR scan). The resulting pooling data is then superimposed 4 times to form an 80×80 array.

[0056] S2, through formula The observed value o t Perform maximum absolute value scaling to normalize it to a value in the range [-1, 1];

[0057] in, and These represent the maximum and minimum ranging range values ​​of the lidar sensor, respectively.

[0058] Next, a deep reinforcement learning network (DRL Network) is trained using proximal policy optimization (PPO) to obtain the control policy. Then, the observations, after being scaled by maximum absolute value, are input into the control policy network trained by reinforcement learning to obtain the drone's flight actions. And design corresponding reward functions based on flight maneuvers.

[0059]

[0060] in, This represents the forward velocity of the UAV in the coordinate system at time t;

[0061] This represents the yaw angle rotational speed in the UAV coordinate system at time t;

[0062] This represents the pitch angle and rotational speed in the UAV coordinate system at time t;

[0063] This represents the roll angle rotational speed in the UAV coordinate system at time t;

[0064] It signifies a reward for progress made toward a goal;

[0065] This indicates a punishment for passively approaching or colliding with an obstacle;

[0066] This indicates a rapid change in the direction of the penalty yaw angle;

[0067] This indicates a rapid change in the direction of the pitch angle as a penalty.

[0068] This indicates a rapid change in the direction of the roll angle as a penalty.

[0069] This indicates a reward for actively avoiding obstacles and moving towards the sub-target;

[0070] The designed reward function enables drones to complete the inspection of power transmission lines in the shortest possible time, while avoiding collisions with any stationary or moving objects.

[0071] Specifically, reward value The value of is given by formula (2).

[0072]

[0073] Where, r goal This represents the reward value when the distance between the drone's current location and the final inspection target location is within 0.5m.

[0074] r path This indicates the reward value for the drone during its approach to the inspection target location, even before it has reached the target location.

[0075] g m This represents the threshold distance at which the drone approaches the inspection target location; when the distance between the drone and the inspection target location is less than g... m At that time, it indicates that the drone has reached the inspection target location;

[0076] t max This represents the maximum time it takes for the drone to fly from its current position at time t to the location of the inspection target. If the drone's flight time exceeds t... max If so, the flight is considered a failure;

[0077] This represents the target position of the UAV at time t (in the UAV coordinate system);

[0078] t represents a specific moment in the drone's operation, typically the current moment;

[0079] In this embodiment, r is set goal =20, r path =3.2, g m =0.5m,

[0080] v max This is the maximum flight speed of the drone when performing power line inspection tasks;

[0081] d max It is the maximum distance between the positions of the sub-targets;

[0082] Specifically, penalty value The value of is given by formula (3).

[0083]

[0084] Where, r collision This represents the penalty value when a drone collides with other objects in the inspection environment or when a drone flies into a no-fly zone caused by electromagnetic interference.

[0085] r obstacle This represents the penalty value when a drone gets too close to other objects in the environment or to a no-fly zone.

[0086] d r This indicates the distance at which a drone is judged to collide with an obstacle or no-fly zone boundary in the environment if it gets too close. It is set to 0.5m to avoid the drone from physically colliding with objects; it is essentially a safety margin.

[0087] d m This represents another distance value between the drone and the boundary of the obstacle or the no-fly zone. When the distance between the drone and the boundary of the obstacle or the no-fly zone is less than d... m When the time comes, a corresponding penalty value will be given;

[0088] This indicates the center position of the obstacle or no-fly zone at time t (in the UAV coordinate system);

[0089] In this embodiment, r is set collision =-20, r obstacle =-0.2,d r =0.5m, d m =1.2m;

[0090] Specifically, penalty value The values ​​of are all given by formula (4).

[0091]

[0092] Where, r rotation This indicates the penalty value for excessively rapid changes in the drone's yaw, pitch, or roll angles.

[0093] ω m Indicates the maximum permissible rate of angular change;

[0094] General The penalty value of any one of them, due to the penalty value The given formulas are the same, so the symbol * is used to refer to one of them in general;

[0095] General Since the formulas are the same, the symbol * refers to any one of them.

[0096] In this embodiment, r is set rotation = -0.1, and ω m = 1 rad / s;

[0097] rad is a unit of measurement in radians;

[0098] Specifically, reward value The value of is given by formula (5).

[0099]

[0100] Where, r angle This represents the reward value for speed flight obtained by solving the speed obstacle tracking mechanism (VO-3D) for unmanned aerial vehicles;

[0101] θ m This is the maximum permissible heading deviation;

[0102] The velocity vector v of the UAV at time t represents the actual flight velocity of the UAV. t With the desired collision avoidance velocity vector The angle between them;

[0103] In this embodiment, r is set angle =0.5,

[0104] Reward Value The key is to find the velocity in the desired direction toward the target while avoiding collisions. To achieve this, the present invention uses the concept of existing three-dimensional velocity obstacles (VO-3D) to solve for the velocity.

[0105] S3. Determine whether the drone has flown to the inspection target location. If not, repeat step S1. If yes, end.

[0106] Compared with existing technologies, this invention achieves the following advantages by employing sensor data combination and embedding a three-dimensional velocity obstacle concept (VO-3D) into the reward function:

[0107] 1) The control strategy obtained by deep reinforcement learning training used in this invention sets corresponding penalty terms for changes in yaw rotation angle, pitch angle and roll angle of the UAV. This enables the UAV to fly more stably during the inspection of power transmission lines, thereby ensuring that the sensing data of the UAV's onboard sensors on the power transmission lines are more accurate and avoiding the situation that the inspection data has large noise due to flight shaking.

[0108] 2) Compared to most reinforcement learning-based methods, this invention introduces a three-dimensional velocity obstacle mechanism (VO-3D) when designing the reward function. This mechanism can guide the UAV to actively avoid dynamic and static obstacles (such as buildings, trees, mountains, etc.) and no-fly zones caused by electromagnetic interference in the power grid inspection environment during flight. Furthermore, it improves the generalization of the control strategy obtained from reinforcement learning training, enabling the UAV to fly safely and avoid newly emerging obstacles even when there are local changes in the power grid inspection scenario. This further reduces the cost of map maintenance in the power grid inspection scenario.

[0109] 3) Compared with traditional methods, the control strategy of the present invention can achieve end-to-end control, that is, only the perception data of the UAV's onboard sensors need to be input into the control strategy to generate the UAV's flight speed and guide the UAV to maintain stable flight during the flight process.

[0110] 4) Since the present invention also establishes guidance for the sub-target position, in general inspection scenarios, even without global path guidance, the UAV can traverse most non-convex scenes using only its own control strategy.

[0111] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0112] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A motion planning method for smooth flight of a power inspection UAV, characterized in that, The method comprises the following steps: S1, according to the unmanned aerial vehicle sensor power inspection environment, the unmanned aerial vehicle obtains part of the observation value o from the environment at time t t = [l t ,d t ,g t ]; wherein, l t is the laser radar data; d t is the obstacle kinematics information, including the current position and speed of the obstacle; g t is the sub-target position; S2, training the deep reinforcement learning network using a proximal optimization strategy to obtain a control policy, and then inputting the observation value into the control policy network trained by reinforcement learning to obtain the action of the UAV flight and designing a corresponding reward function according to the flight action wherein, Vt represents the forward velocity of the UAV in the UAV coordinate system at time t; represents the yaw angle rotation velocity of the UAV at time t in the UAV coordinate system; represents the pitch angle rotation velocity in the UAV coordinate system at time t; represents the roll angle rotation velocity of the UAV at time t in the UAV coordinate system; represents a reward for making progress toward a goal; represents a penalty for passive approach or collision with an obstacle; represents a penalty for rapid changes in yaw angle direction; represents a penalty for rapid changes in pitch angle direction; represents a penalty for the roll angle direction rapid change; represents a reward for actively avoiding obstacles and moving towards the subgoal; S3, judging whether the UAV flies to the inspection target position, if not, repeating the step S1, if yes, ending. 2.The motion planning method for smooth flight of a power inspection UAV according to claim 1, wherein, Before the observation o is input to the control policy network trained by reinforcement learning, the observation o is scaled by the formula The observation o t is scaled by the maximum absolute value, where and respectively represent the maximum and minimum ranging range values of the lidar sensor. 3.The motion planning method for smooth flight of a power inspection UAV according to claim 1, wherein, reward value The value of the reward value is given by equation (2) wherein, r goal represents the reward value when the distance between the current position of the UAV and the final inspection target position is within the set distance g m . r path represents the reward value when the UAV has not yet arrived at the inspection target position but is in the process of approaching the inspection target position; g m a threshold distance representing the proximity of the UAV to the target location for inspection; t max represents the maximum time for the UAV to fly from the current position at time t to the inspection target position; represents the target position of the UAV at time t. 4.The motion planning method for smooth flight of a power inspection UAV according to claim 1, wherein, penalty value is given by equation (3) wherein, r collision represents the penalty value of the UAV colliding with other objects in the inspection environment or the penalty value when the UAV flies into the electromagnetic interference area. r obstacle represents a penalty value when the UAV is too close to other objects or no-fly zones in the environment; d r represents the judgment distance that the distance between the UAV and the boundary of the obstacle or the no-fly area in the environment is too close to cause a collision; d m another distance value representing a boundary of an obstacle or a no-fly zone; represents the center position of an obstacle or a no-fly zone.

5. The motion planning method for smooth flight of a power inspection UAV according to claim 1, wherein, penalty value The values of the parameters are given by equation (4) wherein r rotation represents a penalty value for the UAV yaw angle, pitch angle, or roll angle changing too fast; ω m denotes the maximum allowed angular velocity of change; penalty value of any one of the middle in general any one of 6.The motion planning method for smooth flight of a power inspection UAV according to claim 1, wherein, reward value The value of the reward value is given by equation (5) wherein r angle represents the reward value of the speed flight obtained by the UAV tracking speed obstacle mechanism solution; θ m is the maximum allowed heading deviation; represents the angle between the desired collision avoidance velocity vector t and the actual flight velocity vector v of the UAV at time t. represents the angle between the desired collision avoidance velocity vector

Images