Motion planning method for smooth flight of electric power inspection unmanned aerial vehicle

Abstract

The invention provides a motion planning method for smooth flight of an electric power inspection unmanned aerial vehicle, which can provide a stable flight instruction for the unmanned aerial vehicle, improve the inspection efficiency, improve the global planning capability of the unmanned aerial vehicle and efficiently avoid an electromagnetic field interference area and an obstacle area. Comprising the following steps: S1, according to an unmanned aerial vehicle sensor electric power inspection environment condition, an unmanned aerial vehicle obtains a part of observation values ot = [lt, dt, gt] from an environment at a t moment; wherein the lt is laser radar data; dt is kinematic information of the obstacle and comprises the current position and speed of the obstacle; gt is a sub-target position; s2, training a deep reinforcement learning network by using a near-end optimization strategy to obtain a control strategy, then inputting an observation value into a control strategy network obtained through reinforcement learning training, obtaining a flight action # imgabs0 # of the unmanned aerial vehicle, and designing a corresponding reward function according to the flight action; and S3, judging whether the unmanned aerial vehicle flies to the inspection target position or not, if not, repeating the step S1, and if yes, ending.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a motion planning method for smooth flight of an electric power inspection unmanned aerial vehicle. Background Art

[0002] Power inspection is crucial to the safe operation of the power system, mainly including the inspection of overhead lines, power towers and related power equipment. However, in mountainous areas of my country, many power towers are installed in mountain streams and gullies with complex terrain, resulting in complex and diverse terrains for high-voltage transmission lines. In this case, manual inspection has many disadvantages, such as: large safety risks of manual inspection, large fluctuations in the accuracy of power equipment inspection, and low efficiency of inspection operations.

[0003] With the development of drone technology, it is inevitable that rotary-wing drones will replace manual inspections in power inspection tasks due to their agility. However, in the above-mentioned power inspection scenarios, drones need to avoid obstacles in the environment such as trees, buildings, and mountains, as well as dangerous areas and no-fly areas caused by the influence of the electromagnetic environment around the transmission lines. In addition, drones are often required to fly as smoothly as possible to ensure that the onboard sensors can more accurately perceive the status of power equipment.

[0004] The motion planning method of UAV can ensure that the UAV flies safely and quickly to the designated power tower location, while maintaining a safe distance from the high-voltage transmission line during flight. It is one of the key technologies for realizing UAV power inspection. Existing motion planning methods usually use a fast random exploration tree based on RRT to search for paths, and then optimize based on feasible paths to obtain a smooth trajectory. However, the path obtained by the RRT-based method can only be asymptotically optimal, which will cause the UAV to fly unnecessary distances and affect the inspection efficiency of the UAV; although the method based on three-dimensional graph search (such as A* and its improved algorithm) can obtain the shortest distance path, it requires rasterization of the power inspection scene, resulting in a large amount of calculation; in addition, in the face of different power inspection environments, professionals are often required to readjust the parameters of the UAV motion planning algorithm to adapt to the target environment; on the contrary, the method based on reinforcement learning is expected to provide an end-to-end planning strategy that can be used. The inspection mission and environmental map of the drone are input into the drone, and the control strategy obtained through reinforcement learning training directly outputs the corresponding speed to control the drone to perform the inspection mission and realize the inspection of overhead lines, towers and corresponding power equipment. However, the current reinforcement learning method still has the following problems: 1) The reward function of reinforcement learning directly affects the quality of the trained control strategy, and it is often difficult to design a suitable reward function in reality; 2) Due to local planning, the drone may deadlock when there is a lack of global guidance path; 3) When the power inspection scenario used for training is very different from the actual power inspection scenario, it is often necessary to train the control strategy to avoid electromagnetic field interference areas and obstacle areas. Summary of the invention

[0005] In response to the above problems, the present invention provides a motion planning method for the smooth flight of a power inspection UAV, which can provide smooth flight instructions for the UAV, improve inspection efficiency, and enhance the global planning capability of the UAV, and can efficiently avoid electromagnetic field interference areas and obstacle areas.

[0006] The present invention adopts the following technical solution, a motion planning method for smooth flight of a power inspection UAV, characterized in that it includes the following steps:

[0007] S1. According to the power inspection environment of the drone sensor, the drone obtains some observation values ​​o from the environment at time t. t =[l t , d t , g t ];

[0008] Among them, l t is the laser radar data; d t is the kinematic information of the obstacle, including the current position and speed of the obstacle; g t is the sub-target position;

[0009] S2: Use the proximal optimization strategy to train the deep reinforcement learning network to obtain the control strategy, and then input the observation value into the control strategy network trained by reinforcement learning to obtain the UAV flight action. And design corresponding reward functions according to flight actions

[0010]

[0011] in, represents the forward velocity in the drone coordinate system at time t;

[0012] Represents the yaw rotation speed in the drone coordinate system at time t;

[0013] Indicates the pitch rotation speed in the drone coordinate system at time t;

[0014] represents the roll angle rotation speed in the drone coordinate system at time t;

[0015] It means rewarding progress toward a goal;

[0016] It means punishing passive approach or collision with obstacles;

[0017] It means to penalize the rapid change of yaw angle direction;

[0018] It means that the rapid change of pitch angle direction is penalized;

[0019] It means that the roll angle direction changes rapidly.

[0020] It indicates that the reward is to actively avoid obstacles and move toward the sub-goal;

[0021] S3, determine whether the drone has flown to the inspection target location, if not, repeat the step S1, if yes, then end. Further, before the observation value is input into the control strategy network trained by reinforcement learning, the formula The observed value o t Perform maximum absolute value scaling, where and Respectively represent the maximum and minimum ranging range values ​​of the LiDAR sensor;

[0022] Furthermore, the reward value The value of is given by formula (2),

[0023]

[0024] Among them, r goal Indicates that the distance between the current position of the drone and the final inspection target position is within the set distance g m Reward value when within

[0025] r path Indicates the reward value when the drone has not yet reached the inspection target location but is approaching the inspection target location;

[0026] g m Indicates the threshold distance of the drone approaching the inspection target location;

[0027] t max It represents the maximum time for the drone to fly from the current position at time t to the inspection target position;

[0028] represents the target position of the UAV at time t;

[0029] Furthermore, the penalty value The value of is given by formula (3),

[0030]

[0031] Among them, r collision Indicates the penalty value when the drone collides with other objects in the inspection environment or flies into a no-fly zone caused by electromagnetic field interference;

[0032] r obstacle Indicates the penalty value when the drone is too close to other objects or no-fly zones in the environment;

[0033] d r Indicates the distance at which the drone is judged to collide with obstacles or no-fly zones in the environment when the distance is too close.

[0034] d m Another distance value representing the distance between the drone and the obstacle boundary or the no-fly zone boundary;

[0035] Indicates the center position of an obstacle or a no-fly zone;

[0036] Furthermore, the penalty value The values ​​of are given by formula (4),

[0037]

[0038] Among them, r rotationIndicates the penalty value for the drone's yaw, pitch or roll angle changing too quickly;

[0039] ω m Indicates the maximum permissible angle change speed;

[0040] General The penalty value of any one of them;

[0041] General Any one of;

[0042] Furthermore, the reward value The value of is given by formula (5),

[0043]

[0044] Among them, r angle It represents the reward value of the speed flight obtained by solving the speed obstacle mechanism of the drone tracking;

[0045] θ m is the maximum permissible heading deviation;

[0046] The velocity vector v represents the actual flight speed of the drone at time t t and the desired collision avoidance velocity vector The angle between.

[0047] The beneficial effect of the present invention is that the observation values ​​are input into the control strategy obtained by deep reinforcement learning training to obtain the yaw rotation angle change, pitch angle change, and roll angle change of the UAV, and by setting the corresponding reward function, the UAV can fly more smoothly during the transmission line inspection process, thereby ensuring that the perception data of the UAV's onboard sensors detecting the transmission line is more accurate, avoiding the situation where there is large noise in the inspection data due to flight jitter, and more efficiently avoiding electromagnetic field interference areas and obstacle areas in the power inspection scenario, and further improving the global planning ability of the UAV, with good economic use value. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0049] like Figure 1 As shown, the present invention provides a motion planning method for smooth flight of a power inspection UAV, comprising the following steps:

[0050] S1. According to the power inspection environment of the drone sensor, the drone obtains some observation values ​​o from the environment at time t.t =[l t , d t , g t ];

[0051] Among them, l t For LiDAR data;

[0052] d t The kinematic information of the obstacle, including the current position and speed of the obstacle. The speed of a static obstacle is 0.

[0053] g t is the sub-target position; considering that the UAV can pass through the non-convex power inspection environment, the present invention uses a plurality of sub-target points selected along the collision avoidance path to the final inspection target to guide the UAV flight;

[0054] And o t All data in are expressed in the coordinate system of the drone;

[0055] Specifically, the lidar data l t It is a combination of minimum pooling and average pooling in the historical records of 10 laser scans of the drone (in fact, it is the perception data obtained by the laser radar scan, taking the minimum value and average value respectively), and then superimposing the obtained pooled data 4 times to finally form an 80×80 array;

[0056] S2, through the formula The observed value o t Perform maximum absolute value scaling and normalize it to a value in the range of [-1, 1];

[0057] in, and Respectively represent the maximum and minimum ranging range values ​​of the LiDAR sensor;

[0058] Then, the proximal policy optimization (PPO) strategy is used to train the deep reinforcement learning network (DRL Network) to obtain the control strategy. Then, the observations after maximum absolute value scaling are input into the control strategy network trained by reinforcement learning to obtain the UAV flight action. And design corresponding reward functions according to flight actions

[0059]

[0060] in, represents the forward velocity in the drone coordinate system at time t;

[0061] Represents the yaw rotation speed in the drone coordinate system at time t;

[0062] Indicates the pitch rotation speed in the drone coordinate system at time t;

[0063] represents the roll angle rotation speed in the drone coordinate system at time t;

[0064] It means rewarding progress toward a goal;

[0065] It means punishing passive approach or collision with obstacles;

[0066] It means to penalize the rapid change of yaw angle direction;

[0067] It means that the rapid change of pitch angle direction is penalized;

[0068] It means that the roll angle direction changes rapidly.

[0069] It indicates that the reward is to actively avoid obstacles and move toward the sub-goal;

[0070] The designed reward function enables the drone to complete the inspection of the transmission line in the shortest time while avoiding collision with any stationary or moving objects;

[0071] Specifically, the reward value The value of is given by formula (2),

[0072]

[0073] Among them, r goal Indicates the reward value when the distance between the current position of the drone and the final inspection target position is within 0.5m;

[0074] r path Indicates the reward value when the drone has not yet reached the inspection target location but is approaching the inspection target location;

[0075] g m Indicates the threshold distance of the drone approaching the inspection target location; when the distance between the drone and the inspection target location is less than g m When , it means that the UAV has reached the inspection target location;

[0076] t max It indicates the maximum time for the drone to fly from the current position at time t to the inspection target position. If the drone flight time exceeds t max , it is considered a flight failure;

[0077] represents the target position of the UAV at time t (in the UAV coordinate system);

[0078] t represents a certain moment of the drone, generally the current moment;

[0079] In this embodiment, r is set goal =20, r path =3.2, g m =0.5m,

[0080] v max It is the maximum flight speed of the UAV when performing power inspection tasks;

[0081] d max is the maximum distance between sub-goal positions;

[0082] Specifically, the penalty value The value of is given by formula (3),

[0083]

[0084] Among them, r collision Indicates the penalty value when the drone collides with other objects in the inspection environment or flies into a no-fly zone caused by electromagnetic field interference;

[0085] r obstacle Indicates the penalty value when the drone is too close to other objects or no-fly zones in the environment;

[0086] d r Indicates the distance at which the drone is too close to the boundary of an obstacle or a no-fly zone in the environment, resulting in a collision. It is set to 0.5m to avoid physical collision between the drone and the object. It is essentially a safety margin.

[0087] d m Indicates another distance value between the drone and the obstacle boundary or the no-fly zone boundary. When the distance between the drone and the obstacle boundary or the no-fly zone boundary is less than d m When , the corresponding penalty value is given;

[0088] Indicates the center position of the obstacle or no-fly zone at time t (in the drone coordinate system);

[0089] In this embodiment, r is set collision =-20, r obstacle =-0.2, d r =0.5m,d m =1.2m;

[0090] Specifically, the penalty value The values ​​of are given by formula (4),

[0091]

[0092] Among them, r rotation Indicates the penalty value for the drone's yaw, pitch or roll angle changing too quickly;

[0093] ω m Indicates the maximum permissible angle change speed;

[0094] General The penalty value of any one of The given formula is the same, so the symbol * is used to refer to one of them;

[0095] General Since the formula is the same, the symbol * refers to any one of them;

[0096] In this embodiment, r is set rotation =-0.1, and ω m =1rad / s;

[0097] rad is the unit of radians;

[0098] Specifically, the reward value The value of is given by formula (5),

[0099]

[0100] Among them, r angle It represents the reward value of speed flight obtained by solving the UAV tracking speed obstacle mechanism (VO-3D);

[0101] θ m is the maximum permissible heading deviation;

[0102] The velocity vector v represents the actual flight speed of the drone at time t t and the desired collision avoidance velocity vector The angle between

[0103] In this embodiment, r is set angle =0.5,

[0104] Reward Value The key is to find the speed in the desired direction towards the target while keeping no collision. To achieve this, the present invention uses the concept of velocity obstacles in three-dimensional space (Velocity Obstacles, VO-3D) to solve and obtain;

[0105] S3: Determine whether the UAV has flown to the inspection target location. If not, repeat step S1. If yes, end.

[0106] Compared with the prior art, the present invention uses a combination of sensor data and embeds the concept of speed obstacle in three-dimensional space (VO-3D) in the reward function, which can achieve the following advantages:

[0107] 1) The control strategy obtained by deep reinforcement learning training used in the present invention sets corresponding penalty items for the changes in the yaw rotation angle, pitch angle, and roll angle of the drone, which enables the drone to fly more smoothly during the inspection of the power transmission line, thereby ensuring that the perception data of the transmission line detected by the drone's onboard sensors is more accurate, avoiding the situation where the inspection data has large noise due to flight jitter;

[0108] 2) Compared with most reinforcement learning-based methods, the present invention introduces a three-dimensional speed obstacle mechanism (VO-3D) when designing the reward function, which can guide the UAV to actively avoid dynamic and static obstacles (such as buildings, trees, mountains, etc.) in the power inspection environment and no-fly zones formed by electromagnetic field interference during flight; and improves the generalization of the control strategy obtained by reinforcement learning training, so that the UAV can still fly safely and avoid new obstacles when there are local changes in the power inspection scene, further reducing the cost of map maintenance of the power inspection scene;

[0109] 3) Compared with the traditional method, the control strategy of the present invention can realize end-to-end control, that is, only by inputting the perception data of the drone's airborne sensors into the control strategy, the flight speed of the drone can be generated, and the drone can be guided to maintain a stable flight during the flight process;

[0110] 4) Since the present invention also provides guidance for the sub-target positions, in general inspection scenarios, the UAV can traverse most non-convex scenarios simply by using its own control strategy in the absence of global path guidance.

[0111] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0112] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A motion planning method for smooth flight of a power inspection UAV, characterized in that: The following steps are involved: S1. According to the power inspection environment of the drone sensor, the drone obtains some observation values ​​o from the environment at time t. t =[l t ,d t ,g t ]; Among them, l t is the laser radar data; d t is the kinematic information of the obstacle, including the current position and speed of the obstacle; g t is the sub-target position; S2: Use the proximal optimization strategy to train the deep reinforcement learning network to obtain the control strategy, and then input the observation value into the control strategy network trained by reinforcement learning to obtain the UAV flight action. And design corresponding reward functions according to flight actions in, represents the forward velocity in the drone coordinate system at time t; Represents the yaw rotation speed in the drone coordinate system at time t; Indicates the pitch rotation speed in the drone coordinate system at time t; It represents the rolling angle rotation speed in the drone coordinate system at time t; It means rewarding progress toward a goal; It means punishing passive approach or collision with obstacles; It means to penalize the rapid change of yaw angle direction; It means that the rapid change of pitch angle direction is penalized; It means that the roll angle direction changes rapidly. It indicates that the reward is to actively avoid obstacles and move toward the sub-goal; S3, determine whether the UAV has flown to the inspection target location, if not, repeat step S1, if yes, end.

2. According to claim 1, a motion planning method for smooth flight of a power inspection UAV is characterized in that: Before the observations are input into the control strategy network trained by reinforcement learning, the formula The observed value o t Perform maximum absolute value scaling, where and Respectively represent the maximum and minimum ranging range values ​​of the lidar sensor.

3. The motion planning method for smooth flight of a power inspection UAV according to claim 1 is characterized in that: Reward Value The value of is given by formula (2), Among them, r goal Indicates that the distance between the current position of the drone and the final inspection target position is within the set distance g m Reward value when within r path Indicates the reward value when the drone has not yet reached the inspection target location but is approaching the inspection target location; g m Indicates the threshold distance of the drone approaching the inspection target location; t max It represents the maximum time for the drone to fly from the current position at time t to the inspection target position; represents the target position of the UAV at time t.

4. The motion planning method for smooth flight of a power inspection UAV according to claim 1 is characterized in that: Penalty value The value of is given by formula (3), Among them, r collision Indicates the penalty value when the drone collides with other objects in the inspection environment or flies into a no-fly zone caused by electromagnetic field interference; r obstacle Indicates the penalty value when the drone is too close to other objects or no-fly zones in the environment; d r Indicates the distance at which the drone is judged to collide with obstacles or no-fly zones in the environment when the distance is too close. d m Another distance value representing the distance between the drone and the obstacle boundary or the no-fly zone boundary; Indicates the center of an obstacle or no-fly zone.

5. The motion planning method for smooth flight of a power inspection UAV according to claim 1 is characterized in that: Penalty value The values ​​of are given by formula (4), Among them, r rotation Indicates the penalty value for the drone's yaw, pitch or roll angle changing too quickly; ω m Indicates the maximum permissible angle change speed; General The penalty value of any one of them; General Any one of them.

6. The motion planning method for smooth flight of a power inspection UAV according to claim 1 is characterized in that: Reward Value The value of is given by formula (5), Among them, r angle It represents the reward value of the speed flight obtained by solving the speed obstacle mechanism of the drone tracking; θ m is the maximum permissible heading deviation; The velocity vector v represents the actual flight speed of the drone at time t t and the desired collision avoidance velocity vector The angle between.

Images