Substation autonomous unmanned aerial vehicle self-evolution method based on training integrated platform

Abstract

The invention relates to the technical field of power system health monitoring, and discloses a substation autonomous unmanned aerial vehicle self-evolution method based on a training integrated platform, which comprises the following steps: establishing an edge computing and fog computing collaborative framework; training an autonomous obstacle avoidance and navigation strategy of the unmanned aerial vehicle by using airborne sensor data and environment information through a navigation algorithm based on deep learning; a privilege learning and inverse reinforcement learning method is adopted, autonomous learning is carried out, and different environment changes are adapted; and cross-domain migration from simulation to reality is realized by using data of a simulation environment and a real environment. The automation level and accuracy of substation unmanned aerial vehicle inspection can be effectively improved, the safety and stability of a substation are enhanced, and powerful support is provided for efficient operation and maintenance of a power system.

Description

Technical Field

[0001] The present invention relates to the technical field of power system health monitoring, and in particular to a substation autonomous unmanned aerial vehicle self-evolution method based on an operation and training integrated platform. Background Art

[0002] Existing autonomous drone navigation and inspection technologies face multiple challenges when applied in complex substation environments. For example, drones often need to fly in narrow and crowded spaces. This environment limits the maneuverability of drones, making flight path planning and real-time adjustment particularly difficult. Existing navigation systems usually rely on preset routes, which may not be able to cope with sudden obstacles or environmental changes in actual operation, resulting in the inability of drones to effectively complete inspection tasks.

[0003] In addition, the environment of substations usually has high temperature and humidity, and traditional drone technology is generally designed with only standard operating environments in mind. It is not adaptable to high temperature and high humidity environments, which can easily lead to equipment failures or inaccurate inspection data. Existing drone technology usually requires a lot of human intervention when dealing with complex environments. These technologies often rely on manually set flight routes and obstacle avoidance strategies, but in dynamically changing environments such as substations, the effect of human intervention is limited and does not have high flexibility and adaptability.

[0004] Therefore, there is an urgent need for a drone technology that can autonomously learn and adapt to different environments. This technology should enable drones to have intelligent perception, real-time environmental analysis and adaptive flight capabilities, thereby improving the efficiency and reliability of autonomous inspections by drones in substation environments. This technology should combine advanced artificial intelligence algorithms, environmental perception technology and adaptive control strategies to overcome the limitations of traditional methods and provide more intelligent and adaptive drone solutions. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a substation autonomous UAV self-evolution method based on an integrated operation and training platform. By establishing an advanced computing architecture, efficient processing of UAV data and dynamic allocation of tasks are achieved, ensuring the smooth completion of inspection tasks.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A substation autonomous UAV self-evolution method based on an integrated operation and training platform, comprising:

[0008] Layered computing architecture: The edge computing unit is integrated on the UAV platform, and a pruned and quantized lightweight deep learning model is deployed on the edge computing unit; the fog computing node is set on the ground station, communication base station or edge server; the edge computing unit is responsible for the collection and preliminary processing of real-time data, and the fog computing node analyzes and calculates the data preliminarily processed by the edge computing unit;

[0009] Cross-domain autonomous navigation based on privileged learning: This is a more short-term strategy. It uses privileged learning to train the drone navigation strategy to predict the future flight trajectory of the drone and enable it to traverse obstacles.

[0010] Inverse reinforcement learning self-evolution navigation: This is a strategy that is more inclined to long-term learning and evolution. In inverse reinforcement learning, experts control drones to conduct inspections, obtain demonstration data of drone flights, learn the expert's reward structure through the demonstration data, establish a reward function that adapts to different environments, and then train the imitation strategy through reinforcement learning based on the reward function, that is, learn the drone navigation behavior from the expert's demonstration data, and transform the behavior into an autonomous decision-making process, ultimately achieving safe flight of the drone;

[0011] The cross-domain autonomous navigation and inverse reinforcement learning self-evolution navigation are both run on fog computing nodes, and the cross-domain autonomous navigation and inverse reinforcement learning self-evolution navigation are performed alternately to enhance the navigation capability of the UAV; they play a complementary role in the system, the cross-domain autonomous navigation based on privileged learning can provide an instant and high-precision navigation path, and the inverse reinforcement learning self-evolution navigation provides the UAV with long-term autonomous adaptation and optimization capabilities.

[0012] Furthermore, the privileged learning method is used to train the drone navigation strategy to predict the future flight trajectory of the drone and enable the drone to traverse obstacles, specifically including:

[0013] In the privileged learning process, the privileged expert uses a sampling-based motion planning algorithm to obtain the three-dimensional point cloud data of the drone's state and environment; a set of collision-free trajectories τ generated by the privileged expert is used as the expected state of the drone in the next second; the collision-free trajectories are sampled based on probability distribution, and the encoding of the probability distribution represents the distance to the obstacle and the degree of proximity to the reference trajectory;

[0014] Among them, the trajectory cost function is defined as:

[0015]

[0016] Among them, λ1 represents the weight of the control reward item, R represents the reward item, WTA(T e ,T n ) represents the time-based weight function, T e represents the expected trajectory, Tn represents the current actual trajectory, λ2 represents the weight of the control collision cost term, c k represents the expected path point at time step k, τ n represents the state of the drone at time step k, t represents the time it takes for the drone to pass through time step k, and λ c =1000; Represents the collision cost, which is an indicator of the distance between the drone and the center point C of the obstacle; the drone is modeled as a radius of r q sphere, and define the collision cost as d c The truncated quadratic function of ;

[0017] During testing, the predicted trajectory is projected into the polynomial space to retrieve the complete state information. Define the projection polynomial P x (t), is solved by the following optimization problem:

[0018]

[0019] Among them, P x (t k′ ) is the projection polynomial, x pred (t k′ ) is the predicted trajectory point, n represents the number of points sampled in the time series, t k′ represents the time corresponding to the k′th sampling point, a i Denotes the projection polynomial P x (t) coefficients; reducing speed variations in flight by constraining the mean speed of the projection polynomial to a desired value;

[0020] Finally, the predicted trajectory with the minimum input cost is selected as the future flight trajectory of the UAV.

[0021] Compared with the prior art, the beneficial technical effects of the present invention are:

[0022] 1. Hierarchical computing architecture: The hierarchical processing mechanism combining edge computing and fog computing achieves a balance between real-time response and global optimization. The edge computing unit runs lightweight models on the drone for real-time reasoning, while the fog computing node is responsible for complex computing tasks.

[0023] 2. Privileged learning for cross-domain navigation: The privileged learning method is introduced to guide the autonomous learning of UAVs through the demonstration trajectories provided by expert strategies, thereby improving the autonomous navigation capabilities of UAVs in complex environments.

[0024] 3. Self-evolution capability of inverse reinforcement learning: Inverse reinforcement learning algorithms can learn reward structures from expert demonstrations, enabling drones to imitate the behavior of human experts, adapt to different environments, and enhance their self-evolution capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a general flow chart of a substation autonomous UAV self-evolution method based on an operation and training integrated platform provided according to an embodiment of the present invention;

[0026] Figure 2 is an algorithm diagram of cross-domain autonomous navigation based on privileged learning provided according to an embodiment of the present invention;

[0027] Figure 3 is a flow chart of the Sim2Real method provided according to an embodiment of the present invention;

[0028] Figure 4 is a flow chart of a domain adaptation paradigm method provided according to an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of an inverse reinforcement learning method provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0031] 1. Collaborative framework based on edge computing and fog computing:

[0032] In the implementation process of the present invention, in order to improve the real-time performance and reliability of the drone inspection system, the present invention introduces intelligent edge computing and fog computing architecture, which specifically includes the following aspects:

[0033] Integrate edge computing units (such as low-power processors, FPGAs, or TPUs) on drone platforms. These units are responsible for real-time processing and analysis of data from drone sensors, including video streams, environmental perception data, and flight status information.

[0034] Secondly, we deploy optimized lightweight deep learning models on edge computing units for real-time reasoning. These lightweight deep learning models are optimized through pruning and quantization to adapt to the computing resource limitations of edge devices, ensuring efficient computing performance while meeting accuracy requirements.

[0035] Fog computing nodes are set up on ground stations, communication base stations or edge servers. These fog computing nodes have strong computing power and storage resources. Assume that the computing power of the i-th fog computing node is C i The fog computing nodes are connected through the network to collect and process data. The goal is to maximize the utilization of computing resources U:

[0036]

[0037] Among them, Task iRepresents the current task load of the i-th fog computing node.

[0038] Furthermore, the edge computing unit and the fog computing node are combined into a collaborative framework. In the entire architecture, data processing is carried out in a hierarchical manner. The edge computing unit is responsible for the collection and preliminary processing of real-time data, while the fog computing node performs higher-level analysis and calculation. Through this hierarchical processing method, the system can achieve a balance between real-time response and global optimization.

[0039] The edge computing unit and the fog computing node synchronize data in real time through a high-speed data link to ensure data consistency and timely update. Especially in dynamically changing inspection tasks, fast status updates and information sharing are crucial to ensure the safety of drones and efficient execution of tasks, and to achieve high-efficiency, low-load, and low-loss performance improvements for the integrated training and operation platform.

[0040] 2. UAV cross-domain autonomous navigation algorithm based on privileged learning:

[0041] The present invention uses information from onboard sensors to train drone navigation strategies through privileged learning to predict future flight trajectories and help drones traverse obstacles in complex environments. The strategy output includes possible motion paths and collision risk estimates, and the model predictive controller tracks these paths by minimizing collision probabilities and input costs. Privileged learning enables drones to obtain accurate environmental structure and state information in a simulated environment, collect artificial demonstration data, generate unlimited training samples, and ensure the safety of the training process.

[0042] In privileged learning, the privileged expert uses a sampling-based motion planning algorithm. The privileged expert directly obtains the state and environment information of the platform, and both use the complete 3D map only in the simulated environment. The set of collision-free trajectories τ generated by the privileged expert represents the expected state of the drone in the next second: starting from the current state of the drone τ(0) = x, and sampling from a probability distribution P, which encodes the distance to the obstacle and the proximity to the reference trajectory. Collision-free trajectories The distribution reference trajectory τ ref and environmental structure as point cloud If it is far away from the obstacle and close to the reference object τ, the distribution reference trajectory τ ref The probability of occurrence is large, and P is defined as:

[0043]

[0044] in is the normalization factor, is a cost function that represents the proximity to the reference point and the distance to the obstacle.

[0045] Define the trajectory cost function L(T e ,T n )for:

[0046]

[0047] where λ c =1000, Q is the positive semidefinite cost matrix, The collision cost is a metric that measures the distance from the drone to the center of point C. The drone is modeled as a vehicle with a radius of r. q sphere, and define the collision cost as d c The truncated quadratic function of , i.e. the distance between the UAV vehicle and the closest point in the environment:

[0048]

[0049] In a complex power station environment, there may be obstacles that hinder the drone but can be avoided in many ways, so the path solution for the drone is often multimodal. The analytical calculation of P usually uses a random sampling method to reach an approximate density p.

[0050] To estimate P, the Metropolis–Hastings algorithm requires a target score function:

[0051]

[0052] where the definition c(·) is the cost of trajectory τ.

[0053] The student strategy accesses onboard sensor measurements to generate collision-free trajectories in real time. These measurements include depth images estimated using semi-global matching, platform velocity and attitude, and desired flight direction. The desired flight direction is a normalized vector toward a reference point 1 second into the future relative to the most recent reference state. It is assumed that this information is sufficient to generate the distribution The highest probability sample of

[0054] The input of this architecture is the depth image, platform velocity, drone attitude represented by a rotation matrix and reference direction, and a set of trajectories τ is predicted n and their relative collision costs, where c k ∈R + Unlike the privileged expert, the trajectory predicted by the network does not describe the complete state evolution but only its positional part: Network Trajectory for:

[0055]

[0056] Where p(t i)∈R 3 The drone is at t = t i The position at time θ relative to the current state x. This representation is more general than the b-spline with three control points used by the sampling-based planner, and in order to reduce the computational cost of interpolation at test time, a supervised learning method is used to train a neural network on the three trajectories with the lowest cost found by the expert.

[0057] During testing, the predicted trajectory is independently projected onto the polynomial space for each axis and the complete state information is retrieved from the predicted trajectory. This representation enforces the continuity of position, velocity, and acceleration with respect to the current state and promotes dynamic feasibility due to differential flatness. For example, when considering the x-axis, the polynomial projection is defined as The projection term a is obtained by solving the following optimization problem x :

[0058]

[0059] in is the x-component of element i th , is the position x of the drone and its derivative estimated from the current state. The latter corresponds to the true state when deployed in the simulation. In order to reduce the sudden changes in velocity during flight that lead to strong pitching motions, the average velocity of the polynomial is constrained to the desired value v des The polynomial μ x (t) time t is scaled by factors, Once all predicted trajectories are projected, one is selected for execution. This method ultimately selects c* / c k ≥0.95(c*=minc k ), the input cost is calculated according to the minimum acceleration method. Tracking the minimum input cost by the model predictive controller enforces the temporal continuity of the trajectory. For example, when avoiding an obstacle on the right, it is very unlikely to turn to the left in the next iteration.

[0060] In the above, the UAV can be simulated for training. Since the simulator has errors in modeling the physical environment, the optimal strategy learned in the simulation environment cannot be directly applied in the real environment. This method uses Sim2Real to solve this problem.

[0061] Sim2Real can easily generate a large amount of labeled data in a simulation environment without the need for data collection in the real world, which can greatly reduce the cost and risk of data collection. In Sim2Real technology, researchers tend to use simulators to train reinforcement training models, then deploy these models in real environments and use domain adaptation technology to transform simulation-based models into models suitable for real environments.

[0062] Domain adaptation methods use data from a source domain to improve the performance of a learning model in a different target domain, where data is not always directly available. The simulation environment built on the integrated training platform allows researchers to iterate and debug the model. Drones can perform multiple flight tests in a simulated environment to improve their navigation strategies without being affected by time, weather or other constraints.

[0063] Specifically, we formalize the domain adaptation scenario in a reinforcement learning setting. According to the definition of MDP, we denote the source domain as

[0064] D S =((S S ,A S ,P S ,R S );

[0065] The target domain is:

[0066] D T =(S T ,A T ,P T ,R T );

[0067] In reinforcement learning scenarios, due to the gap between perception and reality, the state spaces S of the source and target domains may be completely different, while the two domains share action spaces and transitions:

[0068] P(A S ≈A T P S ≈P T );

[0069] And their reward functions R have structural similarity

[0070] R S ≈R T ;

[0071] The main focus of the project is on tasks related to reinforcement learning and applied to drones, to generalize policies learned from synthetic data or to accelerate learning in real-world drones.

[0072] 3. Research on self-evolving navigation algorithm based on inverse reinforcement learning

[0073] Inverse reinforcement learning is the problem of inferring the reward function of an agent given its policy or observed behavior. Similar to reinforcement learning, inverse reinforcement learning is viewed as both a problem and a class of methods. Methods based on inverse reinforcement learning alternate between two main phases of learning: one phase uses demonstration data to infer the hidden reward or cost function, and the other phase uses reinforcement learning to learn an imitative policy based on the inferred reward function. The basic principle of inverse reinforcement learning is that inverse reinforcement learning chooses a reward function R to optimize the policy, and makes any reward function different from π E (a E ~π E ) action decision (a∈A\a E ), in which the greatest possible loss is incurred.

[0074] In inverse reinforcement learning, demonstration behaviors are performed by expert operators, such as manually controlling a drone to inspect or set inspection points. These demonstration behaviors are key to learning, providing mission information and helping the drone understand the mission goals and requirements. By learning the reward structure of the expert, the drone is able to take the right actions autonomously. This method has broad application prospects in achieving autonomous decision-making tasks, especially for applications in the current special project context.

[0075] The inverse reinforcement learning method is used to help drones cope with complex flight scenarios in simulators and actual environments. The core is to establish a reliable perception system to ensure accurate environmental information and provide key data for formulating correct flight strategies. Advanced sensor technology is used to obtain high-quality environmental data, including obstacles, equipment status, and environmental conditions. By recording human operation data, a reward function that adapts to different environments is established to guide drones to imitate human actions in complex scenarios. This method supports cross-domain migration from simulators to real environments, adapts to changes in different substation environments, and autonomously searches for inspection points to ensure that drones operate safely in a high-security power station environment, ultimately improving equipment stability and reliability and reducing maintenance costs and personnel risks.

[0076] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0077] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A self-evolution method for autonomous UAV in a substation based on an integrated operation and training platform, characterized in that: include: Layered computing architecture: Integrate edge computing units on the drone platform, and deploy pruned and quantized lightweight deep learning models on the edge computing units; The fog computing nodes are set up in ground stations, communication base stations or edge servers; the edge computing unit is responsible for the collection and preliminary processing of real-time data, and the fog computing nodes analyze and calculate the data preliminarily processed by the edge computing unit; Cross-domain autonomous navigation based on privileged learning: The UAV navigation strategy is trained through privileged learning method to predict the future flight trajectory of the UAV and enable the UAV to traverse obstacles; Inverse reinforcement learning self-evolution navigation: In inverse reinforcement learning, experts control drones to conduct inspections, obtain demonstration data of drone flight, learn the expert's reward structure through the demonstration data, establish a reward function that adapts to different environments, and then train the imitation strategy through reinforcement learning based on the reward function, that is, learn the drone navigation behavior from the expert's demonstration data, and transform the behavior into an autonomous decision-making process, ultimately achieving safe flight of the drone; The cross-domain autonomous navigation and inverse reinforcement learning self-evolution navigation are both run on the fog computing node, and the cross-domain autonomous navigation and inverse reinforcement learning self-evolution navigation are performed alternately to enhance the navigation capability of the UAV; Cross-domain autonomous navigation based on privileged learning can provide instant navigation paths, and inverse reinforcement learning self-evolutionary navigation provides drones with autonomous adaptability.

2. The substation autonomous UAV self-evolution method based on the training and operation integrated platform according to claim 1 is characterized in that: The privileged learning method is used to train the drone navigation strategy to predict the future flight trajectory of the drone and enable the drone to cross obstacles, specifically including: In the privileged learning process, the privileged expert uses a sampling-based motion planning algorithm to obtain the three-dimensional point cloud data of the drone's state and environment; a set of collision-free trajectories τ generated by the privileged expert is used as the expected state of the drone in the next second; the collision-free trajectories are sampled based on probability distribution, and the encoding of the probability distribution represents the distance to the obstacle and the degree of proximity to the reference trajectory; Among them, the trajectory cost function is defined as: Among them, λ1 represents the weight of the control reward item, R represents the reward item, WTA(T e ,T n ) represents the time-based weight function, T e represents the expected trajectory, T n represents the current actual trajectory, λ2 represents the weight of the control collision cost term, c k represents the expected path point at time step k, τ n represents the state of the drone at time step k, t represents the time it takes for the drone to pass through time step k, and λ c =1000; Represents the collision cost, which is an indicator of the distance between the drone and the center point C of the obstacle; the drone is modeled as a radius of r q sphere, and define the collision cost as d c The truncated quadratic function of ; During testing, the predicted trajectory is projected into the polynomial space to retrieve the complete state information; the projection polynomial is defined and solved by the following optimization problem: Among them, P x (t k′ ) is the projection polynomial, x pred (t k′ ) is the predicted trajectory point, n represents the number of points sampled in the time series, t k′ represents the time corresponding to the k′th sampling point, a i Denotes the projection polynomial P x (t) coefficients; reducing speed variations in flight by constraining the mean speed of the projection polynomial to a desired value; Finally, the predicted trajectory with the minimum input cost is selected as the future flight trajectory of the UAV.

Images