Method and system for automatically planning route of unmanned aerial vehicle for complex space of transformer substation

Abstract

The invention relates to the technical field of electric power, in particular to an unmanned aerial vehicle route automatic planning method and system for a complex space of a transformer substation. Analyzing at least one inspection task instruction input by a user to obtain a task type corresponding to each inspection task instruction and a data acquisition requirement corresponding to the task type; according to a preset three-dimensional semantic map of the transformer substation and the data acquisition requirement of each inspection task instruction, a flight route of at least one unmanned aerial vehicle is automatically generated by adopting a path planning algorithm, and the flight route comprises a waypoint sequence, a flight height, a shooting angle and a holder action instruction; the three-dimensional semantic map comprises no-fly zone information in the transformer substation, a plurality of devices, the position of each device, the device type, the electrification attribute and the safety rule label. And controlling the corresponding unmanned aerial vehicle to inspect target equipment in the transformer substation according to the flight route of each unmanned aerial vehicle. And large-scale and standardized inspection operation is realized.

Description

Technical Field

[0001] This invention relates to the field of power technology, and more specifically to an automatic flight path planning method and system for unmanned aerial vehicles (UAVs) in the complex spaces of substations. Background Technology

[0002] Substations are critical nodes in the power system, characterized by dense internal equipment, complex structures, harsh electromagnetic environments, and numerous high-voltage energized devices, placing extremely high demands on their safe operation. Traditional manual inspection methods suffer from low efficiency, high risk, and numerous blind spots. In recent years, drone technology has been introduced into substation inspection, but its application still faces the following challenges:

[0003] Environmental complexity: Substations are filled with equipment and have limited space, making them prone to collisions with drones.

[0004] Route planning dependence: Currently, it relies heavily on manual control by the pilot or preset fixed routes, which cannot adapt to different tasks and equipment, resulting in poor flexibility.

[0005] Insufficient safety: There is a lack of digital understanding of safety rules such as live equipment, safe distances, and no-fly zones, which poses potential safety hazards.

[0006] Unstable data acquisition quality: Factors such as flight altitude, angle, and lighting significantly affect data quality, and there is a lack of closed-loop optimization based on quality.

[0007] In the prior art, patent application CN116594426A proposes a method and system for planning the route of a substation drone inspection, but this patent cannot distinguish between equipment types and safety rules, and the planned route is still "blind". Summary of the Invention

[0008] The purpose of this invention is to provide an automatic flight path planning method and system for UAVs in the complex space of substations, which solves the problem that existing technologies cannot distinguish between equipment types and safety rules, and the planned flight paths are still "blind".

[0009] This invention is achieved through the following technical solution:

[0010] In a first aspect, embodiments of the present invention provide a method for unmanned aerial vehicle (UAV) inspection of complex spaces in substations, including:

[0011] The system parses at least one inspection task instruction input by the user to obtain the task type and corresponding data acquisition requirements for each inspection task instruction. The data acquisition requirements include the target device, acquisition angle, resolution, and safety constraints.

[0012] Based on the preset 3D semantic map of the substation and the data acquisition requirements of each inspection task instruction, a path planning algorithm is used to automatically generate at least one flight path for a UAV. The flight path includes waypoint sequence, flight altitude, shooting angle and gimbal action instructions. The 3D semantic map includes no-fly zone information in the substation, multiple devices, the location of each device, device type, electrical attributes and safety rule labels.

[0013] The corresponding drone is controlled to inspect the target equipment in the substation according to the flight path of each drone.

[0014] Preferably, the step of parsing at least one inspection task instruction input by the user to obtain the task type and corresponding data collection requirements for each inspection task instruction includes:

[0015] The system parses at least one inspection task instruction input by the user to obtain the task type corresponding to each inspection task instruction. The task type includes at least one of the following: local equipment inspection, global three-dimensional reconstruction, infrared temperature measurement, and insulator detection.

[0016] Based on the correspondence between each task type and the preset task type and data collection requirements, the data collection requirements corresponding to each task type are obtained.

[0017] Preferably, the three-dimensional semantic map is obtained in the following way:

[0018] Acquire 3D point cloud data and multi-view image data of the substation;

[0019] The three-dimensional point cloud data is semantically segmented to obtain segmentation results, which include equipment and equipment types in the substation, no-fly zones, and safety passages.

[0020] The segmentation results are fused with the multi-view image data to generate a three-dimensional semantic map with texture and semantic information.

[0021] Preferably, the step of automatically generating at least one UAV flight path using a path planning algorithm based on a preset three-dimensional semantic map of the substation and the data acquisition requirements of each inspection task instruction includes:

[0022] Based on the spatial structure and safety rule labels of the three-dimensional semantic map, as well as the target device, acquisition angle and resolution in the data acquisition requirements, the navigation target and task target are extracted and quantified to obtain multiple optimization targets. The multiple optimization targets include at least one of the following: total path length, overall safety cost, task completion degree and expected energy consumption.

[0023] Based on the task priority of the current task, assign weights to each optimization objective and construct a weighted multi-objective cost function;

[0024] The Pareto optimal solution is obtained by solving the weighted multi-objective cost function.

[0025] Based on the task priority, a path that satisfies the current task intent is selected from the Pareto optimal solution as the flight path of the UAV.

[0026] Preferably, solving the weighted multi-objective cost function to obtain the Pareto optimal solution includes:

[0027] Based on the weighted multi-objective cost function, path sampling and evaluation are performed to obtain a set of candidate paths;

[0028] Based on the dominance relationships between the candidate paths, candidate paths are filtered to obtain a non-dominated solution set;

[0029] Based on congestion or clustering algorithms, candidate paths in the non-dominated solution set are filtered to obtain a set of uniformly distributed Pareto optimal solutions.

[0030] Preferably, solving the weighted multi-objective cost function to obtain the Pareto optimal solution includes:

[0031] Based on the weighted multi-objective cost function, a multi-objective reward function is designed to obtain the fused instantaneous reward signal;

[0032] Based on the instant reward signal, the agent's strategy is trained offline or inferred online to obtain a set of flight strategies;

[0033] Based on the set of flight strategies, parallel trajectory deduction is performed to obtain a set of candidate flight trajectories;

[0034] Based on the Pareto dominance relationship, the candidate flight trajectories are screened to obtain the Pareto optimal solution.

[0035] Preferably, the step of controlling the corresponding UAV to inspect the target equipment in the substation according to the flight path of each UAV includes:

[0036] Based on the flight path of each UAV and the three-dimensional semantic map, a multi-UAV task allocation and conflict detection are performed using a co-evolutionary algorithm to obtain a conflict-free flight path for each UAV.

[0037] Display the collision-free flight path of each drone in a visual interface;

[0038] In response to the user's confirmation of the conflict-free flight path for each drone, the corresponding drone is controlled to inspect the target equipment in the substation according to the conflict-free flight path of each drone.

[0039] Preferably, the method further includes:

[0040] For each drone, when the drone is in flight, the onboard sensors of the drone are used to perceive changes in the environment in real time to determine whether the drone is in a safe state.

[0041] If the drone is not in a safe state, the flight path will be adjusted in real time until the drone is in a safe state.

[0042] Preferably, the method further includes:

[0043] The three-dimensional semantic map is converted into a spatiotemporal semantic graph, in which nodes represent devices, node attributes are the latest state of the devices, and edges represent the spatial and functional relationships between devices.

[0044] The spatiotemporal semantic graph is updated based on the latest equipment status of the target equipment in each inspection.

[0045] Based on the updated spatiotemporal semantic graph, predict the predicted state of other devices associated with the target device;

[0046] If the predicted status of other devices is inconsistent with the actual inspection status of other devices, an early warning will be issued.

[0047] Secondly, embodiments of the present invention provide an unmanned aerial vehicle (UAV) inspection system for complex spaces in substations, the system comprising:

[0048] The parsing module is used to parse at least one inspection task instruction input by the user to obtain the task type and corresponding data acquisition requirements for each inspection task instruction. The data acquisition requirements include the target device, acquisition angle, resolution and safety constraints.

[0049] The path planning module is used to automatically generate at least one flight path for a UAV based on a preset 3D semantic map of the substation and the data acquisition requirements of each inspection task instruction. The flight path includes waypoint sequence, flight altitude, shooting angle and gimbal action instructions. The 3D semantic map includes no-fly zone information in the substation, multiple devices, the location of each device, device type, electrical attributes and safety rule labels.

[0050] The inspection module is used to control the corresponding drone to inspect the target equipment in the substation according to the flight path of each drone.

[0051] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0052] 1. Standardized and precise input of inspection tasks has been achieved.

[0053] By automatically parsing abstract inspection instructions into quantifiable parameters that include target equipment, acquisition angle, resolution, and safety constraints, the subjectivity and uncertainty inherent in traditional manual instruction interpretation are eliminated. This process transforms vague operational requirements into precise data acquisition specifications that can be directly invoked by the algorithm, ensuring the clarity of inspection targets and the consistency of operational standards from the outset.

[0054] 2. An optimized flight path was generated, deeply integrating environmental perception with mission requirements.

[0055] Based on a 3D semantic map that integrates semantic information from various devices and specific data acquisition requirements, the system generates flight paths that not only meet the geometrical collision-free conditions but also achieve intelligent decision-making at the semantic level. This enables the drone to differentiate between different devices and automatically match the optimal shooting parameters, thereby significantly improving the targeting and completeness of data acquisition while ensuring absolute safety.

[0056] 3. An automated workflow that seamlessly integrates planning and execution has been established.

[0057] By directly issuing optimized flight path commands to the UAV flight control system, a precise conversion from digital planning to physical execution is achieved. This closed-loop control mechanism eliminates the intermediate link of manual operation in traditional methods, reducing safety risks caused by operational errors and ensuring the repeatability of the inspection process and the consistency of data quality through programmed execution, laying the foundation for large-scale and standardized inspection operations. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0059] Figure 1 A flowchart illustrating the UAV inspection method for complex spaces in substations provided by this invention.

[0060] Figure 2 This is a schematic diagram of the structure of the UAV inspection system for complex spaces in substations provided by the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0062] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0063] It should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and regulations of the locality and with authorization from the owner of the relevant device.

[0064] Example 1

[0065] Please see Figure 1 This invention provides a method for unmanned aerial vehicle (UAV) inspection of complex spaces in substations, comprising:

[0066] S1. Parse at least one inspection task instruction input by the user to obtain the task type and corresponding data acquisition requirements for each inspection task instruction. The data acquisition requirements include target device, acquisition angle, resolution and safety constraints.

[0067] This step aims to transform high-level user task instructions into precise, executable operational parameters for the UAV. Its motivation lies in addressing the inefficiencies, inconsistent standards, and error-prone nature of existing technologies that rely on pilot experience to manually set data acquisition parameters. Specifically, the system receives abstract task instructions from the user, such as insulator crack detection or circuit breaker infrared temperature measurement, and parses them using an embedded task knowledge base. This knowledge base pre-defines the mapping relationships between different task types and specific data acquisition requirements; taking insulator detection as an example, the parsing process automatically outputs that the target device is a specific insulator string, the acquisition angle must cover all surface normal directions, the resolution must be better than 2 mm per pixel, and automatically adds a constraint of maintaining a safe distance of 0.7 meters from the energized equipment. Through this automated parsing, this step transforms vague task intentions into a set of quantifiable data acquisition specifications that can be directly processed by subsequent algorithms, achieving standardization of task input, laying a solid foundation for fully automated flight path planning, and effectively avoiding substandard acquisition quality or safety risks caused by human misunderstanding or oversight.

[0068] S2. Based on the preset three-dimensional semantic map of the substation and the data acquisition requirements of each inspection task instruction, a path planning algorithm is used to automatically generate at least one flight path for a UAV. The flight path includes waypoint sequence, flight altitude, shooting angle and gimbal action instructions. The three-dimensional semantic map includes no-fly zone information in the substation, multiple devices, the location of each device, device type, electrical attributes and safety rule labels.

[0069] The core motivation behind this invention lies in automating the conversion from known environmental semantics and explicit data acquisition requirements to safe, efficient, and executable flight routes. This overcomes the shortcomings of fixed routes, such as poor flexibility, low efficiency of manual planning, and difficulty in accommodating complex safety rules. In implementation, this process relies on two main inputs provided by the preceding steps: a 3D semantic map rich in device semantic information and quantified data acquisition requirements. The system first constructs a dynamic safety cost map based on no-fly zones, device electrical attributes, and safety rule labels in the 3D semantic map. For example, it generates a cost field around a charged conductor where the cost decreases with distance. Subsequently, a path planning algorithm, such as a fast-exploration random tree algorithm or its variants, performs a spatial search under the constraints of this cost map. Its optimization objective simultaneously considers path length, overall safety cost, and task completion. The algorithm weights these objectives according to task priority, ultimately generating one or more collision-free waypoint sequences. Simultaneously, the system automatically calculates the flight altitude, gimbal pitch angle, and shooting commands corresponding to each waypoint based on the acquisition angle and resolution specified in the data acquisition requirements. The technical advantage of this step is that it directly encodes abstract safety procedures and mission requirements into specific flight trajectories, achieving intelligent and adaptive route planning and ensuring the inherent unity of flight safety and data acquisition quality.

[0070] S3. Control the corresponding drone to inspect the target equipment in the substation according to the flight path of each drone.

[0071] The motivation behind this invention is to eliminate manual operation, achieving full automation and unmanned operation of the inspection process, thereby improving work efficiency and completely eliminating operational risks that may arise from human-machine interaction. The process is as follows: The flight path file, generated in step S2 and containing waypoint sequences and action commands, is transmitted via wireless communication to the designated UAV's flight control system. The UAV's flight control system then takes over flight control, driving the UAV to fly autonomously along the preset path, strictly following the shooting angles and gimbal action commands embedded in the path, automatically triggering the camera to collect data upon reaching the target waypoint. During this process, the UAV can achieve high-precision positioning and attitude stabilization using onboard sensors, ensuring the geometric quality of the collected data. The technical advantage of this step is that it achieves seamless integration of planning and execution, enabling the entire substation inspection process to be completed automatically without on-site pilot intervention. This not only significantly saves manpower but also ensures consistency and repeatability of flight operations and data acquisition processes through programmed execution.

[0072] In some implementations, S1 involves parsing at least one inspection task instruction input by the user to obtain the task type and corresponding data collection requirements for each inspection task instruction, including:

[0073] The system parses at least one inspection task instruction input by the user to obtain the task type corresponding to each inspection task instruction. The task type includes at least one of the following: local equipment inspection, global three-dimensional reconstruction, infrared temperature measurement, and insulator detection.

[0074] Based on the correspondence between each task type and the preset task type and data collection requirements, the data collection requirements corresponding to each task type are obtained.

[0075] This embodiment configures the system with an embedded task knowledge base, which is essentially a predefined mapping table that associates task types (such as partial equipment inspection, global 3D reconstruction, infrared thermography, insulator detection, etc.) with their respective necessary and optimized data acquisition requirements. When a user inputs a command such as "insulator detection," the system first identifies the task type as "insulator detection" through pattern matching or keyword extraction. Then, it queries the knowledge base to automatically retrieve and output the complete set of data acquisition requirements bound to this type. For example, for the insulator detection task, the knowledge base specifies that the target device is an insulator string, the acquisition angle must be circumferential to cover all its surfaces, the resolution must reach the accuracy of identifying millimeter-level cracks, and automatically applies a constraint of maintaining a specific safe distance from the energized equipment.

[0076] In some implementations, the three-dimensional semantic map is obtained in the following way:

[0077] Acquire 3D point cloud data and multi-view image data of the substation;

[0078] The three-dimensional point cloud data is semantically segmented to obtain segmentation results, which include equipment and equipment types in the substation, no-fly zones, and safety passages.

[0079] The segmentation results are fused with the multi-view image data to generate a three-dimensional semantic map with texture and semantic information.

[0080] This embodiment aims to create a digital environment model for drone inspection that can be deeply understood and utilized by machines. Its motivation lies in addressing the blindness caused by existing technologies where drones rely solely on purely geometric maps lacking semantic information for planning. For example, they cannot distinguish between live equipment and grounding structures, thus hindering rule-based intelligent obstacle avoidance and mission-oriented flight. The specific implementation process is as follows: First, using sensors such as LiDAR and oblique photography cameras mounted on the drone, 3D point cloud data and multi-view image data of the target substation are simultaneously acquired. The point cloud data provides accurate spatial geometry, while the image data provides rich surface texture and color information. Subsequently, a pre-trained deep learning semantic segmentation model, such as PointNet++ or RandLA-Net, is used to process the acquired 3D point cloud data. This model, trained on a large number of power equipment samples, can identify the category of each point in the point cloud, thus obtaining the segmentation result. This result not only identifies equipment such as circuit breakers, disconnect switches, and insulators and their specific equipment types, but also distinguishes no-fly zones (such as the airspace above safety passages) and critical areas such as safety passages. Finally, through 3D reconstruction and texture mapping technology, the segmentation results containing device type and region labels are registered and fused with multi-view image data. The image data gives the model realistic surface texture, while the segmentation results attach its identity, function and rule attributes to each 3D entity, ultimately generating a 3D semantic map that integrates visual appearance and machine-readable semantics.

[0081] In some implementations, S2, based on a preset three-dimensional semantic map of the substation and the data acquisition requirements of each inspection task instruction, automatically generates at least one flight path for a UAV using a path planning algorithm, including:

[0082] Based on the spatial structure and safety rule labels of the three-dimensional semantic map, as well as the target device, acquisition angle and resolution in the data acquisition requirements, the navigation target and task target are extracted and quantified to obtain multiple optimization targets. The multiple optimization targets include at least one of the following: total path length, overall safety cost, task completion degree and expected energy consumption.

[0083] Based on the task priority of the current task, assign weights to each optimization objective and construct a weighted multi-objective cost function;

[0084] The Pareto optimal solution is obtained by solving the weighted multi-objective cost function.

[0085] Based on the task priority, a path that satisfies the current task intent is selected from the Pareto optimal solution as the flight path of the UAV.

[0086] This embodiment transforms the environmental semantic information and specific task requirements output from the aforementioned steps into a flight path that achieves an optimal balance among multiple conflicting performance indicators. Its motivation lies in addressing the difficulty of single-objective planning (such as the shortest path) adapting to the coexistence of multiple constraints such as safety, quality, and efficiency in substation inspections, thereby achieving intelligent and adaptive optimization of flight paths in complex decision-making spaces. The specific implementation process is as follows: The system first extracts and quantifies navigation and task objectives based on the spatial structure and safety rule labels provided by the 3D semantic map, combined with the target equipment, acquisition angle, and resolution explicitly specified in the data acquisition requirements. For example, the total path length optimization objective can be derived from the spatial structure; the overall safety cost (such as the integral of the minimum distance between each point on the flight path and the energized equipment) can be derived from the safety rule labels; and the task completion rate (such as the effective coverage ratio of the target equipment surface) can be derived from the target equipment and acquisition requirements. Finally, the expected energy consumption is estimated using the UAV dynamics model. Subsequently, the system assigns corresponding weights to the multiple optimization objectives based on the task priority of the current task. For example, for an urgent defect review task, higher weights are given to task completion and total path length to pursue speed and accuracy, while for routine detailed inspections, higher weights are given to overall safety costs to prioritize safety. This constructs a weighted multi-objective cost function. Next, a multi-objective optimization algorithm, such as a multi-objective fast exploration random tree algorithm or a genetic algorithm based on non-dominated sorting, is used to solve this weighted multi-objective cost function. Due to the trade-offs between the objectives, the algorithm outputs a Pareto optimal solution set consisting of multiple candidate paths. Each path in this set represents an optimal choice with different objective emphases. Finally, the system again selects, based on task priority, the path that best matches the core intent of the current task from this Pareto optimal solution set. For example, it selects the shortest path while ensuring safety and completion, as the final determined UAV flight path.

[0087] In some implementations, solving the weighted multi-objective cost function to obtain the Pareto optimal solution includes:

[0088] Based on the weighted multi-objective cost function, path sampling and evaluation are performed to obtain a set of candidate paths;

[0089] Based on the dominance relationships between the candidate paths, candidate paths are filtered to obtain a non-dominated solution set;

[0090] Based on congestion or clustering algorithms, candidate paths in the non-dominated solution set are filtered to obtain a set of uniformly distributed Pareto optimal solutions.

[0091] This embodiment systematically filters a set of candidate solutions from a large number of possible flight paths, selecting those that perform well across multiple optimization objectives and are mutually non-dominant. The motivation behind this invention is to address the problem in multi-objective optimization where a single solution cannot fully reflect the trade-offs between all objectives, thus providing a rich and high-quality selection space for subsequent contextualized decision-making. The specific implementation process is as follows: First, the path planning algorithm performs large-scale path sampling in the state space defined by the 3D semantic map based on the weighted multi-objective cost function constructed in the preceding steps. For each candidate path generated, its specific values ​​for multiple preset objectives, such as total path length and overall safety cost, are immediately calculated according to the cost function, resulting in a set of candidate paths with varying performance advantages and disadvantages. Subsequently, the system filters these candidate paths based on Pareto dominance. Specifically, if path A performs no worse than path B in all optimization objectives and is strictly superior to path B in at least one objective, then path A is said to dominate path B. By comparing all paths pairwise, all dominated paths are filtered out, ultimately resulting in a non-dominant solution set. No path in this set can achieve further improvement in other objectives without sacrificing the performance of at least one objective. Finally, to avoid the obtained solution set being too concentrated in the objective function space, leading to a lack of diversity in choices, the system further manages the distribution of candidate paths in the non-dominated solution set based on either congestion calculation or clustering algorithms. Congestion calculation filters solutions by measuring the density of each solution among its neighbors, retaining solutions located in sparse regions. Clustering algorithms directly group the solution set according to its position in the objective space and select representatives from each group. Through one of these two methods, a set of Pareto optimal solutions evenly distributed across multiple optimization objectives is ultimately obtained. The technical advantage of this step is that it ensures that the final decision-making process provides a diverse set of high-quality route options covering different performance focuses, rather than a single compromise solution. This allows the system to flexibly select the most suitable flight strategy based on real-time changing task requirements, enhancing the robustness and adaptability of the entire route planning system.

[0092] In some implementations, solving the weighted multi-objective cost function to obtain the Pareto optimal solution includes:

[0093] Based on the weighted multi-objective cost function, a multi-objective reward function is designed to obtain the fused instantaneous reward signal;

[0094] Based on the instant reward signal, the agent's strategy is trained offline or inferred online to obtain a set of flight strategies;

[0095] Based on the set of flight strategies, parallel trajectory deduction is performed to obtain a set of candidate flight trajectories;

[0096] Based on the Pareto dominance relationship, the candidate flight trajectories are screened to obtain the Pareto optimal solution.

[0097] This embodiment utilizes a reinforcement learning framework to solve the multi-objective optimization problem of UAV inspection paths. The specific implementation process is as follows: First, the system designs a multi-objective reward function based on the aforementioned weighted multi-objective cost function. Optimization objectives such as path length and safety cost are fused into a scalar instantaneous reward signal through linear weighting or other reward shaping techniques. This signal is used to evaluate the merits of each decision action of the agent during training. Subsequently, in a simulation environment, one or more agents perform deep reinforcement learning training based on the environmental state and the aforementioned instantaneous reward signal, using policy gradient or value iteration algorithms. During this process, by adjusting the network initialization parameters or introducing diversity promotion mechanisms, a set of flight strategies with different focuses can be trained; for example, some strategies are more inclined towards flight efficiency, while others focus more on safety. Next, in the planning phase, the system loads this set of trained flight strategies in parallel, using the current 3D semantic map of the substation and the task objective as input, allowing each strategy to independently perform trajectory extrapolation, thereby quickly generating a set of candidate flight trajectories with differences in behavioral characteristics. Finally, the system filters this set of candidate flight trajectories based on Pareto dominance. Specifically, it calculates the true values ​​of each trajectory across the original multiple optimization objectives and removes all individuals dominated by other trajectories. The resulting Pareto optimal solution set contains flight trajectories that perform well across multiple optimization objectives and cannot be improved simultaneously. The technical advantage of this step lies in transforming the complex multi-objective optimization problem into a learning problem for the agent. By leveraging the powerful search capabilities of reinforcement learning in complex spaces, it can efficiently generate a series of high-quality, diverse flight path solutions, providing ample choice for subsequent situational adaptive decision-making and enhancing the system's flexibility and reliability in handling variable inspection tasks.

[0098] In some implementations, S3 involves controlling the corresponding drone to inspect the target equipment in the substation according to the flight path of each drone, including:

[0099] Based on the flight path of each UAV and the three-dimensional semantic map, a multi-UAV task allocation and conflict detection are performed using a co-evolutionary algorithm to obtain a conflict-free flight path for each UAV.

[0100] Display the collision-free flight path of each drone in a visual interface;

[0101] In response to the user's confirmation of the conflict-free flight path for each drone, the corresponding drone is controlled to inspect the target equipment in the substation according to the conflict-free flight path of each drone.

[0102] This embodiment addresses the issues of task allocation, flight path conflict resolution, and manual supervision in multi-UAV collaborative inspection scenarios. Its motivation lies in overcoming the potential for task overlap, flight path intersections, or resource competition in multi-UAV systems due to a lack of centralized coordination. Simultaneously, it introduces a manual confirmation step to ensure the reliability of system decisions and the smoothness of human-machine collaboration. The specific implementation process is as follows: First, based on the initially generated flight paths for each UAV and a shared 3D semantic map, the system employs a co-evolutionary algorithm for multi-UAV task allocation and conflict detection. In this process, each individual in a population represents a complete task allocation and flight path combination scheme. The algorithm continuously optimizes the fitness function by simulating the population's evolutionary process. This function considers task completion efficiency, the total length of each UAV's flight path, and key spatiotemporal conflict indicators (such as the minimum safe distance between UAVs). Through crossover, mutation, and selection operations within the population, an optimized task allocation scheme is finally converged, and a refined flight path without spatial or temporal conflicts is generated for each UAV accordingly. Subsequently, the system overlays these conflict-free flight paths onto a 3D semantic map for visualization and rendering, displaying them on the human-machine interface. This clearly presents the planned flight paths, task areas, and expected timings of the multiple drones to the operator. Finally, in response to the user's confirmation of the entire flight path plan, which serves as the final execution command trigger, the system controls the corresponding drone swarm to take off as planned according to the confirmed conflict-free flight paths, conducting collaborative inspections of the target equipment in the substation. The technical advantage of this step lies in ensuring the rationality and safety of task allocation and flight path planning for the multi-drone system through automated algorithms. Simultaneously, the visualization and manual confirmation mechanisms empower the operator with ultimate supervision and control, achieving an effective combination of automated planning and human experience judgment. This improves the efficiency and scale of inspection operations while ensuring the reliability of the entire system and the user-friendliness of the human-machine interface.

[0103] In some embodiments, the method further includes:

[0104] For each drone, when the drone is in flight, the onboard sensors of the drone are used to perceive changes in the environment in real time to determine whether the drone is in a safe state.

[0105] If the drone is not in a safe state, the flight path will be adjusted in real time until the drone is in a safe state.

[0106] This embodiment addresses safety issues that may arise when drones execute predetermined routes in complex substation environments due to sudden or unmodeled environmental factors (such as temporary obstacles or drift caused by sudden strong winds). Its motivation lies in overcoming the shortcomings of static route planning in responding to real-time environmental changes, ensuring continuous and safe operation of inspection work in dynamic and uncertain environments. The specific implementation process is as follows: For each drone in flight, the system continuously senses real-time changes in its surrounding environment at a high frequency using an onboard sensor suite integrated into the drone platform, including but not limited to visual sensors, lidar, and an inertial measurement unit. The acquired data includes changes in the relative positions of the drone to static obstacles in a preset 3D semantic map, and the presence of dynamic obstacles not marked in the model. Based on this real-time sensing data, the system evaluates the situation using a pre-set safety status judgment logic. The core of this logic is to verify whether the drone's current and predicted short-term states meet multiple safety constraints, such as whether the actual distance to any obstacle (whether known or unknown) is greater than the required safety margin, whether the drone's attitude is stable and within a controllable range, and whether its flight trajectory will momentarily enter a no-fly zone. If the assessment indicates that the drone is not in a safe state, for example, if it is detected that the distance between it and an unknown temporary obstacle is rapidly approaching the safety threshold, the system's dynamic obstacle avoidance module is activated. Based on real-time perception information, this module uses reactive obstacle avoidance algorithms, such as the artificial potential field method or the dynamic window method, to quickly calculate a local alternative path that can immediately avoid the current risk within a local space, and generates corresponding flight control commands to adjust the drone's original flight path in real time. This adjustment process continues until the onboard sensors confirm that the drone has escaped the risk and returned to a safe state. The technical effect of this step is that it adds a real-time safety monitoring and protection layer to the fully autonomous flight of the drone, enabling the system to not only rely on prior environmental models for planning but also to respond promptly and safely to unexpected situations that occur during actual flight, thereby significantly improving the robustness and safety of drone inspection operations in complex environments.

[0107] In some embodiments, the method further includes:

[0108] The three-dimensional semantic map is converted into a spatiotemporal semantic graph, in which nodes represent devices, node attributes are the latest state of the devices, and edges represent the spatial and functional relationships between devices.

[0109] The spatiotemporal semantic graph is updated based on the latest equipment status of the target equipment in each inspection.

[0110] Based on the updated spatiotemporal semantic graph, predict the predicted state of other devices associated with the target device;

[0111] If the predicted status of other devices is inconsistent with the actual inspection status of other devices, an early warning will be issued.

[0112] This embodiment aims to elevate a static environmental model into a dynamic knowledge system capable of reflecting the temporal changes in equipment status and performing relational reasoning. Its motivation lies in overcoming the limitations of existing inspection methods, which can only detect apparent defects and cannot predict potential risks based on inter-equipment relationships. This allows for a shift from passive detection to proactive early warning in operation and maintenance. The specific implementation process is as follows: The system first converts the aforementioned constructed 3D semantic map into a graph-structured data model, namely a temporal semantic graph. In this graph, nodes represent various devices within the substation (such as circuit breakers, disconnectors, current transformers, etc.). Node attributes not only include static parameters of the equipment (such as equipment type and nameplate information), but more importantly, record its latest equipment status (such as temperature values ​​obtained from the last inspection, visible light image analysis results, insulator self-explosion index, etc.). Edges represent spatial relationships (such as physical connections and adjacency relationships) and functional relationships (such as electrical connections and control logic associations) between devices. Subsequently, the system updates the state attributes of corresponding nodes in the temporal semantic graph based on the latest equipment status of the target equipment obtained from each inspection task. This allows the graph to dynamically evolve over time, forming a continuous historical record of equipment status. Based on this, the system utilizes graph inference algorithms such as graph neural networks to predict the associated equipment status based on the connections between nodes in the updated temporal semantic graph and the historical state sequence. For example, when the graph shows a circuit breaker node in an open state, it can predict that the current transformer node on that line should be in a no-current state based on its electrical connection with the downstream line. Finally, the system compares the predicted state with the actual inspection state obtained by the equipment during this inspection. If an inconsistency is found (e.g., predicted no current but actual current detected), a logical anomaly is determined, and an early warning signal is generated and issued. The technical effect of this step is that by introducing a temporal dimension and relational reasoning capabilities, the inspection system no longer views each device in isolation, but analyzes it within the operational context of the entire substation. This enables the discovery of hidden faults and logical contradictions that are difficult to identify through conventional testing, providing in-depth decision support for predictive maintenance of power equipment and improving the foresight and intelligence level of operation and maintenance work.

[0113] In some implementations, the drone also includes a strategy model. This strategy model employs an Actor-Critic architecture-based MARL algorithm (such as MADDPG), trained extensively in a simulation environment. The strategy model can negotiate and collaborate with other agents based on its own location, battery level, and task progress, dynamically adjusting its respective task area and flight path. For example, if a drone is running low on battery, nearby drones with sufficient battery power will automatically take over its unfinished inspection area; or when a drone discovers a significant hazard requiring focused scanning, it will "call" a partner to assist in completing its original routine task.

[0114] In some implementations, after completing their mission, the drone automatically returns to the airport, where it has its battery replaced and recharged. The collected data can then be uploaded via a high-speed network, enabling 24 / 7 uninterrupted operation. A lightweight AI model is deployed on the drone for real-time initial defect screening (e.g., "insulator self-explosion"). Suspicious images are then transmitted to the cloud for detailed analysis. The cloud-based analysis model iterates periodically and updates are distributed to the drone via incremental learning. Hardware such as DJI drones is integrated, and lightweight model deployment is achieved using TensorFlow Lite or PyTorch Mobile.

[0115] For example, the implementation process of this method is illustrated by taking the fine inspection of insulator strings in a 500kV substation as an example.

[0116] S101: Constructing a 3D semantic map library

[0117] Using a DJI Matrice 350 RTK drone, equipped with a Zenmuse P1 aerial survey camera and a DJI L1 LiDAR, the target substation was automatically scanned to collect high-precision point cloud and multi-view high-definition images.

[0118] The collected data is uploaded to the ground processing station of this system. The processing station uses the PointNet++ algorithm to perform semantic segmentation on the point cloud, automatically identifying categories such as "insulator", "circuit breaker", "conductor (energized)", "structure (grounded)", and "safety passage", and assigning a label to each point cloud object.

[0119] By integrating labels with 3D models, a 3D semantic map is generated. In this map, each insulator string is not only a 3D model, but its attributes are also marked with "requires detailed inspection" and "safe distance: 0.7 meters"; nearby conductors are marked as "charged body", "danger", and "safe distance: 3 meters".

[0120] It should be noted that the "semantic environment of a substation" refers to a three-dimensional environment model that includes not only geometric shape and spatial location, but more importantly, semantic information such as "what it is", "what its function is", and "what rules it follows".

[0121] A 3D model of a substation with semantic context is not just a bunch of cold points and grids, but rather each piece of equipment has its own identification card and instruction manual:

[0122]

[0123] With semantic context, the "automatic route planning" of drones has changed from "blind men feeling an elephant" to "experienced drivers leading the way".

[0124] Intelligent obstacle avoidance: Instead of avoiding everything, it selectively avoids obstacles. It knows to stay away from live equipment (3 meters away), but can get close to insulators (within 1 meter) for detailed shooting.

[0125] Mission-oriented flight path planning: For infrared thermography missions, the aircraft will automatically prioritize flying upwind of heat-generating equipment such as transformers and circuit breakers for imaging. For insulator crack detection, the aircraft will automatically generate a circling flight path close to the insulator string to ensure that all angles are captured.

[0126] Safety and compliance: The system kernel integrates safety procedures, automatically avoids no-fly zones (such as airspace above safety corridors) during route planning, and ensures that the distance between any waypoint and powered equipment is greater than the threshold set by the safety procedures.

[0127] Efficient data acquisition: Knowing where the "key points" are, avoiding a large amount of ineffective shooting of empty ground and irrelevant equipment, and only collecting mission-related and valuable data.

[0128] Collaborative operation: Semantic maps can serve as a "public knowledge base" shared by multiple drones, facilitating the collaborative inspection of a large substation by multiple drones, each responsible for different semantic areas (e.g., drone A inspects the transformer area, drone B inspects the circuit breaker area).

[0129] S102: Analyzing Inspection Tasks

[0130] The operator selects the "Automatic Insulator Crack Detection" task package in the system's task management interface and selects the specific insulator string area that needs to be inspected.

[0131] The system analyzed the task and determined that its core requirement was to acquire multi-angle, high-resolution (better than 2mm / pixel) and unobstructed images of the target insulator string.

[0132] S103: Automatic route generation

[0133] The route planning module receives tasks and semantic maps.

[0134] The module first automatically avoids nearby charged conductors (maintaining a distance of more than 3 meters) and safe passages (no-fly zones) based on semantic information.

[0135] Subsequently, for the specific device type of "insulator string", the preset orbital scan route template is invoked.

[0136] Based on the specific size and location of the insulator, the algorithm automatically calculates the optimal orbital radius (0.8 meters, slightly larger than the safety distance), flight speed (0.5 m / s to ensure image clarity), waypoint interval, and gimbal angle, generating a complete and safe-to-execute refined inspection route.

[0137] S104: Dynamic Execution and Obstacle Avoidance

[0138] The flight path is sent to the drone via the communication module. The drone takes off autonomously and executes the flight path.

[0139] During the operation, a sudden gust of strong wind caused the drone to drift slightly. The onboard visual sensor detected that the distance to nearby structures was less than a preset threshold.

[0140] The dynamic obstacle avoidance module immediately intervenes, calculating a local detour path within milliseconds, instructing the drone to fine-tune its attitude, bypass potential risk points, and then rejoin the original route.

[0141] S105: Quality Assessment and Closed-Loop Optimization

[0142] After completing its mission, the drone automatically transmits the 200 high-definition images it captured back to the system.

[0143] The quality assessment module automatically performs image sharpness (by calculating image gradient) and coverage (by calculating 3D reconstruction rate using SFM).

[0144] Analysis revealed that the image of the lower surface of one insulator was blurred and incomplete due to obstruction by a fallen leaf. The system automatically marked this area as "requiring reshooting" and generated a simplified reshoot flight path specifically for this missing area.

[0145] After the operator confirmed, the drone took off again and completed the reshoot in just one minute, ensuring the integrity of the data.

[0146] Example 2

[0147] Please see Figure 2 This invention provides an unmanned aerial vehicle (UAV) inspection system for complex spaces in substations, the system comprising:

[0148] The parsing module 201 is used to parse at least one inspection task instruction input by the user to obtain the task type corresponding to each inspection task instruction and its corresponding data acquisition requirements. The data acquisition requirements include the target device, acquisition angle, resolution and safety constraints.

[0149] The path planning module 202 is used to automatically generate at least one flight path of a UAV based on the preset three-dimensional semantic map of the substation and the data acquisition requirements of each inspection task instruction. The flight path includes waypoint sequence, flight altitude, shooting angle and gimbal action instructions. The three-dimensional semantic map includes no-fly zone information in the substation, multiple devices, the location of each device, device type, electrical attributes and safety rule labels.

[0150] The inspection module 203 is used to control the corresponding UAV to inspect the target equipment in the substation according to the flight path of each UAV.

[0151] The system is deployed on edge computing devices or cloud platforms and supports real-time communication and command issuance with the UAV flight control system.

[0152] It should be noted that each module and unit in the UAV inspection system for complex substation spaces in this embodiment corresponds one-to-one with each step in the UAV inspection method for complex substation spaces in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned UAV inspection method for complex substation spaces, and will not be repeated here.

[0153] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) inspection of complex spaces in substations, characterized in that, include: The system parses at least one inspection task instruction input by the user to obtain the task type and corresponding data acquisition requirements for each inspection task instruction. The data acquisition requirements include the target device, acquisition angle, resolution, and safety constraints. Based on the preset 3D semantic map of the substation and the data acquisition requirements of each inspection task instruction, a path planning algorithm is used to automatically generate at least one flight path for a UAV. The flight path includes waypoint sequence, flight altitude, shooting angle and gimbal action instructions. The 3D semantic map includes no-fly zone information in the substation, multiple devices, the location of each device, device type, electrical attributes and safety rule labels. The corresponding drone is controlled to inspect the target equipment in the substation according to the flight path of each drone.

2. The method according to claim 1, characterized in that, The process of parsing at least one inspection task instruction input by the user to obtain the task type and corresponding data collection requirements for each inspection task instruction includes: The system parses at least one inspection task instruction input by the user to obtain the task type corresponding to each inspection task instruction. The task type includes at least one of the following: local equipment inspection, global three-dimensional reconstruction, infrared temperature measurement, and insulator detection. Based on the correspondence between each task type and the preset task type and data collection requirements, the data collection requirements corresponding to each task type are obtained.

3. The method according to claim 1, characterized in that, The three-dimensional semantic map was obtained in the following way: Acquire 3D point cloud data and multi-view image data of the substation; The three-dimensional point cloud data is semantically segmented to obtain segmentation results, which include equipment and equipment types in the substation, no-fly zones, and safety passages. The segmentation results are fused with the multi-view image data to generate a three-dimensional semantic map with texture and semantic information.

4. The method according to claim 1, characterized in that, The method involves automatically generating at least one UAV flight path using a path planning algorithm based on a preset 3D semantic map of the substation and the data acquisition requirements of each inspection task instruction, including: Based on the spatial structure and safety rule labels of the three-dimensional semantic map, as well as the target device, acquisition angle and resolution in the data acquisition requirements, the navigation target and task target are extracted and quantified to obtain multiple optimization targets. The multiple optimization targets include at least one of the following: total path length, overall safety cost, task completion degree and expected energy consumption. Based on the task priority of the current task, assign weights to each optimization objective and construct a weighted multi-objective cost function; The Pareto optimal solution is obtained by solving the weighted multi-objective cost function. Based on the task priority, a path that satisfies the current task intent is selected from the Pareto optimal solution as the flight path of the UAV.

5. The method according to claim 4, characterized in that, Solving the weighted multi-objective cost function to obtain the Pareto optimal solution includes: Based on the weighted multi-objective cost function, path sampling and evaluation are performed to obtain a set of candidate paths; Based on the dominance relationships between the candidate paths, candidate paths are filtered to obtain a non-dominated solution set; Based on congestion or clustering algorithms, candidate paths in the non-dominated solution set are filtered to obtain a set of uniformly distributed Pareto optimal solutions.

6. The method according to claim 4, characterized in that, Solving the weighted multi-objective cost function to obtain the Pareto optimal solution includes: Based on the weighted multi-objective cost function, a multi-objective reward function is designed to obtain the fused instantaneous reward signal; Based on the instant reward signal, the agent's strategy is trained offline or inferred online to obtain a set of flight strategies; Based on the set of flight strategies, parallel trajectory deduction is performed to obtain a set of candidate flight trajectories; Based on the Pareto dominance relationship, the candidate flight trajectories are screened to obtain the Pareto optimal solution.

7. The method according to any one of claims 1-6, characterized in that, The step of controlling the corresponding drone to inspect the target equipment in the substation according to the flight path of each drone includes: Based on the flight path of each UAV and the three-dimensional semantic map, a multi-UAV task allocation and conflict detection are performed using a co-evolutionary algorithm to obtain a conflict-free flight path for each UAV. Display the collision-free flight path of each drone in a visual interface; In response to the user's confirmation of the conflict-free flight path for each drone, the corresponding drone is controlled to inspect the target equipment in the substation according to the conflict-free flight path of each drone.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: For each drone, when the drone is in flight, the onboard sensors of the drone are used to perceive changes in the environment in real time to determine whether the drone is in a safe state. If the drone is not in a safe state, the flight path will be adjusted in real time until the drone is in a safe state.

9. The method according to any one of claims 1-6, characterized in that, The method further includes: The three-dimensional semantic map is converted into a spatiotemporal semantic graph, in which nodes represent devices, node attributes are the latest state of the devices, and edges represent the spatial and functional relationships between devices. The spatiotemporal semantic graph is updated based on the latest equipment status of the target equipment in each inspection. Based on the updated spatiotemporal semantic graph, predict the predicted state of other devices associated with the target device; If the predicted status of other devices is inconsistent with the actual inspection status of other devices, an early warning will be issued.

10. A drone inspection system for complex spaces in substations, characterized in that, The system includes: The parsing module is used to parse at least one inspection task instruction input by the user to obtain the task type and corresponding data acquisition requirements for each inspection task instruction. The data acquisition requirements include the target device, acquisition angle, resolution and safety constraints. The path planning module is used to automatically generate at least one flight path for a UAV based on a preset 3D semantic map of the substation and the data acquisition requirements of each inspection task instruction. The flight path includes waypoint sequence, flight altitude, shooting angle and gimbal action instructions. The 3D semantic map includes no-fly zone information in the substation, multiple devices, the location of each device, device type, electrical attributes and safety rule labels. The inspection module is used to control the corresponding drone to inspect the target equipment in the substation according to the flight path of each drone.

Images