Transformer substation unmanned aerial vehicle autonomous inspection method and system based on dynamic environment perception

Abstract

The invention discloses a transformer substation unmanned aerial vehicle autonomous inspection method and system based on dynamic environment awareness, and the method comprises the steps: carrying out the data collection of a plurality of sensors, carrying out the space registration of laser radar point cloud data collected by a laser radar and an infrared thermogram collected by an infrared thermal imager, based on the fusion data, the vibration sensor data and the electromagnetic sensor data under the unified coordinate system, generating a three-dimensional dynamic model of the transformer substation through a three-dimensional reconstruction algorithm; constructing a topological graph of the transformer substation based on the fusion data under the unified coordinate system and the basic topological structure of the transformer substation; marking the connection relationship of the equipment in the topological graph, and generating an unmanned aerial vehicle inspection path through path planning in combination with the priority of the equipment; dynamically updating the three-dimensional dynamic model, and optimizing the inspection path of the unmanned aerial vehicle; the unmanned aerial vehicle flies according to the optimized inspection path, and substation defect detection is conducted through the inspection model. The unmanned aerial vehicle inspection accuracy and efficiency are improved.

Description

Technical Field

[0001] This invention relates to autonomous inspection of substations by drones, specifically to an autonomous inspection method and system for substations by drones based on dynamic environmental perception. Background Technology

[0002] In the field of power equipment operation and maintenance, traditional manual inspection methods face many challenges, such as low inspection efficiency, high safety risks, strong data subjectivity, and blind spots in inspection, leading to frequent missed inspections and erroneous inspections.

[0003] While some early drone inspection solutions have improved inspection efficiency to a certain extent with the development of drone technology, they still have significant limitations, including the following technical issues:

[0004] ① Complex electromagnetic environment interference: Traditional methods are difficult to effectively deal with strong electromagnetic interference in substations, resulting in inaccurate inspection data;

[0005] ② Insufficient dynamic obstacle avoidance: There are shortcomings in dynamic obstacle recognition and avoidance, which increases flight risks;

[0006] ③ Weak ability to collaboratively analyze multi-source data: The lack of effective fusion and analysis of heterogeneous multi-source data leads to a low fault identification rate.

[0007] Chinese patent CN120428756A discloses a method and system for autonomous inspection of substations using unmanned aerial vehicles (UAVs). Through innovations such as environmental modeling, reinforcement learning path planning, BIM incremental updates, and multi-algorithm fusion optimization, it significantly improves the adaptability of the inspection system to the dynamic environment of substations. This patent achieves dynamic obstacle avoidance and path optimization by dynamically training an obstacle avoidance model through reinforcement learning to autonomously plan obstacle avoidance paths, thus reducing flight risks. However, it still has the following shortcomings:

[0008] ① Limited modeling methods: Relying solely on LiDAR to generate 3D maps, resulting in limited ability to perceive complex environments;

[0009] ② Limitations of reinforcement learning applications: Reinforcement learning models are mainly used for obstacle avoidance path planning, and their adaptability to different environments needs to be improved;

[0010] ③ Data processing workflow is missing: There is a lack of systematic processing workflow for information collected by drones, and no in-depth analysis of equipment status during the inspection process is involved. Summary of the Invention

[0011] The purpose of this invention is to provide a method and system for autonomous inspection of substations by unmanned aerial vehicles (UAVs) based on dynamic environmental perception, thereby solving the problems in the background art.

[0012] The technical solution to achieve the objective of this invention is as follows:

[0013] A method for autonomous inspection of substations using unmanned aerial vehicles (UAVs) based on dynamic environmental perception includes the following steps:

[0014] Data acquisition is performed using lidar, infrared thermal imager, vibration sensor, and electromagnetic sensor.

[0015] Spatial registration is performed between lidar point cloud data collected by lidar and infrared thermal images collected by infrared thermal imager to generate fused data in a unified coordinate system.

[0016] Based on fused data, vibration sensor data, and electromagnetic sensor data under a unified coordinate system, a three-dimensional dynamic model of the substation is generated through a three-dimensional reconstruction algorithm.

[0017] A substation topology map is constructed based on fused data and the basic topology of the substation under a unified coordinate system.

[0018] Mark the connection relationships of devices in the topology diagram, and generate drone inspection paths through path planning based on device priorities;

[0019] Dynamically update the 3D dynamic model and optimize the drone inspection path;

[0020] The drone flies along the optimized inspection path and uses the inspection model to detect defects in the substation.

[0021] Furthermore, an iterative nearest-point algorithm is used to spatially register the lidar data with the infrared thermal imager data. The algorithm iteratively searches for corresponding points in the lidar point cloud and the infrared thermal image, and determines the rotation vector and translation vector by minimizing the distance error between the corresponding points to perform coordinate alignment.

[0022] Furthermore, a three-dimensional dynamic model of the substation is generated through a three-dimensional reconstruction algorithm. Specifically, this includes: firstly, filtering and denoising the lidar point cloud data; then, using the Delaunay triangulation algorithm to generate a three-dimensional mesh model; and finally, mapping the infrared thermal image, vibration spectrum, and electromagnetic distribution information onto the three-dimensional mesh to form a three-dimensional map of the substation containing equipment temperature distribution and electromagnetic radiation status labels.

[0023] Furthermore, a graph neural network is used to construct the substation topology map. The spatially registered data and the basic topology of the substation are used as inputs. The graph neural network learns the connection relationships and feature information between equipment nodes and outputs the topology map.

[0024] Furthermore, the 3D dynamic model is dynamically updated to optimize the UAV inspection path, specifically including:

[0025] Edge computing nodes are set up within the substation to perform real-time iterative updates of the 3D dynamic model. The edge computing nodes acquire incremental data collected by lidar, infrared thermal imager, vibration sensor, and electromagnetic sensor in real time. Based on the currently received incremental data, the device status labels and device priorities of the current 3D dynamic model are corrected.

[0026] Build a reinforcement learning model, and train and optimize the reinforcement learning model;

[0027] Adjust the drone inspection path by training and optimizing a good reinforcement learning model.

[0028] Furthermore, the reinforcement learning model is based on a deep Q-network, and the state space of the deep Q-network... Including drone location and obstacle distribution Equipment priority With electromagnetic interference intensity Action space Including flight direction Speed ​​adjustment Switching with sensors Design reward function Balancing inspection efficiency and safety.

[0029] Furthermore, the reward function In the formula, These are the weighting coefficients.

[0030] Furthermore, a cross-modal attention mechanism is introduced to train and optimize the reinforcement learning model by using ResNet-50 as the teacher model and MobileNetV3 as the student model.

[0031] Furthermore, the inspection model includes visible light branches, infrared branches, vibration branches, and electromagnetic branches, wherein,

[0032] The visible light branch uses the YOLOv7 algorithm to process the visible light images captured by the camera on the drone.

[0033] Infrared branching processes infrared thermal images based on a threshold segmentation method;

[0034] The vibration branch converts the time-domain signal of the spectral data collected by the vibration sensor into a time-frequency domain signal through short-time Fourier transform. By analyzing the characteristic frequency components in the spectrum, it determines whether the equipment has a loosening fault.

[0035] The electromagnetic branch uses wavelet transform to process the signals acquired by the electromagnetic sensor, extracts partial discharge pulses, and classifies the discharge types according to the characteristics of the pulse signals for the purpose of assessing the insulation status of the equipment.

[0036] An autonomous unmanned aerial vehicle (UAV) inspection system for substations based on dynamic environmental perception includes:

[0037] The data acquisition unit performs data acquisition from lidar, infrared thermal imager, vibration sensor, and electromagnetic sensor.

[0038] The data spatial registration unit spatially registers the lidar point cloud data collected by the lidar with the infrared thermal image collected by the infrared thermal imager to generate fused data in a unified coordinate system.

[0039] The three-dimensional dynamic model building unit generates a three-dimensional dynamic model of the substation based on fused data, vibration sensor data, and electromagnetic sensor data under a unified coordinate system through a three-dimensional reconstruction algorithm.

[0040] The topology graph construction unit constructs a topology graph of the substation based on fused data and the basic topology structure of the substation under a unified coordinate system.

[0041] The inspection path generation unit marks the connection relationship of the equipment in the topology map and generates the UAV inspection path through path planning based on the priority of the equipment.

[0042] The inspection path optimization unit dynamically updates the 3D dynamic model and optimizes the inspection path of the UAV.

[0043] The defect detection unit uses drones to fly along optimized inspection paths and perform substation defect detection using inspection models.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] (1) A real-time updated three-dimensional dynamic model of the substation was constructed, incorporating information on temporary obstacles, new equipment and electromagnetic distribution, to solve the problem of poor adaptability of the static model and realize real-time perception of the substation environment and dynamic updating of the model.

[0046] (2) A reinforcement learning-driven route planning algorithm was developed, which dynamically adjusts the path by combining multi-sensor data to avoid electromagnetic interference areas and improves the autonomous obstacle avoidance capability of UAVs in complex electromagnetic and spatial environments.

[0047] (3) Design a cross-modal lightweight identification model to simultaneously process visible light, infrared, vibration and electromagnetic data, solve the problem of high false negative rate in single-modal analysis and improve the accuracy of fault identification. Attached Figure Description

[0048] Figure 1 This is a route diagram of the substation unmanned aerial vehicle (UAV) autonomous inspection method of the present invention. Detailed Implementation

[0049] Combination Figure 1 This embodiment provides a method for autonomous inspection of substations using unmanned aerial vehicles (UAVs) based on dynamic environmental perception. The implementation process is as follows:

[0050] (1) Data collection

[0051] LiDAR Data Acquisition. The lidar device onboard the drone is activated and operates based on the Time-of-Flight (TOF) principle. The lidar emits laser pulses into the surrounding environment and precisely measures the time it takes for the pulse to travel from emission to reflection from an object. Combining this with the speed of light, the distance between the object and the lidar is calculated. By continuously scanning the surrounding environment, a large number of distance data points are acquired, generating a high-precision 3D point cloud model.

[0052] Infrared thermal imager data acquisition. The infrared thermal imager on the drone is activated, using an uncooled vanadium oxide detector to sense the infrared radiation emitted by objects and convert it into electrical signals. The surface of the substation equipment is then scanned at a resolution of 640×480 to obtain temperature distribution information of the equipment surface.

[0053] Vibration sensor data acquisition. The vibration sensor onboard the drone operates using the properties of piezoelectric ceramic elements. When the equipment experiences mechanical vibration, it causes deformation of the piezoelectric ceramic element, thereby generating a corresponding electrical signal. The vibration sensor continuously acquires these electrical signals and converts them into spectral information of the equipment's mechanical vibration.

[0054] Electromagnetic sensor data acquisition. The electromagnetic sensor, based on the Rogowski coil principle, is activated to monitor the partial discharge intensity and electromagnetic radiation distribution around the substation equipment by sensing changes in the magnetic field around its internal conductors.

[0055] (2) Data fusion and 3D modeling

[0056] Data Fusion: Spatial registration of LiDAR point cloud and infrared thermal image solves the problem of spatiotemporal inconsistency in multi-sensor data, generating fused data in a unified coordinate system. After acquiring the 3D point cloud data generated by the LiDAR and the infrared thermal image data generated by the infrared imager, the Iterative Closest Point (ICP) algorithm is used for spatial registration. The basic idea of ​​this algorithm is to iteratively find corresponding points in the LiDAR point cloud and the infrared thermal image, and optimize coordinate alignment by minimizing the distance error between corresponding points. For each point in the LiDAR point cloud, the nearest point in the infrared thermal image is found as the corresponding point, and then the rotation vector and translation vector are calculated based on these corresponding points. The matching formula is as follows:

[0057]

[0058] In the formula, and These are the corresponding points in the lidar point cloud and the infrared thermal image, respectively. and These are the rotation and translation vectors, respectively.

[0059] Generating a 3D Dynamic Model: Registered lidar point cloud data and infrared thermal image data, combined with data from vibration and electromagnetic sensors, are used to generate a preliminary 3D map of the substation through a 3D reconstruction algorithm. This algorithm first filters and denoises the point cloud data, then uses the Delaunay triangulation method to generate a 3D mesh model. Finally, it maps the infrared thermal image, vibration spectrum, and electromagnetic distribution information onto the 3D mesh, forming a 3D map of the substation containing dynamic information such as equipment temperature distribution and electromagnetic radiation.

[0060] (3) Intelligent path planning

[0061] A topology map is constructed and inspection path planning is optimized. A graph neural network is introduced, using spatially registered data and the substation's basic topology as input, and the topology map as the output. Graph neural networks can process graph-structured data, constructing a more accurate topology map by learning the connections and feature information between nodes. The connections of key equipment are marked on the topology map, and combined with the equipment priority information, the inspection path of the UAV is planned, enabling the UAV to complete the inspection of all important equipment in the shortest time and in the most reasonable order. The substation's basic topology is well known in the field and will not be described in detail here.

[0062] (4) Real-time dynamic adjustment and optimization of the planned path

[0063] Real-time model iteration triggering: Edge computing nodes are set up within the substation. These nodes possess powerful computing capabilities and real-time data processing capabilities. Real-time model iteration is achieved through these edge computing nodes, triggering an update process every 5 minutes. The edge computing nodes receive incremental data in real time from LiDAR, infrared thermal imagers, vibration sensors, and electromagnetic sensors, reflecting the latest changes in equipment status.

[0064] Correcting Equipment Status Labels and Inspection Priorities: Based on received incremental data, the equipment status labels in the 3D dynamic model are corrected. Simultaneously, the inspection priorities of the equipment are reassessed based on the updated equipment status. For equipment with defects or risks, their inspection priority is increased to ensure that drones can conduct timely and detailed inspections of these devices, enabling the prompt detection and handling of potential problems.

[0065] Constructing a reinforcement learning framework: Building a decision system based on a deep Q-network (DQN), defining the state space, action space, and reward function; State space Including drone location Obstacle distribution Equipment priority With electromagnetic interference intensity Action space Including flight direction Speed ​​adjustment Switching with sensors Reward function The design balances inspection efficiency and safety, and is as follows:

[0066]

[0067] In the formula, These are the weighting coefficients.

[0068] Real-time flight path adjustment: The environmental map is updated every 10 seconds, and a local cost map is generated using sensor data to reflect the degree of danger in the environment surrounding the drone. Current state information is input into the DQN network, which calculates and outputs the optimal action sequence. Based on this optimal action sequence, the drone's flight path is dynamically adjusted, enabling it to avoid obstacles and dangerous areas and complete the inspection task along the optimal path. When the drone detects strong electromagnetic interference or encounters severe weather conditions, the electromagnetic obstacle avoidance strategy is automatically activated. Based on the intensity and distribution of electromagnetic interference, combined with the drone's current state, the drone's flight attitude and flight path are adjusted to ensure safe flight in complex environments.

[0069] Deep Q-Network Model Optimization: Knowledge distillation is employed, using ResNet-50 as the teacher model and MobileNetV3 as the student model. The basic idea of ​​knowledge distillation is to transfer the knowledge learned by the teacher model to the student model, enabling the student model to achieve near-teacher model performance while maintaining a smaller model size. During training, the high-level features and classification capabilities learned by the ResNet-50 model are transferred to the MobileNetV3 model, achieving model compression. The compressed MobileNetV3 model has fewer parameters and lower computational complexity. A cross-modal attention mechanism is introduced, allowing the model to automatically focus on key regions of the device when processing data from different modalities.

[0070] (5) Defect detection and closed-loop management

[0071] The drone flies along the optimized inspection path and performs substation defect detection using an inspection model, which includes:

[0072] ① Visible Light Branch: The visible light images (photos of substation equipment taken by a camera mounted on a drone during inspection) are processed using the YOLOv7 algorithm. YOLOv7 is a target detection algorithm that can more accurately locate structural defects such as cracks and corrosion on substation equipment. This algorithm achieves rapid location and identification of defects by dividing the image into a grid and predicting the bounding boxes and class probabilities of possible targets in each grid.

[0073] ② Infrared Branching: Infrared thermal images are processed using a threshold segmentation method. A threshold is set based on the normal operating temperature range of the equipment. Areas in the infrared thermal image with temperatures exceeding this threshold are marked as overheating hotspots, thereby locating potential overheating defects in the equipment.

[0074] ③ Vibration Branch: The spectral data collected by the vibration sensor is analyzed using short-time Fourier transform. Short-time Fourier transform can convert time-domain signals into time-frequency domain signals. By analyzing the characteristic frequency components in the spectrum, it can be determined whether the equipment has a loosening fault.

[0075] ④ Electromagnetic Branch: Wavelet transform is used to process the signals acquired by the electromagnetic sensor to extract partial discharge pulses. Wavelet transform has excellent time-frequency localization characteristics, which can effectively separate partial discharge pulse signals from complex electromagnetic signals. Then, based on the characteristics of the pulse signals, the discharge types are classified, providing a basis for assessing the insulation status of the equipment.

[0076] Through the detailed implementation process described above, this solution enables efficient, accurate, and safe inspection of substation equipment, timely detection of equipment defects and potential risks, and provides strong support for the stable operation of the substation.

[0077] This embodiment also provides a substation unmanned aerial vehicle (UAV) autonomous inspection system based on dynamic environment perception, including:

[0078] The data acquisition unit performs data acquisition from lidar, infrared thermal imager, vibration sensor, and electromagnetic sensor.

[0079] The data spatial registration unit spatially registers the lidar point cloud data collected by the lidar with the infrared thermal image collected by the infrared thermal imager to generate fused data in a unified coordinate system.

[0080] The three-dimensional dynamic model building unit generates a three-dimensional dynamic model of the substation based on fused data, vibration sensor data, and electromagnetic sensor data under a unified coordinate system through a three-dimensional reconstruction algorithm.

[0081] The topology graph construction unit constructs a topology graph of the substation based on fused data and the basic topology structure of the substation under a unified coordinate system.

[0082] The inspection path generation unit marks the connection relationship of the equipment in the topology map and generates the UAV inspection path through path planning based on the priority of the equipment.

[0083] The inspection path optimization unit dynamically updates the 3D dynamic model and optimizes the inspection path of the UAV.

[0084] The defect detection unit uses a drone to fly along an optimized inspection path and perform substation defect detection based on an inspection model.

[0085] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for autonomous inspection of substations using unmanned aerial vehicles (UAVs) based on dynamic environmental perception, characterized in that, Including the following steps: Data acquisition is performed using lidar, infrared thermal imager, vibration sensor, and electromagnetic sensor. Spatial registration is performed between lidar point cloud data collected by lidar and infrared thermal images collected by infrared thermal imager to generate fused data in a unified coordinate system. Based on fused data, vibration sensor data, and electromagnetic sensor data under a unified coordinate system, a three-dimensional dynamic model of the substation is generated through a three-dimensional reconstruction algorithm. A substation topology map is constructed based on fused data and the basic topology of the substation under a unified coordinate system. Mark the connection relationships of devices in the topology diagram, and generate drone inspection paths through path planning based on device priorities; Dynamically update the 3D dynamic model and optimize the drone inspection path; The drone flies along the optimized inspection path and uses the inspection model to detect defects in the substation.

2. The method for autonomous inspection of substations using unmanned aerial vehicles (UAVs) based on dynamic environmental perception according to claim 1, characterized in that, An iterative nearest-point algorithm is used to spatially register lidar data with infrared thermal imager data. The algorithm iteratively searches for corresponding points in the lidar point cloud and infrared thermal image, and determines the rotation and translation vectors by minimizing the distance error between corresponding points to perform coordinate alignment.

3. The method for autonomous inspection of substations by unmanned aerial vehicles based on dynamic environmental perception according to claim 1, characterized in that, The process of generating a 3D dynamic model of a substation using a 3D reconstruction algorithm includes: first, filtering and denoising the lidar point cloud data; then, using the Delaunay triangulation algorithm to generate a 3D mesh model; and finally, mapping the infrared thermal image, vibration spectrum, and electromagnetic distribution information onto the 3D mesh to form a 3D map of the substation containing equipment temperature distribution and electromagnetic radiation status labels.

4. The method for autonomous inspection of substations by unmanned aerial vehicles based on dynamic environmental perception according to claim 1, characterized in that, A graph neural network is used to construct the topology map of the substation. The spatially registered data and the basic topology of the substation are used as inputs. The graph neural network learns the connection relationships and feature information between equipment nodes and outputs the topology map.

5. The method for autonomous inspection of substations by unmanned aerial vehicles based on dynamic environmental perception according to claim 1, characterized in that, Dynamically update the 3D dynamic model and optimize the UAV inspection path, specifically including: Edge computing nodes are set up within the substation to perform real-time iterative updates of the 3D dynamic model. The edge computing nodes acquire incremental data collected by lidar, infrared thermal imager, vibration sensor, and electromagnetic sensor in real time. Based on the currently received incremental data, the device status labels and device priorities of the current 3D dynamic model are corrected. Build a reinforcement learning model, and train and optimize the reinforcement learning model; Adjust the drone inspection path by training and optimizing a good reinforcement learning model.

6. The method for autonomous inspection of substations by unmanned aerial vehicles based on dynamic environmental perception according to claim 5, characterized in that, The reinforcement learning model is based on a deep Q-network, and the state space of the deep Q-network... Including drone location and obstacle distribution Equipment priority With electromagnetic interference intensity Action space Including flight direction Speed ​​adjustment Switching with sensors Design reward function Balancing inspection efficiency and safety.

7. The method for autonomous inspection of substations by unmanned aerial vehicles based on dynamic environmental perception according to claim 6, characterized in that, The reward function In the formula, These are the weighting coefficients.

8. A method for autonomous inspection of substations using unmanned aerial vehicles (UAVs) based on dynamic environmental perception, as described in claim 5, is characterized in that... We adopted ResNet-50 as the teacher model and MobileNetV3 as the student model to introduce a cross-modal attention mechanism for training and optimizing the reinforcement learning model.

9. A method for autonomous inspection of substations using unmanned aerial vehicles (UAVs) based on dynamic environmental perception, as described in claim 1, is characterized in that... The inspection model includes visible light branch, infrared branch, vibration branch, and electromagnetic branch, among which, The visible light branch uses the YOLOv7 algorithm to process the visible light images captured by the camera on the drone. Infrared branching processes infrared thermal images based on a threshold segmentation method; The vibration branch converts the time-domain signal of the spectral data collected by the vibration sensor into a time-frequency domain signal through short-time Fourier transform. By analyzing the characteristic frequency components in the spectrum, it determines whether the equipment has a loosening fault. The electromagnetic branch uses wavelet transform to process the signals acquired by the electromagnetic sensor, extracts partial discharge pulses, and classifies the discharge types according to the characteristics of the pulse signals for the purpose of assessing the insulation status of the equipment.

10. A substation unmanned aerial vehicle (UAV) autonomous inspection system based on dynamic environment perception, implementing the method of any one of claims 1-9, characterized in that, include: The data acquisition unit performs data acquisition from lidar, infrared thermal imager, vibration sensor, and electromagnetic sensor. The data spatial registration unit spatially registers the lidar point cloud data collected by the lidar with the infrared thermal image collected by the infrared thermal imager to generate fused data in a unified coordinate system. The three-dimensional dynamic model building unit generates a three-dimensional dynamic model of the substation based on fused data, vibration sensor data, and electromagnetic sensor data under a unified coordinate system through a three-dimensional reconstruction algorithm. The topology graph construction unit constructs a topology graph of the substation based on fused data and the basic topology structure of the substation under a unified coordinate system. The inspection path generation unit marks the connection relationship of the equipment in the topology map and generates the UAV inspection path through path planning based on the priority of the equipment. The inspection path optimization unit dynamically updates the 3D dynamic model and optimizes the inspection path of the UAV. The defect detection unit uses drones to fly along optimized inspection paths and perform substation defect detection using inspection models.

Images