Unmanned aerial vehicle electric power inspection fault positioning method and system based on AI real-time identification

Abstract

The invention belongs to the technical field of intelligent inspection of power equipment, and particularly relates to an unmanned aerial vehicle power inspection fault positioning method and system based on AI real-time identification. The method comprises the following steps: constructing a target electric power inspection path according to path planning in an initial electric power inspection task in combination with unmanned aerial vehicle parameters and current external environment parameters; according to the target electric power inspection path, acquiring multi-modal sensing data of the electric power equipment at the electric power inspection point, and performing real-time analysis and identification on the acquired multi-modal sensing data by using an AI real-time identification technology to obtain defects of the electric power equipment; obtaining the position of the unmanned aerial vehicle, and performing mapping positioning on the defect according to the position of the unmanned aerial vehicle to form a fault point; processing defects and fault points of the power equipment by using a data transmission communication technology to obtain inspection data; and according to the inspection data, marking on the three-dimensional map platform to obtain a three-dimensional map platform which is visually displayed, and generating an inspection report. And the inspection efficiency and accuracy are improved.

Description

Technical Field

[0001] This application belongs to the field of intelligent power equipment inspection technology, and in particular relates to a method and system for fault location in power equipment inspection using drones based on AI real-time identification. Background Technology

[0002] Traditional power line inspection methods suffer from fundamental limitations in fault location. When inspectors discover equipment anomalies manually, they can only rely on transmission tower numbers or surrounding terrain features to describe the location. In vast mountainous and densely forested areas lacking clear landmarks, there is often a significant discrepancy between the recorded fault location and its actual spatial coordinates. This ambiguity is particularly pronounced in micro-defect scenarios. Traditional methods are completely incapable of obtaining precise three-dimensional coordinates for minute fault sources such as broken conductor strands or insulator breakdowns. This lack of spatial positioning capability forces maintenance teams to spend considerable time in complex environments performing secondary location re-analysis, severely delaying fault repair cycles.

[0003] While the application of mobile observation technology has improved spatial positioning capabilities, existing solutions still face profound challenges. In canyons or urban areas where satellite signals are blocked, location calibration results exhibit systematic shifts, which become unpredictable in complex electromagnetic environments. Furthermore, image data of equipment defects is often disconnected from geographic location information, lacking an effective spatial mapping mechanism, making it difficult to accurately correlate the three-dimensional coordinates of fault points with visual evidence. In the face of dynamic faults such as power line galloping, existing positioning systems are unable to capture the real-time movement trajectory of defect points, further deteriorating maintenance positioning accuracy.

[0004] The core dilemma currently facing the industry lies in the inherent contradiction between accuracy and efficiency in power equipment fault location. High-precision location requirements often necessitate complex computational processes, failing to meet the timeliness requirements of rapid response to power grid faults; static calibration methods are ill-suited to dynamically changing equipment scenarios; and macroscopic tower-level positioning accuracy cannot meet the maintenance needs of microscopic components. These structural contradictions have not been effectively resolved within the existing technological framework, particularly hindering the operation and maintenance challenges of rapid and accurate location of hidden faults in modern power grids. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and system for fault location in power line inspection based on AI real-time identification to address the aforementioned technical problems.

[0006] Firstly, this application provides a method for fault location in power line inspection using unmanned aerial vehicles (UAVs) based on real-time AI identification, including:

[0007] Based on the path planning in the initial power inspection task, combined with the UAV parameters and the current external environmental parameters, a target power inspection path is constructed, wherein the power inspection path includes multiple power inspection points.

[0008] Based on the target power inspection path, multimodal sensor data of the power equipment at the power inspection point is acquired, and AI real-time recognition technology is used to analyze and identify the acquired multimodal sensor data in real time to obtain the defects of the power equipment.

[0009] The location of the drone is obtained, and the defect is mapped and located based on the location of the drone to form the fault point;

[0010] Using data transmission communication technology, the defects and fault points of the power equipment are processed to obtain inspection data;

[0011] Based on the inspection data, the data is marked on a 3D map platform to obtain a visualized 3D map platform, which includes defect location, image evidence, and maintenance strategies.

[0012] The initial power inspection task is constructed and an inspection report is generated using the visualized 3D map platform.

[0013] In some feasible methods, the step of constructing a target power inspection path based on the path planning in the initial power inspection task, combined with UAV parameters and current external environmental parameters, includes:

[0014] The total distance of the power inspection task is determined based on the route planning in the initial power inspection task.

[0015] The actual distance traveled by the drone is obtained by acquiring its performance parameters and current external environmental parameters. The performance parameters include maximum flight time and cruising speed, and the current external environmental parameters include wind speed and wind direction.

[0016] The difference between the actual distance traveled by the drone and the total distance traveled by the power line inspection task is calculated to obtain the task coverage assessment.

[0017] Based on the task coverage assessment, the path planning in the initial power inspection task is adjusted to obtain the target power inspection path.

[0018] In some feasible methods, the step of acquiring multimodal sensor data of power equipment at the power inspection point according to the target power inspection path, and using AI real-time recognition technology to perform real-time analysis and recognition of the acquired multimodal sensor data to obtain the defects of the power equipment includes:

[0019] Based on the power inspection points along the target power inspection path, acquire multimodal sensor data for each power inspection point, and preprocess it to form a spatiotemporally synchronized multimodal dataset.

[0020] Feature extraction is performed on the spatiotemporally synchronized multimodal dataset to obtain the key features of the spatiotemporally synchronized multimodal dataset;

[0021] By using a deep learning model and combining it with a long short-term memory network, feature fusion is performed on the key features of a spatiotemporally synchronized multimodal dataset to obtain multimodal data features.

[0022] Using Bayesian inference, the joint probability of various features in multimodal data is calculated, and the comprehensive confidence score is obtained based on the confidence scores of various features.

[0023] The defects of the power equipment are determined based on the overall confidence level.

[0024] In some feasible methods, the step of obtaining the location of the drone and mapping and locating the defect based on the location of the drone to form a fault point includes:

[0025] The position of the UAV is obtained by using a GNSS / INS tightly coupled positioning system, and the UAV pose is obtained, wherein the UAV pose includes position and attitude;

[0026] Using the UAV pose, a matrix transformation is performed on the defect to form a mapping location, thereby obtaining the location of the defect;

[0027] The fault point is obtained based on the location of the defect.

[0028] In some feasible methods, the step of using data transmission communication technology to process the defects and fault points of the power equipment to obtain inspection data includes:

[0029] Using data transmission communication technology, the defects and fault points of the power equipment are structured and encapsulated to obtain encapsulated inspection data.

[0030] In some feasible methods, the step of marking on a 3D map platform based on the inspection data to obtain a visualized 3D map platform includes:

[0031] Based on the defects and fault points in the inspection data, extract them and mark them on the 3D map platform to obtain the visualized 3D map platform;

[0032] Based on the defects and fault points in the inspection data, the corresponding maintenance strategy is obtained by matching them with maintenance suggestions in the maintenance suggestion library.

[0033] The maintenance strategy is displayed on the visualized 3D map platform.

[0034] In some feasible methods, the steps of constructing the initial power inspection task and generating the inspection report using the visualized 3D map platform include:

[0035] The initial power inspection task is constructed using the visualized 3D map platform, the inspection plan, and the defects found in historical inspection processes.

[0036] An inspection report is generated based on the target power inspection route.

[0037] Secondly, this application provides an AI-based real-time identification-based UAV power line inspection fault location system, applied to the aforementioned AI-based real-time identification-based UAV power line inspection fault location method. The system includes:

[0038] A safe flight control system is used to construct a target power inspection path based on the path planning in the initial power inspection task, combined with UAV parameters and current external environmental parameters, wherein the power inspection path includes multiple power inspection points.

[0039] The AI ​​real-time identification system is used to acquire multimodal sensor data of power equipment at the power inspection point according to the target power inspection path, and to use AI real-time identification technology to analyze and identify the acquired multimodal sensor data in real time to obtain the defects of the power equipment.

[0040] A fault precision location system is used to obtain the location of the UAV and, based on the location of the UAV, map and locate the defect to form a fault point;

[0041] The data transmission communication and control system is used to process the defects and fault points of the power equipment using data transmission communication technology to obtain inspection data.

[0042] A real-time monitoring and control system is used to mark on a three-dimensional map platform based on the inspection data to obtain a visualized three-dimensional map platform, wherein the visualized three-dimensional map platform includes defect location, image evidence, and maintenance strategy;

[0043] The inspection task management system is used to construct the initial power inspection task and generate inspection reports using the visualized 3D map platform.

[0044] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned steps of the AI-based real-time identification method for locating faults in power line inspections by unmanned aerial vehicles.

[0045] Fourthly, this application provides a computer program that, when executed by a processor, implements the aforementioned steps of the AI-based real-time identification method for locating faults in unmanned aerial vehicle power line inspections.

[0046] Beneficial Effects: A method for fault location in UAV power line inspection based on real-time AI recognition includes: constructing a target power line inspection path based on path planning in the initial power line inspection task, combined with UAV parameters and current external environmental parameters, wherein the power line inspection path includes multiple power line inspection points; acquiring multimodal sensor data of power equipment at the inspection points based on the target power line inspection path, and using AI real-time recognition technology to analyze and identify the collected multimodal sensor data in real time to obtain the defects of the power equipment; acquiring the position of the UAV, and mapping and locating the defects based on the UAV position to form fault points; processing the defects and fault points of the power equipment using data transmission communication technology to obtain inspection data; marking the inspection data on a 3D map platform to obtain a visualized 3D map platform, wherein the visualized 3D map platform includes defect location, image evidence, and maintenance strategies; and using the visualized 3D map platform to construct the initial power line inspection task and generate an inspection report. By employing the methods described above, drones combined with AI real-time recognition technology can achieve rapid inspection and accurate fault location of power equipment, improving inspection efficiency and accuracy, which has significant practical implications. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a framework diagram of a drone-based real-time identification power line inspection fault location system in one embodiment.

[0049] Figure 2 This is a flowchart of a method for locating faults in power grid inspections using drones based on real-time AI identification, as described in one embodiment. Detailed Implementation

[0050] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all couplings of one or more of the associated listed items.

[0052] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another.

[0053] The following explanations of some terms used in this application are provided to aid in understanding the application:

[0054] GNSS / INS tightly coupled positioning systems are navigation technologies that integrate Global Navigation Satellite System (GNSS) and Inertial Navigation System (INS) through deep data fusion algorithms. Its core is to directly fuse raw GNSS observation data (such as pseudorange and carrier phase) with INS inertial measurement data (acceleration and angular velocity) to achieve higher accuracy and more robust positioning and navigation, especially in environments with limited GNSS signals (such as urban canyons and tunnels), where it significantly outperforms loosely coupled schemes.

[0055] A Convolutional Neural Network (CNN) is a deep learning model specifically designed for processing grid-structured data (such as images, videos, and audio). Its core technology automatically learns spatial or temporal patterns in data through local perception (convolution) and hierarchical feature extraction.

[0056] Long Short-Term Memory (LSTM) is an improved recurrent neural network (RNN) that solves the long-term dependency problem of traditional RNNs through a gating mechanism and is good at processing time-series data (such as text, speech, and sensor sequences).

[0057] Principal Component Analysis (PCA) is an unsupervised linear dimensionality reduction method that projects the original high-dimensional data into a low-dimensional space through orthogonal transformation, preserving the direction of the largest variance in the data (i.e., the "principal component"). It is used to remove redundancy, compress data, or visualize high-dimensional structures.

[0058] Bayesian inference is a probabilistic and statistical method based on Bayes' theorem, used to dynamically update the confidence level of a hypothesis (or event) after observing new evidence (data). Its core idea is to combine prior knowledge with new evidence to obtain a posterior probability distribution.

[0059] Dempster-Shafer Theory is a generalized probabilistic framework used to address the problem of fusing uncertainty and incomplete information. Its core principle is to quantify the confidence in multiple hypotheses using a belief function and a likelihood function, and to combine evidence from different sources using Dempster's combination rule.

[0060] The Softmax layer is a commonly used normalized exponential function in neural networks. It is used to transform any real vector into a probability distribution and is suitable for multi-class classification tasks.

[0061] Sigmoid is a sigmoid function that compresses the input to the (0,1) interval and is suitable for binary classification or probability output.

[0062] PointNet++ (PointNet++ Deep Neural Network) is a deep learning model for point cloud data processing. It is an improvement on PointNet and directly processes unordered point cloud data through hierarchical feature learning and local structure extraction.

[0063] Multi-source navigation fusion technology refers to the technology of improving the accuracy, reliability and robustness of navigation systems by integrating data from multiple navigation sensors (such as GNSS, INS, vision, lidar, etc.) and using information fusion algorithms (such as Kalman filtering, particle filtering, neural networks, etc.).

[0064] Tightly Coupled Algorithm is a multi-source sensor data fusion method in the field of navigation. It specifically refers to the deep fusion of raw observation data from different navigation sensors (such as GNSS and inertial navigation system INS) at the bottom layer, and the improvement of system accuracy through joint state estimation.

[0065] SLAM (Simultaneous Localization and Mapping) is a core technology that enables mobile devices (such as robots, drones, and autonomous vehicles) to build environmental maps in real time and determine their own location in unknown environments.

[0066] Edge computing is a distributed computing paradigm that migrates data processing, storage, and application services from traditional centralized cloud computing centers to the network edge (such as base stations, routers, local servers, or the terminal devices themselves) closer to the data source or terminal device, in order to reduce latency, save bandwidth, and improve data privacy and real-time performance.

[0067] Wireless Ad Hoc Network (WAN) is a decentralized, self-organizing, multi-hop dynamic topology wireless communication network that does not rely on fixed infrastructure (such as base stations or routers). It is formed autonomously by mobile or fixed nodes to achieve direct or relay communication between nodes.

[0068] Traditional power line inspections mainly rely on manual inspections on foot or by car, which are easily affected by complex terrain, bad weather, and high-altitude operations. This method is inefficient, costly, dangerous, and prone to omissions. In recent years, the introduction of drone technology has effectively improved the efficiency of power line inspections. However, existing drone inspection systems have shortcomings in fault identification and location, such as poor real-time performance, low identification accuracy, and inaccurate positioning.

[0069] Therefore, this application utilizes drones equipped with high-definition cameras and sensors, combined with AI real-time recognition technology, to achieve rapid inspection and accurate fault location of power equipment, thereby improving inspection efficiency and accuracy, which has significant practical implications.

[0070] This application mainly consists of an unmanned aerial vehicle (UAV) power line inspection fault location platform and a ground control center power line inspection management platform, such as Figure 1 As shown, in order to achieve accurate fault location for UAV power line inspection based on AI real-time identification, this application integrates multiple subsystems, including a safe flight control system, an AI real-time identification system, a fault accurate location system, a data transmission communication and control system, a real-time monitoring and control system, and an inspection task management system.

[0071] like Figure 2 As shown, in a first aspect, this application provides a method for fault location in power line inspection using drones based on real-time AI recognition, the method comprising:

[0072] The S100 constructs the target power inspection path based on the path planning in the initial power inspection task, combined with the UAV parameters and the current external environmental parameters.

[0073] The power inspection route includes multiple power inspection points.

[0074] Specifically, constructing a target power line inspection path may include the following steps:

[0075] S101, determine the total distance of the power inspection task based on the path planning in the initial power inspection task.

[0076] Specifically, the initial power inspection task should at least include which power inspection points need to be passed during the power inspection process, and form the flight path of the UAV based on these inspection points. If the path is a straight line, no adjustment is needed. However, if multiple power inspection points in the path are not straight lines, but may be irregular shapes such as polygons, then although a path plan is given based on the initial power inspection task, it is still necessary to further plan based on the UAV's own parameters and the location environment of the power inspection points. In other words, the total distance of the power inspection task must first be determined based on the initial power inspection task, so as to facilitate the planning of the target power inspection path in subsequent steps.

[0077] S102, acquire the performance parameters of the drone and the current external environment parameters to obtain the actual distance traveled by the drone.

[0078] The performance parameters include maximum range and cruising speed, and the current external environmental parameters include wind speed and wind direction.

[0079] Specifically, the maximum flight time and flight speed of a drone determine its flight range. However, this range does not take into account the influence of external environmental factors. Therefore, it is also necessary to calculate the impact of the external environment on the flight range.

[0080] A formula for actual usable range is constructed, in which the maximum range is multiplied by the cruising speed and then by the environmental correction factor to obtain the actual usable range.

[0081] It should be noted that in actual work, the initial power line inspection task has certain limitations. The limitation is that it does not take into account the current external environmental parameters, or in other words, the external environmental parameters obtained are theoretical weather forecasts, which are different from the environment when the drone performs the inspection. Therefore, it is necessary to make adjustments based on the drone's parameters and the current external environmental parameters.

[0082] S103, calculate the difference between the actual distance traveled by the UAV and the total distance traveled by the power line inspection task to obtain a task coverage assessment.

[0083] Specifically, after obtaining the actual available distance and the total distance of the power line inspection task in the aforementioned steps, the difference is calculated to obtain the difference distance. Based on this difference distance, the coverage of the power line inspection task is evaluated. For example, a negative difference distance indicates that the actual available distance is less than the total distance of the power line inspection task, requiring the abandonment of some power line inspection points or the increase of the drone's battery power. A positive difference distance indicates that the actual available distance is greater than the total distance of the power line inspection task, thus the path planning in the initial power training task does not need to be adjusted, and execution can proceed according to the planned path.

[0084] S104. Based on the task coverage assessment, the path planning in the initial power inspection task is adjusted to obtain the target power inspection path.

[0085] Specifically, in the aforementioned steps, if some power inspection points are abandoned, the original route plan will be changed. In other words, the route plan formed after abandoning some power inspection points will be used as the target power inspection route.

[0086] In addition, during drone inspections, the current external environmental parameters can be obtained in real time, and the inspection route of the drone can be corrected using an environmental correction coefficient. When the environmental correction coefficient has a significant impact, causing the target power inspection path to be unable to be completed, the drone will promptly abandon the uninspected power inspection points and leave sufficient power for return.

[0087] In this embodiment, the safe flight control system can construct the target power inspection path based on the path planning in the initial power inspection task, combined with the UAV parameters and the current external environmental parameters. Furthermore, the safe flight control system is mounted on the UAV, which carries an inspection pod. The inspection pod integrates a variety of advanced sensor devices, including relevant image and video data acquisition sensors, as well as high-precision fault location data acquisition sensors and data transmission communication modules.

[0088] The drone has safe flight control capabilities, long endurance, and stable flight performance; it can adapt to complex terrain and harsh weather conditions to ensure the smooth execution of inspection tasks; and it supports intelligent path planning and autonomous flight.

[0089] S200: Based on the target power inspection path, acquire multimodal sensor data of the power equipment at the power inspection point, and use AI real-time recognition technology to analyze and identify the acquired multimodal sensor data in real time to obtain the defects of the power equipment.

[0090] Determining the defects of the power equipment may include the following steps:

[0091] S201, based on the power inspection points along the target power inspection path, acquire multimodal sensor data for each power inspection point, and preprocess it to form a spatiotemporally synchronized multimodal dataset.

[0092] Specifically, when the drone arrives at the power inspection point, the multi-sensor collaborative data acquisition system simultaneously activates a visible light camera, an infrared thermal imager, and a lidar, which respectively capture equipment images, temperature data, and 3D point cloud data. Spatiotemporal alignment control uses a tightly coupled GNSS / INS positioning system to bind spatial coordinates with timestamps, and the 3D point cloud data generated by the lidar scan is bound to coordinates.

[0093] S202, perform feature extraction on the spatiotemporally synchronized multimodal dataset to obtain the key features of the spatiotemporally synchronized multimodal dataset.

[0094] Specifically, image feature extraction involves using a Convolutional Neural Network (CNN) to extract edge texture features (such as insulator crack outlines and conductor strand breakage patterns) from images. Temperature feature extraction utilizes a Long Short-Term Memory (LSTM) network to analyze the time-varying patterns of infrared temperature sequences (such as temperature rise rate) and generate temperature feature vectors (short-term temperature change patterns, e.g., temperature changes within 10 seconds). Point cloud feature extraction employs surface curvature algorithms to quantify equipment deformation features (such as displacement and tilt angle) or PointNet++ for feature extraction. Additionally, a 3D modeling program can be used to model the 3D point cloud data, forming a partial 3D model. This allows for the marking of fault points in the partial 3D model in subsequent steps, enabling fault visualization. This not only reduces the space occupied by constructing the partial 3D model but also saves modeling time.

[0095] It should be noted that a partial 3D model represents a portion of the shape within a complete 3D model; that is, this partial 3D model is part of the overall 3D model. For example:

[0096] Using historical inspection data, construct three-dimensional models of each power inspection point;

[0097] Feature extraction is performed on the 3D models of each power inspection point to obtain the feature vectors of the 3D models of each power inspection point.

[0098] Construct a 3D model of the initial power inspection points;

[0099] The feature vectors of the 3D models of each power inspection point are loaded into the initial 3D model of the power inspection point to form the corresponding 3D model of each power inspection point. The actual location is then bound to obtain the target 3D model of the power inspection point.

[0100] Specifically, a complete 3D model of each power inspection point is pre-constructed using historical inspection data. Next, feature extraction is performed to obtain the features of each power inspection point. Then, an initial 3D model of each power inspection point is constructed. Finally, feature vectors are loaded onto the initial model of each power inspection point to form a corresponding 3D model. The purpose of this operation is to create multiple 3D models of power inspection points using only the initial model and the feature vectors of several other power inspection points, eliminating the need to construct a separate 3D model for each power inspection point. In other words, all power inspection points share a single initial 3D model, saving storage overhead. Furthermore, during data transmission, only the location and 3D point cloud data of the power inspection points need to be transmitted to locate them, reducing unnecessary overhead.

[0101] The 3D point cloud data is converted into a partial 3D model. Then, the coordinates of the 3D point cloud data are matched with the coordinates of the target 3D model of the power inspection point pre-loaded in the inspection pod of the UAV. This enables the positioning of the 3D point cloud data, so that it can be visualized when a 3D map platform is involved in subsequent steps.

[0102] It should be noted that an initial 3D model of a power inspection point represents a type of power equipment, such as iron towers and substations. In other words, if different types of power equipment need to be inspected during the inspection process, an initial 3D model of a power inspection point needs to be used for each type of power equipment.

[0103] S203 utilizes a deep learning model and combines it with a long short-term memory network to perform feature fusion on key features of a spatiotemporally synchronized multimodal dataset to obtain multimodal data features.

[0104] Specifically, principal component analysis (PCA) is used to compress redundant information and merge the three types of feature vectors to obtain multimodal data feature vectors.

[0105] It's important to note that while multimodal data feature vectors are merged, the independence of the features remains. Here, fusion refers to establishing a logical connection, while the data itself remains independent. For example, fully connected layers learn cross-modal feature relationships (such as the weight mapping between CNN texture features and LSTM temperature rise features). Features with associated characteristics, already having their respective weights, are further weighted; that is, additional weights are assigned to the related features. For example, if feature A has weight B and feature C has weight D, and feature A and feature C are associated, then feature A and feature C are assigned a shared weight E.

[0106] S204 utilizes Bayesian inference to perform joint probability calculations on various features in multimodal data features, and obtains the comprehensive confidence score based on the confidence scores of various features.

[0107] Specifically, after obtaining the multimodal data features in the aforementioned steps, Bayesian inference is used to calculate the joint probability of the detection results of different sensors for the three types of features in the multimodal data features to determine whether there is an anomaly. Then, the Dempster-Shafer Theory is used to calculate the comprehensive confidence level by allocating confidence levels for different sensors.

[0108] It should be noted that, within the Bayesian inference framework, the prior probability distributions of various faults are first established based on historical operational data. Then, a deep learning model independently analyzes data from various sensors (visible light cameras, infrared thermal imagers, and LiDAR), outputting corresponding probability estimates. Further, a Softmax layer is added after the convolutional neural network to predict image defect probabilities; a Sigmoid activation function is applied to the output of the last layer in the Long Short-Term Memory network to predict temperature defect probabilities; and features are extracted from 3D point cloud data using the PointNet++ architecture, and finally converted into fault probabilities using the Sigmoid activation function. In other words, fault probabilities for images, temperature, and 3D point cloud data are first established. Next, the detection probability results are jointly calculated using Bayesian formulas. For example, for the current image defect probability, the prior probability distribution and the current image defect probability are jointly calculated using conventional Bayesian formulas to obtain the final image defect probability. Similarly, the defect probabilities for temperature and 3D point cloud data are not elaborated further. After obtaining the defect probabilities of the final image, temperature, and 3D point cloud data, these are relatively independent, thus forming single-level defects. However, if there is defect overlap within a single level, the defect probabilities are converted into a unified confidence level representation using the DS evidence theory, ultimately yielding a comprehensive confidence level, i.e., a comprehensive confidence level at the comprehensive level. The DS evidence theory is a conventional processing method, and this application does not limit it.

[0109] It should also be noted that, regarding the joint probability calculation using Bayes' theorem, although it calculates a single-level limitation, the fully connected layer learning cross-modal feature associations, mentioned in S203, directly influences the prior distribution modeling within the Bayesian framework due to the cross-modal association weights. Traditional single-modal analysis can only establish priors based on historical data of that modality, while associated features enable the system to construct a joint prior distribution across modalities. In the multimodal data analysis of power inspection systems, the calculation of joint probability embodies the core idea of ​​cross-modal feature fusion. This calculation constructs a comprehensive probability assessment through the product of three key elements:

[0110] CNN image feature probability represents the probability of defect presence based on image analysis, derived from the analysis of surface features of power equipment by a convolutional neural network. For example, the probability of detecting an insulator crack may reach 0.87.

[0111] The temporal probability of LSTM reflects the likelihood of temperature anomalies and is obtained by analyzing the time-varying patterns of infrared data through a Long Short-Term Memory (LSTM) network. For example, the probability of overheating at a certain connection point might be 0.76.

[0112] Cross-modal correlation weights quantify the correlation strength between features of different modalities, with values ​​ranging from [0,1]. When image defects coincide with temperature anomalies, this weight may reach 0.95.

[0113] This joint probabilistic calculation mechanism enhances the reliability of Bayesian inference through feature correlation. When there is a physical correlation between image features and temperature features (such as a crack causing partial discharge and heating), the correlation weight significantly increases the final probability value; conversely, if there is no clear correlation between features, the weight is reduced to avoid misjudgment. This preserves the independent judgment criteria for each modality while achieving collaborative verification of evidence through correlation analysis.

[0114] S205, based on the comprehensive confidence level, the defects of the power equipment are obtained.

[0115] Specifically, the overall confidence value in step S205 has two levels of representation: the overall level and the individual level. If there is defect overlap at the overall level, an overall confidence value needs to be generated. However, in actual work, defects do not necessarily overlap. Therefore, the overall confidence value also represents the individual level.

[0116] Next, a confidence threshold can be set. Overall confidence scores greater than the confidence threshold will be marked as defects in the power equipment; otherwise, they will be discarded or highlighted.

[0117] It should be noted that, according to the target power inspection path, the multimodal sensor data of the power equipment at the power inspection point is acquired, and AI real-time recognition technology is used to analyze and identify the acquired multimodal sensor data in real time to obtain the defect steps of the power equipment.

[0118] The AI ​​real-time identification system utilizes various sensors mounted on a drone inspection pod, employing AI real-time identification models (such as CNN, LSTM, and PointNet++) to achieve real-time fault identification of power equipment. The drone carries a rich array of fault identification sensors, which can be equipped as needed, including high-definition visible light cameras, infrared thermal imaging cameras, ultrasonic sensors, and LiDAR, to collect images, temperature data, and 3D point cloud data of the power equipment, detecting abnormal phenomena such as insulation aging and partial discharge. The AI ​​real-time identification technology uses deep learning algorithms to analyze the collected images and data in real time, identifying defects and fault characteristics of the power equipment. The AI ​​real-time identification system can identify various typical defects such as insulator damage, broken conductor strands, and hardware corrosion.

[0119] S300: Obtain the location of the drone, and map and locate the defect based on the location of the drone to form a fault point.

[0120] Specifically, the formation of a fault point may include the following steps:

[0121] S301, using a GNSS / INS tightly coupled positioning system, the position of the UAV is obtained, and the UAV pose is obtained.

[0122] The drone pose includes position and attitude.

[0123] Specifically, the three-dimensional spatial state of the UAV is calculated in real time through deep coupling of GNSS satellite signals (position reference) and INS inertial sensors (motion compensation).

[0124] Position can include location: longitude, latitude and elevation, and attitude: pitch angle, roll angle and yaw angle.

[0125] S302, using the UAV pose, perform matrix transformation on the defect to form a mapping location and obtain the position of the defect.

[0126] Specifically, mapping and positioning are achieved through the principle of geometric projection transformation. This process essentially establishes a mathematical mapping relationship from image space to geographic space, using the UAV's six-degree-of-freedom pose (three-dimensional position + three-axis attitude) as the transformation reference to convert the pixel coordinates of defects in the image into real-world geographic coordinates.

[0127] Furthermore, the transformation from the pixel coordinate system to the camera coordinate system is achieved through inverse perspective projection based on the camera intrinsic parameter matrix; the transformation from the camera coordinate system to the UAV body coordinate system is achieved through a rigid body transformation matrix; and the transformation from the body coordinate system to the geographic coordinate system is finally performed by combining the UAV pose. The same applies to the sensors, so it will not be elaborated further.

[0128] S303, Based on the location of the defect, the fault point is obtained.

[0129] Specifically, after obtaining the location of the defect, this location is marked as the fault point. For example, the defect location can be marked in a partial 3D model composed of 3D point cloud data to form the fault point.

[0130] It should be noted that the precise fault location system is used to perform the steps of acquiring the location of the UAV and mapping and locating the defect based on the UAV's location to form the fault point. The precise fault location system combines multi-source navigation fusion technologies such as GNSS, inertial navigation system (INS), and lidar to improve the positioning accuracy of the UAV. It can fuse satellite positioning data with inertial sensor data through a tightly coupled algorithm and perform data fusion using 3D point cloud data from lidar, achieving high-precision positioning in complex environments and accurately determining the fault location.

[0131] In addition, the fault location system can also use the visual sensors of the drone, such as cameras, combined with the visual positioning algorithm SLAM for real-time positioning and map building. By matching the collected images with the pre-built map, the fault location can be accurately determined, making it convenient for maintenance personnel to quickly locate and handle the fault.

[0132] S400 uses data transmission communication technology to process the defects and fault points of the power equipment to obtain inspection data.

[0133] Specifically, obtaining inspection data may include the following steps:

[0134] Using data transmission communication technology, the defects and fault points of the power equipment are structured and encapsulated to obtain encapsulated inspection data.

[0135] Specifically, the defect feature data {defect type, confidence level, timestamp} and the fault point information {longitude, latitude, elevation, error range} are encapsulated. For example, the Protobuf binary serialization protocol is used to encapsulate them into a unified data frame format, thereby obtaining inspection data that conforms to the data transmission protocol.

[0136] Next, edge computing technology is used to transmit the encapsulated inspection data packets to the ground control station via 4G / 5G or drone self-organizing networks.

[0137] It should be noted that the data transmission communication and control system is used to perform the steps of processing the defects and fault points of the power equipment using data transmission communication technology to obtain inspection data.

[0138] The data transmission communication and control system uses edge computing technology for data preprocessing, and realizes real-time processing and analysis of the collected data. It also transmits the collected data and identification results to the ground control center in real time for data storage and verification analysis.

[0139] The data transmission communication and control system can utilize 4G / 5G communication technology and wireless ad hoc networking technology to transmit images, videos, and sensor data collected by the UAV to the ground control center in real time. The low latency of the 5G network ensures the real-time performance and stability of data transmission. Ad hoc networking technology enables network communication in areas without mobile communication networks, achieving efficient and reliable data communication. This application does not improve upon 4G / 5G communication technology or wireless ad hoc networking technology; it only utilizes existing 4G / 5G communication technology and wireless ad hoc networking technology for data transmission.

[0140] It should also be noted that when generating inspection data, only images and 3D point cloud data of defects and their surrounding preset range can be saved, while others can be discarded, reducing the overhead during data transmission.

[0141] S500: Based on the inspection data, mark the data on the 3D map platform to obtain a visualized 3D map platform.

[0142] The visualized 3D map platform includes defect location, image evidence, and maintenance suggestions.

[0143] Specifically, obtaining the visualized 3D map platform may include the following steps:

[0144] S501, based on the defects and fault points in the inspection data, extract them and mark them on the 3D map platform to obtain the visualized 3D map platform.

[0145] Specifically, the 3D map platform contains 3D models of multiple power inspection points.

[0146] In the inspection data, defects and fault points are matched with the 3D model of power inspection points in the 3D map platform based on their coordinates. The matching is performed using similarity matching, and the closest coordinates are selected for matching. After the matching is completed, the data is loaded into the 3D model to obtain the visualized 3D map platform.

[0147] S502, based on the defects and fault points in the inspection data, match them with the maintenance suggestions in the maintenance suggestion library to obtain the corresponding maintenance strategy.

[0148] Specifically, a maintenance suggestion library is built based on historical maintenance plans and recommendations. Next, the defects are matched with the contents of the maintenance suggestion library, and the matching result with the highest similarity is selected to form a maintenance strategy.

[0149] S503, The maintenance strategy is displayed on the visualized 3D map platform.

[0150] The maintenance strategy is linked to the defects and presented on the visualized 3D map platform.

[0151] It should be noted that the real-time monitoring and control system is used to mark the inspection data on a 3D map platform, resulting in a visualized 3D map platform. The real-time monitoring and control system has data receiving, processing, analysis, and storage functions, enabling real-time monitoring of the UAV status and inspection data, providing fault diagnosis and analysis functions, and determining the fault location and type. Simultaneously, the control center can remotely control the UAV's flight and photography, receive and process data information transmitted from the UAV power inspection platform, combine it with SLAM 3D maps to display fault locations and related information, provide necessary human-machine interfaces, and visualize the fault location information and image data on the 3D map platform, facilitating maintenance personnel to quickly understand the fault location and specific situation.

[0152] S600 utilizes the visualized 3D map platform to construct the initial power inspection task and generate an inspection report.

[0153] Specifically, step S600 may include the following steps:

[0154] S601, using the visualized 3D map platform, the inspection plan, and the defects in the historical inspection process, the initial power inspection task is constructed.

[0155] Based on existing power grid asset information, scheduled inspections, and past fault records, a specific and executable power inspection task is created. This action can be manually set or automatically generated based on the inspection plan.

[0156] S602, Generate an inspection report based on the target power inspection path.

[0157] After completing the inspection task, a structured inspection report is generated based on the collected actual inspection data (along a predetermined path or covered area).

[0158] It should be noted that the inspection task management system is used to construct the initial power inspection task and generate inspection reports using the visualized 3D map platform.

[0159] The inspection task management system, through a control center PC platform client or mobile application, enables the creation, route planning, task allocation, and execution monitoring of inspection tasks; it also allows for automatic route planning and manual route adjustments. Upon completion of an inspection task, it automatically generates an inspection report containing defect location, image evidence, and maintenance recommendations. The report may include defect distribution maps, a detailed defect list, and maintenance strategies.

[0160] Example

[0161] Drone inspection work

[0162] The main tasks of drone inspection include inspection mission planning, automated inspection, data collection and transmission, and the workflow is as follows:

[0163] The process of planning inspection tasks involves planning the flight path of drones automatically or manually based on the geographical information of power lines and inspection requirements.

[0164] The automated inspection process involves the drone flying automatically along a preset route, collecting images and sensor data from the power equipment in real time; during the flight, the drone's status, such as battery level, flight position, and flight speed, is monitored in real time.

[0165] The data acquisition and transmission process involves real-time identification and analysis of images and sensor data collected by the UAV, followed by real-time transmission of key information such as fault location to the ground control center via 4G / 5G networks or wireless ad hoc network communication technology. Simultaneously, the data can also be stored in the UAV's local storage device and uploaded after the mission is completed.

[0166] AI real-time identification and fault location work

[0167] The work of AI real-time identification and fault location mainly includes image preprocessing, real-time fault identification, and precise fault location. The workflow is as follows:

[0168] Image preprocessing involves performing preprocessing operations such as denoising and contrast enhancement on the acquired images to improve image quality.

[0169] Real-time fault identification utilizes AI algorithms (such as deep learning models) to analyze pre-processed images in real time, identifying defects and fault types in power equipment. The identification results include information such as defect type and location coordinates.

[0170] The precise fault location work mainly involves combining GNSS and INS fusion positioning data from UAVs with 3D point cloud data from lidar to accurately locate the fault point. The location results can be visualized on the 3D map software in the control center.

[0171] Inspection task management and report generation

[0172] The work content of inspection task management and report generation mainly includes inspection task management and inspection report generation, and the work process is as follows:

[0173] Inspection task management is done through a PC platform client or mobile application. Maintenance personnel can create inspection tasks, plan routes, assign tasks to drones, and monitor task execution in real time.

[0174] The inspection report generation process takes place after the inspection task is completed. Based on the AI ​​recognition results and location data, the system generates an inspection report, which includes a defect distribution map, a detailed defect list, and maintenance strategies.

[0175] It should be noted that the algorithms mentioned in this application are all conventional algorithms, and no improvements have been made to the algorithm's calculation process or calculation method.

[0176] In summary, this application presents a method for fault location in power equipment inspection using drones equipped with high-definition cameras and sensors, combined with AI real-time recognition technology. This method achieves rapid inspection and accurate fault location of power equipment, improving inspection efficiency and accuracy, and has significant practical implications. Its technical capabilities are summarized below:

[0177] (1) High accuracy of fault location: Multimodal data fusion can make up for the deficiencies of single sensor data and improve the accuracy of fault detection;

[0178] (2) Strong system robustness: The multi-modal fusion algorithm for power inspection in this scheme can work stably under complex environmental conditions, which can reduce false alarms and missed alarms;

[0179] (3) High real-time recognition and efficiency: Through AI-based edge computing and real-time processing, combined with multimodal fusion algorithms, it can respond quickly and improve inspection efficiency.

[0180] (4) Comprehensive system monitoring: Multimodal data fusion can simultaneously monitor parameters such as the appearance image, temperature data and three-dimensional point cloud data of power equipment, providing more comprehensive on-site equipment status information.

[0181] like Figure 1 As shown, in a second aspect, this application provides an AI-based real-time identification-based UAV power line inspection fault location system, applied to the aforementioned AI-based real-time identification-based UAV power line inspection fault location method. The system includes:

[0182] A safe flight control system is used to construct a target power inspection path based on the path planning in the initial power inspection task, combined with UAV parameters and current external environmental parameters, wherein the power inspection path includes multiple power inspection points.

[0183] The AI ​​real-time identification system is used to acquire multimodal sensor data of power equipment at the power inspection point according to the target power inspection path, and to use AI real-time identification technology to analyze and identify the acquired multimodal sensor data in real time to obtain the defects of the power equipment.

[0184] A fault precision location system is used to obtain the location of the UAV and, based on the location of the UAV, map and locate the defect to form a fault point;

[0185] The data transmission communication and control system is used to process the defects and fault points of the power equipment using data transmission communication technology to obtain inspection data.

[0186] A real-time monitoring and control system is used to mark on a three-dimensional map platform based on the inspection data to obtain a visualized three-dimensional map platform, wherein the visualized three-dimensional map platform includes defect location, image evidence, and maintenance strategy;

[0187] The inspection task management system is used to construct the initial power inspection task and generate inspection reports using the visualized 3D map platform.

[0188] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned steps of the AI-based real-time identification method for locating faults in power line inspections by unmanned aerial vehicles.

[0189] Fourthly, this application provides a computer program that, when executed by a processor, implements the aforementioned steps of the AI-based real-time identification method for locating faults in unmanned aerial vehicle power line inspections.

[0190] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0191] The various embodiments in this disclosure are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0192] The scope of protection of this disclosure is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its scope and spirit. If such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A method for fault location in power line inspection using unmanned aerial vehicles (UAVs) based on real-time AI identification, characterized in that, The methods include: Based on the path planning in the initial power inspection task, combined with the UAV parameters and the current external environmental parameters, a target power inspection path is constructed, wherein the power inspection path includes multiple power inspection points. Based on the target power inspection path, multimodal sensor data of the power equipment at the power inspection point is acquired, and AI real-time recognition technology is used to analyze and identify the acquired multimodal sensor data in real time to obtain the defects of the power equipment. The location of the drone is obtained, and the defect is mapped and located based on the location of the drone to form the fault point; Using data transmission communication technology, the defects and fault points of the power equipment are processed to obtain inspection data; Based on the inspection data, the data is marked on a 3D map platform to obtain a visualized 3D map platform, which includes defect location, image evidence, and maintenance strategies. The initial power inspection task is constructed and an inspection report is generated using the visualized 3D map platform.

2. The method for fault location in UAV power line inspection based on real-time AI recognition as described in claim 1, characterized in that, The step of constructing the target power inspection path based on the path planning in the initial power inspection task, combined with UAV parameters and current external environmental parameters, includes: The total distance of the power inspection task is determined based on the route planning in the initial power inspection task. The actual distance traveled by the drone is obtained by acquiring its performance parameters and current external environmental parameters. The performance parameters include maximum flight time and cruising speed, and the current external environmental parameters include wind speed and wind direction. The difference between the actual distance traveled by the drone and the total distance traveled by the power line inspection task is calculated to obtain the task coverage assessment. Based on the task coverage assessment, the path planning in the initial power inspection task is adjusted to obtain the target power inspection path.

3. The method for fault location in UAV power line inspection based on real-time AI recognition according to claim 1, characterized in that, The step of acquiring multimodal sensor data of power equipment at the power inspection point according to the target power inspection path, and using AI real-time recognition technology to analyze and identify the collected multimodal sensor data in real time to obtain the defects of the power equipment includes: Based on the power inspection points along the target power inspection path, acquire multimodal sensor data for each power inspection point, and preprocess it to form a spatiotemporally synchronized multimodal dataset. Feature extraction is performed on the spatiotemporally synchronized multimodal dataset to obtain the key features of the spatiotemporally synchronized multimodal dataset; By using a deep learning model and combining it with a long short-term memory network, feature fusion is performed on the key features of a spatiotemporally synchronized multimodal dataset to obtain multimodal data features. Using Bayesian inference, the joint probability of various features in multimodal data is calculated, and the comprehensive confidence score is obtained based on the confidence scores of various features. The defects of the power equipment are determined based on the overall confidence level.

4. The method for fault location in UAV power line inspection based on real-time AI recognition according to claim 1, characterized in that, The step of obtaining the location of the drone and mapping and locating the defect based on the location of the drone to form a fault point includes: The position of the UAV is obtained by using a GNSS / INS tightly coupled positioning system, and the UAV pose is obtained, wherein the UAV pose includes position and attitude; Using the UAV pose, a matrix transformation is performed on the defect to form a mapping location, thereby obtaining the location of the defect; The fault point is obtained based on the location of the defect.

5. The method for fault location in UAV power line inspection based on real-time AI recognition according to claim 1, characterized in that, The step of processing the defects and fault points of the power equipment using data transmission communication technology to obtain inspection data includes: Using data transmission communication technology, the defects and fault points of the power equipment are structured and encapsulated to obtain encapsulated inspection data.

6. The method for fault location in UAV power line inspection based on real-time AI recognition according to claim 1, characterized in that, The step of marking the inspection data on a 3D map platform to obtain a visualized 3D map platform includes: Based on the defects and fault points in the inspection data, extract them and mark them on the 3D map platform to obtain the visualized 3D map platform; Based on the defects and fault points in the inspection data, the corresponding maintenance strategy is obtained by matching them with maintenance suggestions in the maintenance suggestion library. The maintenance strategy is displayed on the visualized 3D map platform.

7. The method for fault location in UAV power line inspection based on real-time AI recognition according to claim 1, characterized in that, The steps of constructing the initial power inspection task and generating the inspection report using the visualized 3D map platform include: The initial power inspection task is constructed using the visualized 3D map platform, the inspection plan, and the defects found in historical inspection processes. An inspection report is generated based on the target power inspection route.

8. A drone-based power line inspection fault location system based on real-time AI recognition, characterized in that, The system, which is applied to the AI-based real-time identification-based drone power line inspection fault location method according to any one of claims 1-7, comprises: A safe flight control system is used to construct a target power inspection path based on the path planning in the initial power inspection task, combined with UAV parameters and current external environmental parameters, wherein the power inspection path includes multiple power inspection points; The AI ​​real-time identification system is used to acquire multimodal sensor data of power equipment at the power inspection point according to the target power inspection path, and to use AI real-time identification technology to analyze and identify the acquired multimodal sensor data in real time to obtain the defects of the power equipment. A fault precision location system is used to obtain the location of the UAV and, based on the location of the UAV, map and locate the defect to form a fault point; The data transmission communication and control system is used to process the defects and fault points of the power equipment using data transmission communication technology to obtain inspection data. A real-time monitoring and control system is used to mark on a three-dimensional map platform based on the inspection data to obtain a visualized three-dimensional map platform, wherein the visualized three-dimensional map platform includes defect location, image evidence, and maintenance strategy; The inspection task management system is used to construct the initial power inspection task and generate inspection reports using the visualized 3D map platform.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the AI-based real-time identification method for fault location in unmanned aerial vehicle power line inspection as described in any one of claims 1 to 7.

10. A computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the AI-based real-time identification method for fault location in power line inspection by unmanned aerial vehicles as described in any one of claims 1 to 7.

Images