Power inspection method and system based on multiple unmanned aerial vehicles and multi-source data communication

Abstract

The invention discloses an electric power inspection method and system based on multiple unmanned aerial vehicles and multi-source data communication, and belongs to the technical field of electric power inspection, and the method comprises the steps: distributing inspection tasks for a plurality of unmanned aerial vehicles according to the obtained inspection event information; in the inspection process, regulating and controlling the unmanned aerial vehicle to perform video acquisition on the power equipment based on the inspection task to obtain inspection video data; after receiving the inspection video data transmitted by the unmanned aerial vehicle, fusing the inspection data, the operation state data of the power equipment and geographic information to obtain multi-source fusion data; and analyzing the multi-source fusion data through a preset power equipment anomaly identification model, positioning an abnormal point of the power facility and outputting an operation state evaluation result. Therefore, by implementing the method and the device, the problems that the power inspection efficiency is low and the abnormal condition of the power equipment cannot be timely and accurately identified in the prior art can be solved.

Description

Technical Field

[0001] This application belongs to the field of power line inspection technology, specifically relating to a power line inspection method and system based on multiple drones and multi-source data communication. Background Technology

[0002] To ensure the safe operation of the power grid, power generation equipment, and related power facilities, it is necessary to conduct regular power inspections of these power equipment in order to promptly identify and eliminate equipment defects and safety hazards, and effectively prevent major accidents such as power outages, tripping, equipment burnout, and even power grid collapse.

[0003] Traditional power line inspection methods primarily involve manual inspection and single-drone inspection. Manual inspection involves using equipment such as infrared thermometers, partial discharge detectors, ultraviolet imagers, and grounding resistance testers to perform detailed inspections of power equipment and determine its current condition. However, manual power line inspection is not only inefficient but also poses significant safety risks in extreme environments such as natural disasters, making it difficult to complete inspection tasks in a timely manner. Single-drone inspection typically uses only one drone to inspect power equipment on transmission lines. However, the limited communication range of a single drone makes it difficult to guarantee stable data transmission in complex terrain and adverse weather conditions. Furthermore, its data analysis capabilities are insufficient to quickly and accurately identify and handle anomalies in power facilities, failing to meet the requirements of real-time and precise inspection. Summary of the Invention

[0004] This application proposes a power inspection method and system based on multiple UAVs and multi-source data communication, which can solve the problems of low efficiency and inability to identify abnormal conditions of power equipment in the prior art.

[0005] The first aspect of this application provides a power line inspection method based on multiple drones and multi-source data communication, the method comprising:

[0006] Based on the obtained inspection event information, inspection tasks are assigned to several drones.

[0007] During the inspection, the drone is controlled to collect video data of the power equipment based on the inspection task.

[0008] After receiving the inspection video data transmitted by the drone, the inspection data, the operating status data of the power equipment, and the geographic information are integrated to obtain multi-source fused data;

[0009] The multi-source fusion data is analyzed by a preset power equipment anomaly identification model to locate anomalies in power facilities and output operational status assessment results.

[0010] The above solution utilizes multiple drones to collect video data at the inspection destination, significantly improving inspection efficiency compared to manual inspection. Furthermore, drones can perform inspections in extreme environments, expanding the application scenarios for inspections. After receiving the collected inspection video data, the backend combines it with sensor-collected power equipment operating status data and geographic information related to the inspection site for data fusion. This enables multi-dimensional analysis of the inspection data, providing more comprehensive and accurate information for subsequent anomaly handling decisions. Finally, a power equipment anomaly identification model trained on a large amount of normal and abnormal power equipment image data is used to analyze the fused data, accurately identifying the anomaly points and their locations. The analysis results are promptly communicated to relevant maintenance personnel, further improving inspection efficiency.

[0011] In one possible implementation of the first aspect, inspection tasks are assigned to several drones based on the acquired inspection event information, specifically as follows:

[0012] The task priority of the inspection task is obtained based on the fault severity, inspection mileage, and communication link quality in the inspection event information.

[0013] Based on the task priority, the number of drones to perform the inspection task is determined;

[0014] The inspection task is assigned to the drones within a preset time period using a task allocation algorithm corresponding to the number of drones; wherein the computational complexity of the task allocation algorithm is related to the number of drones.

[0015] The above scheme formulates appropriate inspection tasks based on inspection event information. Considering that the computational complexity of inspection tasks is high when there are a large number of drones, which may affect inspection efficiency, the scheme also selects a corresponding task allocation algorithm based on the number of drones to maximize the ability of drone swarms to take off quickly and perform inspection tasks.

[0016] In one possible implementation of the first aspect, during the inspection process, the drone is controlled to collect video data of the power equipment based on the inspection task, specifically as follows:

[0017] During the inspection, the regional images fed back by the drone are analyzed until the current location of the drone is determined to be the target area of ​​the inspection task. Then, the video acquisition device mounted on the drone is used to collect video of the power components of the power equipment to obtain inspection video data.

[0018] During video acquisition, the control precision of the video acquisition device is adjusted by a three-axis mechanical gimbal, while the acquired video image is corrected by a rolling shutter based on a preset delay using an electronic image stabilizer.

[0019] The above solution improves the quality of inspection video data by adjusting the precision of video shooting with a gimbal and performing delay correction on the captured data with an electronic image stabilizer, thus providing clearer and more accurate image data for subsequent analysis and processing.

[0020] In one possible implementation of the first aspect, after receiving the inspection video data transmitted by the UAV, the inspection data, the operating status data of the power equipment, and geographic information are fused to obtain multi-source fused data, specifically as follows:

[0021] The operating status data is collected by sensors deployed on the power equipment to obtain the geographical information related to the location of the power equipment;

[0022] The inspection video data and the operating status data are fused together to evaluate the operating status of the power equipment, resulting in operating status evaluation data.

[0023] The inspection video data and the geographic information are fused together by coordinate alignment to obtain the location of the anomaly point;

[0024] Multi-source fusion data is obtained based on operational status assessment data and anomaly locations.

[0025] The above solution uses a data fusion algorithm to fuse data from different sources and in different formats. By linking and complementing the data, it obtains more accurate data information related to anomalies, providing more comprehensive and sufficient data support for subsequent anomaly assessment and emergency plan generation.

[0026] In one possible implementation of the first aspect, the transmission process of the inspection video data is specifically as follows:

[0027] The inspection video data is transmitted using a preset broadband self-organizing network communication protocol;

[0028] The broadband self-organizing network is also used to transmit control commands to the UAV, as well as to transmit the operational status data and geographic information for data fusion.

[0029] Based on the timed feedback information from the UAV, the global topology of the broadband ad hoc network is calculated, and the routing table of the communication links is updated according to the global topology; wherein, the timed feedback information includes the remaining power of the communication link nodes and the communication link quality;

[0030] Based on the updated routing table, a relay node is determined in the communication link to assist the UAV in transmitting data.

[0031] The above solution takes into account the poor communication environment in remote areas. Therefore, a broadband ad hoc network is constructed between the drone and the control center to achieve stable and high-speed data transmission. The broadband ad hoc network has strong self-organizing and self-healing capabilities, ensuring reliable transmission of video data, control commands, and other multi-source data even without traditional communication infrastructure. Furthermore, it will find more reliable relay nodes for the drone to transmit data based on the communication quality and battery power of each node in the communication link, ensuring that the drone can continuously transmit data stably.

[0032] In one possible implementation of the first aspect, the multi-source fusion data is analyzed using a preset power facility anomaly identification model to locate anomalies in the power equipment and output an operational status assessment result, specifically:

[0033] By using the power facility anomaly identification model, edge-cloud collaborative analysis is performed on the multi-source fusion data to identify the type and location of the anomalies, and at the same time, the operational risks of power equipment are predicted to obtain the operational status assessment results that include safety hazard warnings.

[0034] The power facility anomaly identification model is trained using image data of normal and abnormal power equipment, which improves the identification accuracy of the power facility anomaly identification model for tower materials, insulators and conductors.

[0035] In one possible implementation of the first aspect, edge-cloud collaborative analysis is performed on the multi-source fused data, specifically as follows:

[0036] Defect detection is performed on the multi-source fused data at the edge, and a cropped image containing the anomalies is output.

[0037] The cropped image is verified a second time through the cloud, and combined with the geographic information corresponding to the cropped image, the type and location of the anomaly point are identified.

[0038] The above solution performs data analysis through cloud-edge collaboration. The edge provides the main computing resources to obtain the data analysis results, while the cloud is responsible for correcting the data analysis results, which greatly improves the accuracy of anomaly identification.

[0039] In one possible implementation of the first aspect, when the operational status assessment result indicates an anomaly, an emergency plan is generated by combining the information of the anomaly point.

[0040] The emergency plan is sent to the operation and maintenance personnel, and then the implementation of the emergency plan is tracked and recorded; wherein, the emergency plan includes an emergency reporting process, an emergency activation procedure, on-site emergency response measures, an information collection and reporting mechanism, and a post-processing plan.

[0041] Once an anomaly or potential security hazard is identified, the above-described solution will generate an appropriate emergency plan and notify operations and maintenance personnel for handling, ensuring timely resolution of anomalies or potential security risks. The handling process will also be recorded to provide a basis for subsequent emergency response assessments and improvements.

[0042] A second aspect of this application provides a power line inspection system based on multiple drones and multi-source data communication, the system comprising:

[0043] The inspection task allocation module is used to allocate inspection tasks to several drones based on the acquired inspection event information.

[0044] The inspection data acquisition module is used to control the drone to collect video data of the power equipment based on the inspection task during the inspection process, thereby obtaining inspection video data.

[0045] The data fusion module is used to receive the inspection video data transmitted by the UAV, and then fuse the inspection data, the operating status data of the power equipment, and geographical information to obtain multi-source fused data.

[0046] The anomaly analysis module is used to analyze the multi-source fusion data through a preset power equipment anomaly identification model, locate the anomalies in the power facilities, and output the operation status assessment results.

[0047] A third aspect of this application provides a terminal device, the device comprising: a terminal device including a processor and a memory, the memory storing a computer program, wherein the processor executes the computer program to implement the steps of the power inspection method based on multiple UAVs and multi-source data communication as described in any one of the embodiments of this application. Attached Figure Description

[0048] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram illustrating the specific process of a power line inspection method based on multiple drones and multi-source data communication, provided in an embodiment of this application.

[0050] Figure 2 This is a structural diagram of a power inspection system based on multiple drones and multi-source data communication, provided in an embodiment of this application.

[0051] Figure 3 This application provides a structural diagram of a terminal device according to one embodiment. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0054] First Embodiment

[0055] Power line inspection primarily involves checking, testing, recording, and evaluating electrical equipment within the power system. The vast power system requires regular inspections to promptly identify potential defects and eliminate hazard risks, preventing serious accidents such as widespread power outages. Due to the low efficiency and high cost of manual inspections, increasingly sophisticated intelligent inspection solutions are gaining traction. For example, drone inspections utilize drones equipped with high-definition cameras, infrared thermal imagers, and lidar to conduct comprehensive, all-around aerial photography and inspection of transmission lines. This method is highly efficient and low-risk, making it the current mainstream approach.

[0056] like Figure 1 As shown, to address the problems of low efficiency and inability to identify abnormal conditions of power equipment in the existing technology, the first embodiment of this application provides a detailed flowchart of a power inspection method based on multiple drones and multi-source data communication. The power inspection method based on multiple drones and multi-source data communication in this embodiment includes steps S1 to S4, which are detailed below:

[0057] Step S1: Assign inspection tasks to several drones based on the obtained inspection event information.

[0058] While drone inspections have greatly improved inspection efficiency, this method relies too heavily on the quality of the communication network. In extreme environments or remote mountainous areas where communication is not well developed, the communication range of drones is limited, making it impossible to transmit real-time images of power equipment to the control backend in a timely and complete manner. This results in the control backend being unable to quickly and accurately identify and handle abnormal situations in power facilities, failing to meet the requirements of real-time and accurate inspections.

[0059] To address the aforementioned shortcomings, this application employs multiple drones for inspection. Furthermore, to enhance inspection accuracy, it not only analyzes anomalies based on video data collected by the drones but also integrates multi-source data such as power equipment operating status and geographic information for fusion analysis. This provides more comprehensive and accurate information input, enabling more precise assessment of the operating status of power facilities and fault diagnosis. It helps to identify potential safety hazards in advance, formulate reasonable maintenance plans, reduce equipment failure rates, extend equipment lifespan, and improve the overall reliability and operational efficiency of the power system.

[0060] Therefore, the embodiments of this application can effectively improve the efficiency of power inspection, especially in areas with "three interruptions" (road interruption, network interruption, and power interruption) after natural disasters, where drones can be quickly deployed for inspection.

[0061] First, based on the fault severity, inspection mileage, and communication link quality obtained from the inspection event information, the task priority of the inspection task is determined. The formula for calculating the task priority is:

[0062] w = α × fault severity + β × reciprocal of inspection mileage + γ × communication link quality;

[0063] In the formula, w represents the task priority, and α, β, and γ are all weighting coefficients.

[0064] Optionally, in this embodiment of the application, the weighting coefficients are obtained by Bayesian optimization using 100 sets of historical fault data, so α, β, and γ are taken as 0.5, 0.3, and 0.2, respectively.

[0065] Then, based on the obtained task priority, the number of drones to perform the inspection task is determined. The higher the task priority, the more drones are assigned to perform the inspection task.

[0066] Considering that the computational complexity of the task allocation algorithm used to assign inspection tasks to drones is high when there are a large number of drones, which will affect the take-off time of drones and cause them to be unable to complete the inspection tasks in time, it is also necessary to select a corresponding task allocation algorithm according to the number of drones and quickly assign the inspection tasks to the drones within a preset time period so that the drones can take off in time.

[0067] Specifically, when the number of drones does not exceed 8, a task allocation algorithm based on Consensus-Based Bundle Algorithm is used to complete drone task allocation within 50ms through two nested loops. CBBA, short for Consensus-Based Bundle Algorithm, is a multi-agent cooperative algorithm used to solve resource allocation problems.

[0068] When the number of drones is greater than 8, the k-means spatial clustering algorithm is used to pre-group the drones to reduce the algorithm complexity, and then the task allocation is carried out according to the pre-grouping results.

[0069] Step S2, during the inspection process, based on the inspection task, the drone is controlled to collect video of the power equipment to obtain inspection video data.

[0070] During the flight of the drone, the drone flight control terminal broadcasts messages to the ground station at a frequency of 10 Hz through the communication protocol based on MAVLink 2.0 to inform the current flight status of the drone. The ground station uses a dual-link of 4G / 5G public network + broadband self-organizing network for message reception, and the link switching is based on the link quality LQ, which automatically switches when LQ < 0.4. Among them, LQ = SNR × (1 - PER); SNR is the signal-to-noise ratio, which is directly measured by the radio frequency chip at the drone end and returned in the Hello data packet, reflecting the signal quality of the current wireless channel. The larger the value, the better the signal quality; PER is the packet error rate, which is statistically calculated by the link layer at the drone end and periodically reported through the Hello data packet, and is used to measure the reliability of the current link. The smaller the value, the more stable the link.

[0071] Before collecting video data, the drone needs to identify the target area to determine the location where video data can be collected.

[0072] Specifically, a video acquisition device carried on the drone is used to take pictures in the current area to obtain an image of \(640\times640\times3\), and the image is analyzed. If the central pixel \((x, y)\) of the image satisfies \(0.2W < x < 0.8W\) and \(0.2H < y < 0.8H\) (\(W\) and \(H\) are the width and height of the image), it is considered that the video acquisition device is at the center of the line of sight, and the current position of the drone is the target area of the inspection task. At this time, the inspection video data can be obtained by taking pictures; otherwise, a three-axis mechanical pan-tilt is needed to finely adjust the video acquisition device.

[0073] Among them, the training data used by the inference model for analyzing the above image includes 120,000 power component images containing insulators, conductors, tower materials, and pins. These power component images can be used as a training set after data augmentation.

[0074] In addition, to improve the quality of video data during video capture, this embodiment of the application also adjusts the control precision of the video acquisition device using a three-axis mechanical gimbal to prevent device jitter. The device's six degrees of freedom operation is estimated in real time using an EKF fusion algorithm, and an electronic image stabilizer is used to correct the captured video image with an 8ms delay rolling shutter. After correction, the pixel jitter in the video image is less than 2px. The control precision includes pitch / roll / yaw control precision, with a range of ±0.01°; EKF stands for Extended Kalman Filter, primarily used for state estimation of nonlinear dynamic systems.

[0075] Furthermore, after acquiring the inspection video data, the preprocessing module on the UAV performs preprocessing operations such as image enhancement, noise reduction, and frame rate adjustment to improve video quality and provide clearer and more accurate image data for subsequent analysis and processing. Simultaneously, the preprocessing module is also responsible for compressing and encoding the inspection video data to reduce data transmission volume, improve transmission efficiency, and ensure that the video data can be transmitted back to the control backend in real time within limited communication bandwidth.

[0076] The encoder used is H.265Main@L4.1, with a GOP of 30 and a frame rate of 30fps. The adaptive bitrate is dynamically adjusted based on the RTT (Round-Trip Time). If the RTT is less than 50ms, the adaptive bitrate is 8Mbps; if the RTT is greater than or equal to 50ms and less than or equal to 200ms, the adaptive bitrate is 4Mbps; and if the RTT is greater than 200ms, the adaptive bitrate is 82Mbps.

[0077] The preprocessing module also encrypts the transmitted data and completes a handshake with the ground station 30 seconds before the drone takes off, with the session key being updated every 600 seconds.

[0078] The drone and the control center communicate via a pre-defined broadband ad hoc network to achieve stable and high-speed data transmission. This broadband ad hoc network possesses strong self-organizing and self-healing capabilities, ensuring reliable transmission of video data, control commands, and other multi-source data even without traditional communication infrastructure. Furthermore, the broadband ad hoc network is used to transmit control commands to the drone, as well as to transmit operational status data and geographic information for data fusion.

[0079] The communication protocol of broadband self-organizing networks includes the physical layer, the link layer, routing metrics, and self-healing mechanisms. The physical layer specification is 802.11ax (Wi-Fi 6) 2×2 MIMO, with a center frequency of 5.8GHz, a bandwidth of 40MHz, and a theoretical PHY rate of 1200Mbps; the link layer specification is BATMAN-adv V 2023.0, with a Hello packet interval of 1s and an OGM window size of 64; the expected transmission time (ETT) of the routing metric is calculated as follows: ETT = (1 / (1-PER)) × (Size / Data-Rate); where Size is the payload size of the data packet to be transmitted, in bits; Data-Rate is the available data rate of the physical layer of the current wireless link, in bits / s. In this embodiment, Data-Rate is dynamically determined by the UAV or relay node based on the real-time modulation and coding scheme (MCS) and channel bandwidth, and its value range is 6Mbit / s–1200Mbit / s; the link failure detection time of the self-healing mechanism is less than or equal to 2s, and the rerouting convergence time is less than or equal to 3s.

[0080] As an improvement to the above solution, this application embodiment will also find a relay node for the communication link that can stably transmit data based on the timed feedback information of the UAV. Specifically, the GPS data of the UAV, the remaining power of the communication link node and the quality of the communication link are obtained based on the timed feedback information. Then, the global topology of the broadband ad hoc network is calculated based on this information, and the routing table of the communication link is updated based on the global topology. The frequency of updating the routing table is 2Hz.

[0081] Based on the updated routing table, when the expected transmission time (ETT) between two nodes is greater than 200ms, the node with the lowest battery level (>30%) and shortest distance is automatically selected as the relay node for the drone. One of these two nodes is the drone currently performing the inspection task, and the other is the next-hop communication target that the drone needs to pass through or ultimately reach when transmitting data back. This could be: a fixed ground base station / ground station, a relay node temporarily deployed on a pole or vehicle, or other neighboring drones hovering in the air that can act as relays.

[0082] Therefore, "two nodes" generally refers to any one of the nodes in the drone's current communication link. ETT calculation is used to evaluate the expected transmission time of this one-hop link, thereby determining whether it is necessary to introduce a relay node with sufficient power and closer distance to optimize the route.

[0083] The timing feedback information is transmitted via Hello data packets.

[0084] Step S3: After receiving the inspection video data transmitted by the UAV, the inspection data, the operating status data of the power equipment, and the geographic information are fused to obtain multi-source fused data.

[0085] While receiving inspection video data from drones, it also collects operational status data of power equipment through sensors deployed on the power equipment, and obtains geographical information related to the location of the power equipment.

[0086] This system uses a message bus to receive data, then writes hot data to a storage pool and transfers cold data to a compatible object for storage. In this embodiment, hot data is defined as data received and stored for less than 7 days, while cold data is defined as data older than 7 days. During data storage, all sensors are timestamped with UTC, with a time synchronization error of <1ms. The time synchronization error is the maximum deviation between the UTC timestamp and the actual UTC standard time.

[0087] The timestamp feature marks each piece of data (temperature, infrared, visible light, location, etc.) collected by the sensor with a globally unified UTC absolute time. The purpose is to synchronize multi-source data from different hardware sources, which facilitates subsequent fusion analysis. After the inspection is completed, the corresponding data can be accurately retrieved based on the timestamp to trace back the device status at a certain moment or to verify abnormal alarms.

[0088] The inspection video data and the operational status data are fused to evaluate the operational status of the power equipment, resulting in operational status evaluation data. During the data fusion process, a weighted average fusion is performed based on the sensor type (e.g., temperature / infrared / visible light sensor) corresponding to the operational status data. The weighting coefficients are determined by the error covariance matrix, which is obtained through offline calibration using 50 sets of calibration experiments.

[0089] The error covariance matrix describes the statistical characteristics of measurement errors of various sensors when measuring the same physical quantity. Its diagonal elements represent the variance (i.e., the magnitude of its own noise) of the measurement values ​​of the i-th type of sensor, while the off-diagonal elements represent the correlation of measurement errors between different sensors. The error covariance matrix quantifies "which measurement is more accurate" and "whether errors are coupled," thus providing optimal weights during weighted averaging and fusion.

[0090] The 50 sets of calibration data here are obtained by simultaneously using all sensors to independently measure the state of the power equipment (temperature, hot spots, visible defects, etc.) with known reference values ​​in a controlled environment, collecting 50 sets of paired data of "sensor readings - reference true values". In other words, the UAV carries all sensors and repeats the data collection 50 times along the same flight path and at the same angle.

[0091] The inspection video data and the geographic information are fused using coordinate alignment to obtain the location of the anomaly. During the data fusion process, the video frames in the inspection video data are aligned with the geographic information using GIS coordinates. Specifically, the drone's pose data is acquired through the drone's satellite system, with an accuracy of ±2cm; the pixel coordinates of the video frames are projected into the WGS-84 coordinate system, with a reprojection error of <0.5m.

[0092] The WGS-84 coordinate system is the Global Geocentric-Fixed (ECEF) coordinate system, with its origin at the Earth's center of mass, the Z-axis pointing to the International Geodetic Reference Pole (CTP), and the X-axis pointing to the intersection of the Greenwich Meridian and the equator. The pose data, in the reprojection, provides the UAV's three-dimensional position (X, Y, Z) and attitude (pitch, roll, yaw) for each frame of video, thus obtaining the geographic coordinates (latitude, longitude, and elevation) corresponding to the pixels of each frame.

[0093] Multi-source fusion data is obtained based on operational status assessment data and anomaly locations.

[0094] In the above data fusion, fusing the inspection video data with the operating status data can evaluate and analyze the operating status of power equipment from multiple dimensions; fusing the inspection video data with the geographic information can be used to accurately determine the geographic location of power equipment and the specific location of anomalies.

[0095] Step S4: Analyze the multi-source fusion data using a preset power equipment anomaly identification model, locate the anomalies in the power facilities, and output the operational status assessment results.

[0096] By using a power facility anomaly identification model, edge-cloud collaborative analysis is performed on the multi-source fusion data to identify the type and location of the anomalies, while predicting the operational risks of power equipment, and obtaining the operational status assessment results that include safety hazard warnings.

[0097] The power facility anomaly identification model is trained using a large amount of normal and abnormal power equipment image data. This enables the model to automatically identify abnormal conditions of power equipment in inspection video data, such as conductor damage, insulator flashover, and transformer failure. It also uses other multi-source fusion data to predict and evaluate the operating status of power equipment, identify potential safety hazards in advance, and provide decision support for preventive maintenance.

[0098] Furthermore, the backbone network of the power facility anomaly identification model was fine-tuned to increase the number of inspections on three types of defect datasets: tower materials, insulators, and conductors, thereby improving the identification accuracy of these components. A two-layer LSTM was also used to model 30 days of historical temperature data and load current sequences to enable early warning of transformer overheating risks up to 72 hours in advance, allowing for timely detection of such safety hazards. The LSTM is a Long Short-Term Memory network, a type of recurrent neural network used for processing sequences.

[0099] The edge-cloud collaborative analysis consists of a cloud platform and an edge platform. The edge platform performs defect detection on multi-source fused data, identifying and locating video images containing anomalies, and cropping these anomalies as the detection result. The cloud platform then performs secondary verification on the detection result to improve the accuracy of defect detection. Furthermore, during the verification process, the geographic information corresponding to the cropped image is used to identify the type and location of the anomalies.

[0100] As an improvement to the above solution, this application, after outputting the operational status assessment results, if any abnormalities are found, generates an emergency plan based on the operational status assessment results and information on the abnormal points. The emergency plan is then communicated to maintenance personnel, including emergency command personnel and relevant emergency response personnel, through various means such as software interfaces, SMS, and emails. The emergency plan includes an emergency reporting process, emergency activation procedures, on-site emergency response measures, information collection and reporting mechanisms, and post-incident handling plans. Simultaneously, the implementation of the emergency plan is tracked and recorded to provide a basis for subsequent emergency response assessments and improvements.

[0101] Implementing the embodiments of this application has the following beneficial effects:

[0102] This application embodiment utilizes multiple drones to collect video data from the inspection destination, significantly improving inspection efficiency compared to manual inspection. Furthermore, drones can perform inspections in extreme environments, expanding the application scenarios for inspections. After receiving the collected inspection video data, the backend combines it with sensor-collected power equipment operating status data and geographic information related to the inspection location for data fusion. This enables multi-dimensional analysis of the inspection data, providing more comprehensive and accurate information for subsequent anomaly handling decisions. Finally, a power equipment anomaly identification model trained on a large amount of normal and abnormal power equipment image data is used to analyze the fused data, accurately identifying the anomaly points and their locations. The analysis results are promptly communicated to relevant maintenance personnel, further improving inspection efficiency.

[0103] Second Embodiment

[0104] Furthermore, in order to implement the power inspection system based on multiple UAVs and multi-source data communication corresponding to the above method embodiments, and to achieve the corresponding functions and technical effects, Figure 2 A structural diagram of a power inspection system based on multiple drones and multi-source data communication is provided. For ease of explanation, only the parts relevant to this embodiment are shown. The power inspection system based on multiple drones and multi-source data communication provided in this application embodiment includes:

[0105] The inspection task allocation module 201 is used to allocate inspection tasks to several drones based on the acquired inspection event information.

[0106] In this embodiment of the application, the task priority of the inspection task is obtained based on the fault severity, inspection mileage and communication link quality in the inspection event information;

[0107] Based on the task priority, the number of drones to perform the inspection task is determined;

[0108] The inspection task is assigned to the drones within a preset time period using a task allocation algorithm corresponding to the number of drones; wherein the computational complexity of the task allocation algorithm is related to the number of drones.

[0109] The inspection data acquisition module 202 is used to control the UAV to collect video data of the power equipment based on the inspection task during the inspection process, so as to obtain inspection video data.

[0110] In this embodiment of the application, during the inspection process, the regional images fed back by the UAV are analyzed until it is determined that the current location of the UAV is the target area of ​​the inspection task. Then, the video acquisition device mounted on the UAV is used to acquire video of the power components of the power equipment to obtain inspection video data.

[0111] During video acquisition, the control precision of the video acquisition device is adjusted by a three-axis mechanical gimbal, while the acquired video image is corrected by a rolling shutter based on a preset delay using an electronic image stabilizer.

[0112] The data fusion module 203 is used to receive the inspection video data transmitted by the UAV, and then fuse the inspection data, the operating status data of the power equipment, and the geographic information to obtain multi-source fused data.

[0113] In this embodiment of the application, the operating status data is collected by sensors deployed on the power equipment to obtain the geographical information related to the location of the power equipment;

[0114] The inspection video data and the operating status data are fused together to evaluate the operating status of the power equipment, resulting in operating status evaluation data.

[0115] The inspection video data and the geographic information are fused together by coordinate alignment to obtain the location of the anomaly point;

[0116] Multi-source fusion data is obtained based on operational status assessment data and anomaly locations.

[0117] The anomaly analysis module 204 is used to analyze the multi-source fusion data through a preset power equipment anomaly identification model, locate the anomaly points of the power facilities, and output the operation status assessment results.

[0118] In this embodiment of the application, the multi-source fusion data is analyzed by edge-cloud collaborative analysis through the power facility anomaly identification model to identify the type and location of the anomaly points, and at the same time predict the operation risk of the power equipment to obtain the operation status assessment result containing safety hazard warnings.

[0119] The power facility anomaly identification model is trained using image data of normal and abnormal power equipment, which improves the identification accuracy of the power facility anomaly identification model for tower materials, insulators and conductors.

[0120] In some embodiments, the inspection task allocation module 201 specifically comprises:

[0121] While drone inspections have greatly improved inspection efficiency, this method relies too heavily on the quality of the communication network. In extreme environments or remote mountainous areas where communication is not well developed, the communication range of drones is limited, making it impossible to transmit real-time images of power equipment to the control backend in a timely and complete manner. This results in the control backend being unable to quickly and accurately identify and handle abnormal situations in power facilities, failing to meet the requirements of real-time and accurate inspections.

[0122] To address the aforementioned shortcomings, this application employs multiple drones for inspection. Furthermore, to enhance inspection accuracy, it not only analyzes anomalies based on video data collected by the drones but also integrates multi-source data such as power equipment operating status and geographic information for fusion analysis. This provides more comprehensive and accurate information input, enabling more precise assessment of the operating status of power facilities and fault diagnosis. It helps to identify potential safety hazards in advance, formulate reasonable maintenance plans, reduce equipment failure rates, extend equipment lifespan, and improve the overall reliability and operational efficiency of the power system.

[0123] Therefore, the embodiments of this application can effectively improve the efficiency of power inspection, especially in areas with "three interruptions" (road interruption, network interruption, and power interruption) after natural disasters, where drones can be quickly deployed for inspection.

[0124] First, based on the fault severity, inspection mileage, and communication link quality obtained from the inspection event information, the task priority of the inspection task is determined. The formula for calculating the task priority is:

[0125] w = α × fault severity + β × reciprocal of inspection mileage + γ × communication link quality;

[0126] In the formula, w represents the task priority, and α, β, and γ are all weighting coefficients.

[0127] Optionally, in this embodiment of the application, the weighting coefficients are obtained by Bayesian optimization using 100 sets of historical fault data, so α, β, and γ are taken as 0.5, 0.3, and 0.2, respectively.

[0128] Then, based on the obtained task priority, the number of drones to perform the inspection task is determined. The higher the task priority, the more drones are assigned to perform the inspection task.

[0129] Considering that the computational complexity of the task allocation algorithm used to assign inspection tasks to drones is high when there are a large number of drones, which may affect the take-off time of drones and cause them to be unable to complete the inspection tasks in time, a corresponding task allocation algorithm is selected according to the number of drones to quickly assign the inspection tasks to the drones within a preset time period so that the drones can take off in time.

[0130] Specifically, when the number of drones does not exceed 8, a task allocation algorithm based on Consensus-Based Bundle Algorithm is used to complete drone task allocation within 50ms through two nested loops. CBBA, short for Consensus-Based Bundle Algorithm, is a multi-agent cooperative algorithm used to solve resource allocation problems.

[0131] When the number of drones is greater than 8, the drones are pre-grouped using the k-means spatial clustering algorithm to reduce the algorithm complexity, and then the tasks are assigned based on the pre-grouping results.

[0132] In some embodiments, the inspection data acquisition module 202 specifically comprises:

[0133] During drone flight, the drone's flight control unit broadcasts messages to the ground station at a frequency of 10Hz via the MAVLink 2.0-based communication protocol to inform the drone of its current flight status. The ground station uses a dual-link system of 4G / 5G public network + broadband self-organizing network for message reception. The link switching is based on link quality (LQ), and automatic switching occurs when LQ < 0.4. Wherein, LQ = SNR × (1 - PER).

[0134] Before collecting video data, the drone needs to identify the target area and determine the location where video data can be collected.

[0135] Specifically, a video acquisition device mounted on a drone is used to capture images in the current area, obtaining images of 640×640×3, and the images are analyzed. If the central pixel (x, y) of the image satisfies 0.2W < x < 0.8W and 0.2H < y < 0.8H (W and H are the width and height of the image), it is considered that the video acquisition device is at the center of the line of sight, and the current position of the drone is the target area of the inspection task. At this time, the inspection video data can be captured; otherwise, a three-axis mechanical gimbal is required to fine-tune the video acquisition device.

[0136] Among them, the training data used by the inference model for analyzing the above images includes 120,000 power component images containing insulators, conductors, tower materials, and pins. These power component images can be used as a training set after data augmentation.

[0137] In addition, in order to improve the quality of video data during video shooting, the embodiment of the present application also adjusts the control accuracy of the video acquisition device through a three-axis mechanical gimbal to prevent the jitter of the video acquisition device. The six-degree-of-freedom operation of the device is estimated in real time through the EKF fusion algorithm to use an electronic image stabilizer to perform rolling shutter correction with an 8ms delay on the captured video images. After correction, the pixel jitter of the video images is less than 2px. Among them, the control accuracy includes pitch / roll / yaw control accuracy, and its range is ±0.01°; EKF is an extended Kalman filter, mainly used for state estimation of nonlinear dynamic systems.

[0138] Further, after the inspection video data is captured, a preprocessing module on the drone performs preprocessing operations on the inspection video data, including image enhancement, denoising, frame rate adjustment, etc., to improve the video quality and provide clearer and more accurate image data for subsequent analysis and processing. At the same time, the preprocessing module is also responsible for compressing and encoding the inspection video data to reduce the data transmission volume, improve the transmission efficiency, and ensure that the video data can be transmitted back to the control background in real time under limited communication bandwidth.

[0139] Among them, the encoder model used is H.265 Main@L4.1, the coding unit GOP is 30, and the frame rate is 30fps. The adaptive bit rate is dynamically adjusted according to the RTT (round-trip delay). If the RTT is less than 50ms, the adaptive bit rate is 8Mbps. If the RTT is greater than or equal to 50ms and less than or equal to 200ms, the adaptive bit rate is 4Mbps; if the RTT is greater than 200ms, the adaptive bit rate is 82Mbps.

[0140] The preprocessing module also encrypts the transmitted data and completes a handshake with the ground station 30s before the drone takes off. The session key is updated every 600s.

[0141] The drone and the control center communicate via a pre-defined broadband ad hoc network to achieve stable and high-speed data transmission. This broadband ad hoc network possesses strong self-organizing and self-healing capabilities, ensuring reliable transmission of video data, control commands, and other multi-source data even without traditional communication infrastructure. Furthermore, the broadband ad hoc network is used to transmit control commands to the drone, as well as to transmit operational status data and geographic information for data fusion.

[0142] The communication protocol of a broadband self-organizing network includes a physical layer, a link layer, a routing metric, and a self-healing mechanism. The physical layer specification is 802.11ax (Wi-Fi 6) 2×2 MIMO, with a center frequency of 5.8 GHz, a bandwidth of 40 MHz, and a theoretical PHY rate of 1200 Mbps. The link layer specification is BATMAN-adv V 2023.0, with a Hello packet interval of 1 second and an OGM window size of 64. The expected transmission time (ETT) of the routing metric is calculated as: ETT = (1 / (1-PER)) × (Size / Data-Rate). The self-healing mechanism has a link failure detection time of less than or equal to 2 seconds and a rerouting convergence time of less than or equal to 3 seconds.

[0143] As an improvement to the above solution, this application embodiment will also find a relay node for the communication link that can stably transmit data based on the timed feedback information of the UAV. Specifically, the GPS data of the UAV, the remaining power of the communication link node and the quality of the communication link are obtained based on the timed feedback information. Then, the global topology of the broadband ad hoc network is calculated based on this information, and the routing table of the communication link is updated based on the global topology. The frequency of updating the routing table is 2Hz.

[0144] Based on the updated routing table, when the expected transmission time (ETT) between two nodes is greater than 200ms, the routing node with a battery level greater than 30% and the shortest distance is automatically selected as the relay node for the drone.

[0145] The timing feedback information is transmitted via Hello data packets.

[0146] In some embodiments, the data fusion module 203 specifically comprises:

[0147] While receiving inspection video data from drones, it also collects operational status data of power equipment through sensors deployed on the power equipment, and obtains geographical information related to the location of the power equipment.

[0148] This system uses a message bus to receive data, then writes hot data to a storage pool and transfers cold data to a compatible object for storage. In this embodiment, hot data is defined as data received and stored for less than 7 days, while cold data is defined as data older than 7 days. UTC timestamps are also applied to all sensors during data storage, with an error of <1ms.

[0149] The inspection video data and the operational status data are fused to evaluate the operational status of the power equipment, resulting in operational status evaluation data. During the data fusion process, a weighted average fusion is performed based on the sensor type (e.g., temperature / infrared / visible light sensor) corresponding to the operational status data. The weighting coefficients are determined by the error covariance matrix, which is calibrated offline through 50 sets of calibration experiments.

[0150] The inspection video data and the geographic information are fused using coordinate alignment to obtain the location of the anomaly. During the data fusion process, the video frames in the inspection video data are aligned with the geographic information using GIS coordinates. Specifically, the drone's pose data is acquired through the drone's satellite system, with an accuracy of ±2cm; the pixel coordinates of the video frames are projected into the WGS-84 coordinate system, with a reprojection error of <0.5m.

[0151] Multi-source fusion data is obtained based on operational status assessment data and anomaly locations.

[0152] In the above data fusion, fusing the inspection video data with the operating status data can evaluate and analyze the operating status of power equipment from multiple dimensions; fusing the inspection video data with the geographic information can be used to accurately determine the geographic location of power equipment and the specific location of anomalies.

[0153] In some embodiments, the anomaly analysis module 204 specifically comprises:

[0154] By using a power facility anomaly identification model, edge-cloud collaborative analysis is performed on the multi-source fusion data to identify the type and location of the anomalies, while predicting the operational risks of power equipment, and obtaining the operational status assessment results that include safety hazard warnings.

[0155] The power facility anomaly identification model is trained using a large amount of normal and abnormal power equipment image data. This enables the model to automatically identify abnormal conditions of power equipment in inspection video data, such as conductor damage, insulator flashover, and transformer failure. It also uses other multi-source fusion data to predict and evaluate the operating status of power equipment, identify potential safety hazards in advance, and provide decision support for preventive maintenance.

[0156] Furthermore, the backbone network of the power facility anomaly identification model was fine-tuned to increase the number of inspections on three types of defect datasets: tower materials, insulators, and conductors, thereby improving the identification accuracy of these components. A two-layer LSTM was also used to model 30 days of historical temperature data and load current sequences to enable early warning of transformer overheating risks up to 72 hours in advance, allowing for timely detection of such safety hazards. The LSTM is a Long Short-Term Memory network, a type of recurrent neural network used for processing sequences.

[0157] The edge-cloud collaborative analysis consists of a cloud platform and an edge platform. The edge platform performs defect detection on multi-source fused data, identifying and locating video images containing anomalies, and cropping these anomalies as the detection result. The cloud platform then performs secondary verification on the detection result to improve the accuracy of defect detection. Furthermore, during the verification process, the geographic information corresponding to the cropped image is used to identify the type and location of the anomalies.

[0158] As an improvement to the above solution, this application, after outputting the operational status assessment results, if any abnormalities are found, generates an emergency plan based on the operational status assessment results and information on the abnormal points. The emergency plan is then communicated to maintenance personnel, including emergency command personnel and relevant emergency response personnel, through various means such as software interfaces, SMS, and emails. The emergency plan includes an emergency reporting process, emergency activation procedures, on-site emergency response measures, information collection and reporting mechanisms, and post-incident handling plans. Simultaneously, the implementation of the emergency plan is tracked and recorded to provide a basis for subsequent emergency response assessments and improvements.

[0159] Implementing the embodiments of this application has the following beneficial effects:

[0160] This application embodiment utilizes multiple drones to collect video data from the inspection destination, significantly improving inspection efficiency compared to manual inspection. Furthermore, drones can perform inspections in extreme environments, expanding the application scenarios for inspections. After receiving the collected inspection video data, the backend combines it with sensor-collected power equipment operating status data and geographic information related to the inspection location for data fusion. This enables multi-dimensional analysis of the inspection data, providing more comprehensive and accurate information for subsequent anomaly handling decisions. Finally, a power equipment anomaly identification model trained on a large amount of normal and abnormal power equipment image data is used to analyze the fused data, accurately identifying the anomaly points and their locations. The analysis results are promptly communicated to relevant maintenance personnel, further improving inspection efficiency.

[0161] Furthermore, Figure 3 This is a structural diagram of a terminal device provided in one embodiment of this application. Figure 3 As shown, the terminal device 3 of this embodiment includes: at least one processor 30 (in... Figure 3 The present invention includes a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor. When the processor 30 executes the computer program 32, it can implement the steps of a power inspection method based on multiple UAVs and multi-source data communication as described in any one of the embodiments of this application.

[0162] The terminal device 3 may be a computing device such as a desktop computer, a cloud server, or a laptop computer, and the computing device may include, but is not limited to, a processor 30 and a memory 31. Figure 3 This is merely an example of terminal device 3 and does not constitute a limitation on terminal device 3. It may include more or fewer components than those shown in the figure.

[0163] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, or improvements made by those skilled in the art within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A power line inspection method based on multiple unmanned aerial vehicles (UAVs) and multi-source data communication, characterized in that, include: Based on the obtained inspection event information, inspection tasks are assigned to several drones. During the inspection, the drone is controlled to collect video data of the power equipment based on the inspection task. After receiving the inspection video data transmitted by the drone, the inspection data, the operating status data of the power equipment, and the geographic information are integrated to obtain multi-source fused data; The multi-source fusion data is analyzed by a preset power equipment anomaly identification model to locate anomalies in power facilities and output operational status assessment results.

2. The power line inspection method based on multiple UAVs and multi-source data communication according to claim 1, characterized in that, The process of assigning inspection tasks to several drones based on the acquired inspection event information is as follows: The task priority of the inspection task is obtained based on the fault severity, inspection mileage, and communication link quality in the inspection event information. Based on the task priority, the number of drones to perform the inspection task is determined; The inspection task is assigned to the drones within a preset time period using a task allocation algorithm corresponding to the number of drones; wherein the computational complexity of the task allocation algorithm is related to the number of drones.

3. The power line inspection method based on multiple UAVs and multi-source data communication according to claim 1, characterized in that, During the inspection process, the drone is controlled to collect video data of the power equipment based on the inspection task, specifically as follows: During the inspection, the regional images fed back by the drone are analyzed until the current location of the drone is determined to be the target area of ​​the inspection task. Then, the video acquisition device mounted on the drone is used to collect video of the power components of the power equipment to obtain inspection video data. During video acquisition, the control precision of the video acquisition device is adjusted by a three-axis mechanical gimbal, while the acquired video image is corrected by a rolling shutter based on a preset delay using an electronic image stabilizer.

4. The power line inspection method based on multiple UAVs and multi-source data communication according to claim 1, characterized in that, After receiving the inspection video data transmitted by the drone, the inspection data, the operating status data of the power equipment, and geographical information are fused to obtain multi-source fused data, specifically: The operating status data is collected by sensors deployed on the power equipment to obtain the geographical information related to the location of the power equipment; The inspection video data and the operating status data are fused together to evaluate the operating status of the power equipment, resulting in operating status evaluation data. The inspection video data and the geographic information are fused together by coordinate alignment to obtain the location of the anomaly point; Multi-source fusion data is obtained based on operational status assessment data and anomaly locations.

5. The power line inspection method based on multiple UAVs and multi-source data communication according to claim 1, characterized in that, The transmission process of the inspection video data is as follows: The inspection video data is transmitted using a preset broadband self-organizing network communication protocol; The broadband self-organizing network is also used to transmit control commands to the UAV, as well as to transmit the operational status data and geographic information for data fusion. Based on the timed feedback information from the UAV, the global topology of the broadband ad hoc network is calculated, and the routing table of the communication links is updated according to the global topology; wherein, the timed feedback information includes the remaining power of the communication link nodes and the communication link quality; Based on the updated routing table, a relay node is determined in the communication link to assist the UAV in transmitting data.

6. The power line inspection method based on multiple UAVs and multi-source data communication according to claim 1, characterized in that, The process of analyzing the multi-source fusion data using a preset power facility anomaly identification model to locate anomalies in power equipment and output operational status assessment results specifically involves: By using the power facility anomaly identification model, edge-cloud collaborative analysis is performed on the multi-source fusion data to identify the type and location of the anomalies, and at the same time, the operational risks of power equipment are predicted to obtain the operational status assessment results that include safety hazard warnings. The power facility anomaly identification model is trained using image data of normal and abnormal power equipment, which improves the identification accuracy of the power facility anomaly identification model for tower materials, insulators and conductors.

7. The power line inspection method based on multiple UAVs and multi-source data communication according to claim 1, characterized in that, The edge-cloud collaborative analysis of the multi-source fused data specifically includes: Defect detection is performed on the multi-source fused data at the edge, and a cropped image containing the anomalies is output. The cropped image is verified a second time through the cloud, and combined with the geographic information corresponding to the cropped image, the type and location of the anomaly point are identified.

8. The power line inspection method based on multiple UAVs and multi-source data communication according to any one of claims 1 to 7, characterized in that, When the operational status assessment result indicates an anomaly, an emergency plan is generated based on the information of the anomaly point. The emergency plan is sent to the operation and maintenance personnel, and then the implementation of the emergency plan is tracked and recorded; wherein, the emergency plan includes an emergency reporting process, an emergency activation procedure, on-site emergency response measures, an information collection and reporting mechanism, and a post-processing plan.

9. A power line inspection system based on multiple unmanned aerial vehicles (UAVs) and multi-source data communication, characterized in that, include: The system includes a patrol task allocation module, a patrol data acquisition module, a data fusion module, and an anomaly analysis module. The inspection task allocation module is used to allocate inspection tasks to several drones based on the acquired inspection event information. The inspection data acquisition module is used to control the drone to collect video data of the power equipment based on the inspection task during the inspection process, thereby obtaining inspection video data. The data fusion module is used to receive the inspection video data transmitted by the UAV, and then fuse the inspection data, the operating status data of the power equipment, and geographical information to obtain multi-source fused data. The anomaly analysis module is used to analyze the multi-source fusion data through a preset power equipment anomaly identification model, locate the anomalies in the power facilities, and output the operation status assessment results.

10. A terminal device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the power inspection method based on multiple UAVs and multi-source data communication as described in any one of claims 1 to 8.

Images