Transformer fault detection method and system based on unmanned aerial vehicle inspection

Abstract

The invention discloses a transformer fault detection method and system based on unmanned aerial vehicle inspection, and the method comprises the steps: constructing a detection range of a transformer, and planning a flight path of an unmanned aerial vehicle cluster according to the detection range; then, acquiring infrared data and visible light data of the transformer collected when the unmanned aerial vehicle cluster flies along the flight path; and then, calling a heating detection model integrated with a multi-scale feature extraction module, a decoupling multi-modal learning module, a spatial offset module, an offset guide module, a decoupling feature fusion module, an attention module, a detection head module and the like, and performing detection according to the infrared data and the visible light data to obtain a heating detection result. According to the embodiment of the invention, technologies of detection range construction, path planning, multi-modal data acquisition, fusion detection and the like are integrated, the core problems of difficult target identification, low path planning efficiency, poor data fusion effect and the like in unmanned aerial vehicle power inspection are systematically solved, and the detection precision of the abnormal heating transformer is effectively improved.

Description

Technical Field

[0001] The embodiments of this application relate to, but are not limited to, the field of image processing technology, and in particular to a transformer fault detection method and system based on UAV inspection. Background Technology

[0002] With the widespread application of unmanned aerial vehicle (UAV) technology in power system inspection, it can efficiently acquire data from power equipment by carrying various types of sensors. However, in transformer inspection, due to limitations such as small target size, complex background, and varied shapes, existing inspection technologies struggle to achieve high-precision and high-efficiency fault identification.

[0003] In collaborative inspections using swarm drones, flight path planning technology has significant shortcomings. While Particle Swarm Optimization (PSO) can alleviate the local optima problem, the time-consuming grouping process leads to slow system response. Ant Colony Optimization (ACO), which uses an improved artificial potential field, has stringent environmental requirements, limited business adaptability and task completion rate, and thus affects inspection efficiency.

[0004] In terms of target detection technology, traditional manually designed feature methods lack sufficient accuracy and robustness in complex environments. While deep learning algorithms such as R-CNN and YOLO series have made progress, they still suffer from missed detections and false positives for small targets like transformers due to their low pixel ratio and blurred details. Meanwhile, power line inspections have high real-time requirements, necessitating a balance between accuracy and speed to meet the demands of high-speed data processing.

[0005] In the field of data fusion, traditional feature-level and decision-level fusion methods are unable to fully tap the potential of multimodal data and have poor detection performance for small transformer targets in complex scenarios.

[0006] Currently, there are still technical bottlenecks in unmanned aerial vehicle (UAV) power inspection in terms of target identification, path planning, and data fusion for transformer fault detection. There is an urgent need to innovate methods to improve the reliability and efficiency of power system inspection. Summary of the Invention

[0007] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0008] This application provides a transformer fault detection method and system based on UAV inspection, which solves problems such as difficulty in target identification, low efficiency in path planning, and poor data fusion effect, thereby effectively improving the reliability and efficiency of detecting abnormally heated transformers at night.

[0009] In a first aspect, embodiments of this application provide a transformer fault detection method based on UAV inspection, comprising: constructing a detection range for the transformer and planning a flight path for a UAV swarm based on the detection range; acquiring infrared and visible light data of the transformer collected by the UAV swarm while flying along the flight path; calling a pre-trained heat detection model to perform detection based on the infrared and visible light data to obtain a heat detection result; the heat detection model includes a multi-scale feature extraction module, a decoupled multimodal learning module, a spatial offset module, an offset guidance module, a decoupled feature fusion module, an attention enhancement module, and a detection head module, wherein the multi-scale feature extraction module is used to extract features from the infrared and visible light data to obtain visible light multimodal features; the decoupled multimodal learning module is used to combine the visible light features and the infrared and visible light data... External features are aligned and fused to obtain visible light mode-invariant (VMI) features and infrared mode-invariant (IRM) features. The spatial offset module performs spatial offset processing on the VMI features and IRM features to obtain a spatial offset amount. The offset guidance module performs offset compensation and adaptive alignment processing based on the VMI features, IRM features, and the spatial offset amount to obtain visible light mode-specific (VLS) features and IRM mode-specific (IRM) features. The decoupled feature fusion module performs feature fusion processing on the IRM features, the VMI features, the VMI features, and the IRM features to obtain fused features. The attention enhancement module performs feature enhancement processing on the fused features to obtain a feature map. The detection head module detects the feature map to obtain a heat detection result.

[0010] In conjunction with the first aspect, in one embodiment of this application, the multi-scale feature extraction module includes a first feature extraction unit and a second feature extraction unit; the first feature extraction unit is used to extract features from the infrared data to obtain infrared multimodal features, and the second feature extraction unit is used to extract features from the visible light data to obtain visible light multimodal features.

[0011] In conjunction with the first aspect, in one embodiment of this application, the decoupled multimodal learning module includes a common subspace encoder, a specific subspace encoder, a similarity loss function, a difference loss function, and a semantic loss function. The common subspace encoder includes interconnected convolutional layers and a SILU activation function, and the specific subspace encoder includes interconnected convolutional layers and a bottleneck layer.

[0012] In conjunction with the first aspect, in one embodiment of this application, the spatial offset module includes a spatial attention unit, a channel difference enhancement unit, and a convolutional layer. The spatial attention unit is used to capture key region features of the visible light mode-invariant features and the infrared mode-invariant features. The channel difference enhancement unit is used to calculate spatial channel enhancement features based on the key region features. The convolutional layer is used to nonlinearly fuse the spatial channel enhancement features to obtain a spatial offset.

[0013] In conjunction with the first aspect, in one embodiment of this application, the attention enhancement module includes a dilated normalized activation unit and a sigmoid activation function connected in sequence, wherein the dilated normalized activation unit includes a dilated convolutional layer, a normalized layer and a ReLU activation function connected in sequence.

[0014] In conjunction with the first aspect, in one embodiment of this application, the detection head module is used to detect the feature map to obtain a heat detection result, including: analyzing the feature map using the global self-attention mechanism of the detection head module to capture global context information of abnormal regions; generating abnormal candidate regions of different sizes based on the global context information of the abnormal regions using a multi-scale feature pyramid strategy; filtering the abnormal candidate regions according to a preset temperature threshold and texture measurement method; applying non-maximum suppression processing to the filtered abnormal candidate regions to obtain target abnormal candidate regions; and mapping the target abnormal candidate regions from image coordinates to geographic coordinates to obtain the heat detection result.

[0015] In conjunction with the first aspect, in one embodiment of this application, the step of constructing the detection range of the transformer and planning the flight path of the UAV swarm based on the detection range includes: determining the detection area of ​​the transformer based on the preset starting point and target point of the UAV swarm; arranging a target attraction potential field and an obstacle repulsion potential field at the target point, superimposing them to generate a total potential field for UAV swarm path planning; and using the discrete particle swarm optimization algorithm, with the total potential field as a constraint, combined with the artificial potential field and the arc weights of the Thiessen polygon graph, calculating the flight path of the UAV swarm in the detection area.

[0016] In conjunction with the first aspect, in one embodiment of this application, after obtaining the heat detection result, the method further includes: determining an abnormally overheating transformer based on the heat detection result; analyzing the heat detection result and historical detection data to determine the abnormality level of the abnormally overheating transformer; and outputting early warning information and maintenance suggestions corresponding to the abnormality level.

[0017] Secondly, embodiments of this application provide a transformer fault detection system based on UAV inspection, applied to the aforementioned transformer fault detection method based on UAV inspection. The system includes: a path planning module for constructing the transformer's detection range and planning the flight path of a UAV swarm based on the detection range; a multimodal perception module for acquiring infrared and visible light data of the transformer collected by the UAV swarm as it flies along the flight path; and an anomaly detection module for calling a pre-trained heat detection model to perform detection based on the infrared and visible light data, obtaining a heat detection result. The thermal detection model includes a multi-scale feature extraction module, a decoupled multimodal learning module, a spatial offset module, an offset guidance module, a decoupled feature fusion module, an attention enhancement module, and a detection head module. The output of the multi-scale feature extraction module is connected to the input of the decoupled multimodal learning module. The output of the decoupled multimodal learning module is connected to both the spatial offset module and the offset guidance module. The output of the offset guidance module is connected to the input of the decoupled feature fusion module. The output of the decoupled feature fusion module is connected to the input of the attention enhancement module. The attention enhancement module is connected to the detection head module.

[0018] In conjunction with the second aspect, in one embodiment of this application, the multimodal sensing module includes an infrared thermal imager and an industrial camera. The infrared thermal imager is used to capture infrared data of the transformer, and the industrial camera is used to acquire visible light images of the transformer.

[0019] This application first constructs the transformer detection range and plans the flight path of the UAV swarm to achieve efficient and reasonable path planning, reducing the risk of blind spots and improving inspection efficiency. Then, it utilizes the UAV swarm to collect infrared and visible light multimodal data of the transformer, fully leveraging the data complementarity. Next, a pre-trained overheating detection model integrating modules such as multi-scale feature extraction, decoupled multimodal learning, spatial offset, offset guidance, decoupled feature fusion, attention, and detection head processes the data. Each module, through feature extraction, alignment fusion, spatial offset processing, offset compensation, feature fusion, and enhancement, overcomes the bottleneck of insufficient accuracy and robustness in small transformer detection using traditional techniques. Simultaneously, the model optimizes the data processing flow through a modular architecture, balancing detection accuracy and speed, providing timely and reliable data support for fault diagnosis. In summary, this application integrates technologies such as detection range construction, path planning, multimodal data acquisition, and fusion detection, systematically solving core problems in UAV power line inspection such as difficult target identification, low path planning efficiency, and poor data fusion effects, effectively improving the detection accuracy of abnormally overheating transformers in complex environments (such as nighttime inspections). Attached Figure Description

[0020] Figure 1 This is a flowchart of a transformer fault detection method based on UAV inspection provided in one embodiment of this application; Figure 2 This is provided in one embodiment of the present application. Figure 1 The detailed flowchart of step 110; Figure 3 This is a structural diagram of a heat detection model provided in one embodiment of this application; Figure 4 This is a structural diagram of a pyramid feature extractor provided in one embodiment of this application; Figure 5 This is a schematic diagram of the decoupled multimodal learning module provided in one embodiment of this application; Figure 6 This is a schematic diagram of the spatial offset module provided in one embodiment of this application; Figure 7 This is a schematic diagram of the offset guidance module provided in one embodiment of this application; Figure 8 This is a schematic diagram of the decoupling feature fusion module provided in one embodiment of this application; Figure 9 This is a schematic diagram of the attention enhancement module provided in one embodiment of this application; Figure 10 This is a flowchart of obtaining the fever detection result provided in one embodiment of this application; Figure 11 This is a flowchart of the overall process of transformer fault detection based on UAV inspection, provided as a specific example in this application. Figure 12 This is a structural diagram of a transformer fault detection system based on UAV inspection provided in one embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] It should be noted that although the flowchart shows a logical order, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. The terms "first," "second," etc., used in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the structures, proportions, sizes, etc., depicted in the drawings are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the implementation conditions of this application. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to size, without affecting the effects and purposes achieved by this application, should still fall within the scope of the technical content disclosed in this application. Similarly, the terms such as "upper," "lower," "left," "right," "middle," and "one" used in this specification are only for clarity of description and are not used to limit the scope of implementation of this application. Changes or adjustments in their relative relationships, without substantially altering the technical content, should also be considered within the scope of implementation of this application.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0024] With the development of unmanned aerial vehicle (UAV) technology, its application in power system inspection is becoming increasingly widespread. Equipped with various types of sensors, such as visible light, infrared thermal imaging, or lidar, UAVs can efficiently acquire high-definition images and 3D data of transmission lines, substations, and distribution network equipment, significantly improving the intelligence level of power inspection. However, in power system inspection tasks, transformers, due to their small size, complex background environment, and varied shapes and structures, face many technical bottlenecks in existing detection methods, making it difficult to achieve high-precision and high-efficiency fault identification.

[0025] In collaborative inspection using swarm drones, a single drone often struggles to complete complex power line inspection tasks independently. To ensure safety and efficiency, swarm drones have been widely adopted and promoted. However, flight path planning for swarm drones still faces technical challenges: while Particle Swarm Optimization (PSO) alleviates local optima problems through particle feature classification, its grouping process is time-consuming, leading to system response delays; ant colony optimization based on improved artificial potential fields can optimize path planning, but it has stringent requirements for the application environment and limitations in adaptability to business scenarios and task completion rates, thus affecting the efficiency and quality of transformer inspection.

[0026] In the field of target detection technology, traditional target detection methods based on manually designed features (such as Haar and HOG) suffer from insufficient accuracy and robustness in complex backgrounds, lighting changes, and target shape alterations. Although deep learning target detection algorithms (such as the R-CNN series and YOLO series) have made significant progress in UAV image detection, they still suffer from missed detections and false detections in the detection of small targets such as transformers due to the low pixel ratio and blurred details. Meanwhile, power line inspections have extremely high real-time requirements. Balancing detection accuracy and speed to meet the massive data processing needs of UAVs flying at high speeds has become a critical problem that urgently needs to be solved.

[0027] At the data fusion technology level, traditional feature-level and decision-level multimodal fusion methods cannot fully tap the complementary potential of multimodal data when dealing with the detection of small transformer targets in complex scenarios, resulting in limited detection performance.

[0028] In summary, existing UAV power inspection technologies still have significant shortcomings in terms of target identification accuracy, path planning efficiency, and multimodal data fusion effects in transformer fault detection. There is an urgent need to develop a new detection method to overcome these technical bottlenecks and improve the safety and reliability of power system inspection.

[0029] In view of this, this application provides a transformer fault detection method and system based on UAV inspection. This method eliminates blind spots and improves inspection efficiency by pre-constructing the transformer detection range and optimizing the flight path planning of the UAV swarm. Simultaneously, it utilizes the UAV swarm to collect infrared and visible light multimodal data, and relies on a heat detection model composed of modules such as multi-scale feature extraction and decoupled multimodal learning to deeply fuse data and enhance feature capture capabilities, solving the problem of insufficient accuracy and robustness of traditional technologies for detecting small transformers in complex environments. Furthermore, a modular architecture achieves a balance between detection accuracy and processing speed, providing efficient support for power equipment fault diagnosis. This solution systematically solves the core technical bottlenecks of path planning, target recognition, and data fusion in UAV power inspection for transformer fault detection, effectively improving the detection accuracy and reliability of abnormally heated transformers at night.

[0030] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0031] Reference Figure 1 , Figure 1 This is a flowchart of a transformer fault detection method based on UAV inspection, according to one embodiment of this application. The process may specifically include, but is not limited to, steps 110 to 130.

[0032] Step 110: Construct the detection range of the transformer and plan the flight path of the drone swarm based on the detection range; Step 120: Acquire infrared and visible light data of the transformer collected by the drone swarm as it flies along the flight path; Step 130: Call the pre-trained heat detection model to perform detection based on infrared and visible light data to obtain the heat detection results.

[0033] Steps 110 to 130 will be described in detail below.

[0034] In one feasible embodiment, the transformer detection range refers to the pre-defined spatial area within the power system where fault detection is required for transformer equipment. By clearly defining the target detection area of ​​the drone swarm, missed or invalid inspections due to ambiguous scope can be avoided. The drone swarm's flight path refers to the optimal set of flight trajectories planned for the drone swarm based on the transformer detection range and inspection task requirements, aiming to ensure efficient and comprehensive completion of the detection task. It should be noted that key aspects of the flight path include: complete coverage of the transformer detection range to ensure no blind spots; avoiding redundant flights to shorten inspection time and improve task execution efficiency; and avoiding obstacles (such as high-voltage lines, buildings), no-fly zones, etc. The paths of multiple drones need to coordinate, such as parallel detection in different areas or sequential relay detection, to avoid collisions or duplicate operations. For example, if the transformer detection range is a cylindrical area with a radius of 10 meters, the drone swarm's flight path can be set as a spiral ascent trajectory around the cylinder or a layered circling trajectory, ensuring that each drone collects data from the transformer at different heights and angles.

[0035] In one feasible embodiment, such as Figure 2 As shown, the execution process of step 110, which constructs the detection range of the transformer and plans the flight path of the drone swarm based on the detection range, may include, but is not limited to, steps 210 to 230.

[0036] Step 210: Determine the detection area of ​​the transformer based on the preset starting point and target point of the drone swarm; Step 220: Arrange the target attraction potential field and the obstacle repulsion potential field at the target point, and superimpose them to generate the total potential field for UAV swarm path planning; Step 230: Using the discrete particle swarm optimization algorithm, with the total potential field as a constraint, and combining the artificial potential field and the arc weights of the Thiessen polygon graph, calculate the flight path of the UAV swarm in the detection area.

[0037] In one feasible embodiment, step 210 aims to define the spatial range of transformers that the drone swarm needs to inspect, providing geographical boundaries for subsequent path planning. The starting point refers to the takeoff point or current location of the drone swarm, and the target point refers to the location of the abnormally overheating transformer to be inspected (which can be determined based on methods such as preliminary fault warning).

[0038] In one feasible embodiment, when determining the detection area of ​​a transformer, a spatial region (such as a cylindrical or rectangular area with radius R) can be defined centered on the target point, taking into account the transformer size, safety distance, and inspection requirements. If there are multiple transformers to be inspected, the detection area may be expanded to a collection of multiple target points (e.g., by dividing the detection range of adjacent transformers using a Voronoi diagram to ensure that weakly covered areas are covered).

[0039] In one feasible embodiment, step 220 aims to simulate the drone's motion tendency under the attraction of a target and the repulsion of obstacles using a physical field model, guiding it to fly along a reasonable path. It is understood that the principle of the artificial potential field method is to analogize the drone's motion in space to the force-driven motion of a charged particle in an electric field. By setting a target attraction potential field at the target point (generating an attractive force to guide the drone towards the target), and simultaneously setting an obstacle repulsion potential field at the obstacle location (generating a repulsive force to allow the drone to avoid the obstacle), these two potential fields are superimposed to form a total potential field. The drone experiences a virtual resultant force within this total potential field, moving along the potential field gradient direction (i.e., the direction of the resultant force), thereby achieving path planning from the starting point to the target point. This method, by simulating the interaction of physical fields, allows the drone to approach the target while avoiding obstacles, and is suitable for path planning in dynamic environments. For example, in a dark environment, the drone cannot rely on visual recognition of obstacles; the potential field model can pre-plan obstacle avoidance paths using preset obstacle data (such as a GIS map). For adjacent transformers, the repulsive potential field can prevent the drone from getting too close to other equipment during detection, ensuring safety.

[0040] In a feasible embodiment, step 230 aims to generate the optimal flight path covering the detection area under the constraint of the total potential field using an intelligent optimization algorithm, balancing efficiency and coverage integrity. The Discrete Particle Swarm Optimization (D-PSO) algorithm abstracts the UAV swarm into particles, each representing a potential flight path, and iteratively searches for the optimal path (particle position). Particles must move within the total potential field, and the path must satisfy the attraction and repulsion rules of the potential field (e.g., avoiding repulsion peak areas). The Voronoi diagram (Thyson polygon) divides the detection area of ​​multiple transformers into multiple Voronoi units, each unit corresponding to the responsibility detection area of ​​one transformer, avoiding duplicate detection by UAVs within the swarm. Arc weights refer to the edges (arcs) of the Voronoi diagram representing the boundaries of adjacent detection areas; the weights can represent the cost of the path (e.g., flight distance, obstacle density). By optimizing the arc weights, the algorithm ensures that after completing the detection of the current transformer, the UAV flies to the adjacent transformer with the minimum cost. The particle swarm continuously adjusts its path during iterations to minimize the total flight distance and the number of obstacle avoidances, while simultaneously covering all Voronoi units (i.e., eliminating blind spots in the detection area). The final output path is a discrete sequence of waypoints, which the UAV follows to complete the inspection task from the starting point to the target point and adjacent transformers.

[0041] It is understandable that visible light data can be represented by visible light images, which can provide rich texture and shape information, while infrared data can be represented by infrared thermal imaging images, which can reflect the thermal characteristics of the target. The fusion of the two modalities can complement each other under different environmental conditions and improve the performance of target detection.

[0042] In one feasible embodiment, to acquire high-quality infrared and visible light data of the transformer, a high-resolution infrared thermal imager (such as the LIR T640) and a high-definition industrial camera (such as the Sony ILX-550) can be mounted on a drone to collect infrared radiation signals and visible light images of the transformer surface, respectively. The infrared thermal imager can detect the temperature distribution on the transformer surface, accurately capturing temperature differences caused by internal faults, uneven loads, or other anomalies; the industrial camera, through high-resolution visible light images, identifies physical damage to the transformer's outer surface (such as cracks, oil stains, corrosion, etc.). To ensure synchronous acquisition of multimodal data, a hardware-triggered synchronization mechanism is employed, simultaneously activating the exposure processes of both the infrared thermal imager and the industrial camera via an external trigger signal, ensuring temporal consistency between the two modal images.

[0043] In one feasible embodiment, the drone can be equipped with a LiDAR or a depth camera (such as an RGB-D camera or a ToF camera) to collect environmental obstruction information around power equipment (such as power transmission lines, substations, etc.) in real time through high-precision three-dimensional environmental perception capabilities, thereby dynamically adjusting the flight path to avoid collision risks.

[0044] See Figure 3 , Figure 3 This is a structural diagram of a heat detection model provided in one embodiment of this application. The model includes interconnected multi-scale feature extraction modules, decoupled multimodal learning modules, spatial offset modules, offset guidance modules, decoupled feature fusion modules, attention enhancement modules, and detection head modules. The functions of these modules are described below: The multi-scale feature extraction module can extract features from multimodal data such as infrared thermal imaging images and visible light images. Infrared data is input in the form of infrared thermal imaging images, and visible light data is input in the form of visible light images. The final output is visible light multimodal features containing multimodal information. The decoupled multimodal learning module generates visible light modality-invariant (VMI) features and infrared modality-invariant (IRM) features by aligning and fusing visible light and infrared features. The spatial offset module performs spatial offset processing on the two types of MMI features and outputs the spatial offset amount. The offset guidance module combines the VMI features, IRM features, and spatial offset amount, and obtains VMI-specific features and IR-specific features through offset compensation and adaptive alignment. The decoupled feature fusion module fuses the IRM features, VMI-specific features, and IRM-specific features to form fused features. The attention enhancement module performs feature enhancement on the fused features and generates a feature map. The detection head module completes detection based on the feature map and outputs the heat detection result.

[0045] In one feasible embodiment, such as Figure 3 As shown, the multi-scale feature extraction module includes a first feature extraction unit and a second feature extraction unit. The first feature extraction unit extracts features from infrared data to obtain infrared multimodal features, and the second feature extraction unit extracts features from visible light data to obtain visible light multimodal features.

[0046] In a feasible embodiment, the first feature extraction unit and the second feature extraction unit may employ independent basic pyramid feature extractors. The visible light image input to the heat detection model is then used. and infrared images Each input is fed into the corresponding independent base pyramid feature extractor. In order to extract multimodal features The first feature extraction unit extracts infrared multimodal features, and the second feature extraction unit extracts visible light multimodal features. The pyramid feature extractor is as follows: Figure 4 As shown, a multi-layer convolutional structure is used, and the output features are bounded random samples. .

[0047] In one feasible embodiment, such as Figure 5 As shown, the decoupled multimodal learning module constructs a common subspace encoder (C) consisting of convolutional layers and SILU activation functions, and a specific subspace encoder (S) containing convolutional layers and a C3 module (containing three convolutional layers and a bottleneck layer). It combines a multi-task loss function system, including similarity loss (minimizing modality-invariant feature distance to learn the common subspace), difference loss (maximizing modality-specific feature differences to capture complementary information), and semantic loss (guiding feature alignment to true semantics), to achieve efficient alignment and fusion of visible light and infrared image features. Among the similarity loss, difference loss, and semantic loss, similarity loss can be used to minimize visible light modality-invariant features. Infrared mode invariance characteristics The distance between them is used to learn the common subspace C; secondly, the difference loss is used to highlight... and The differences between them have been supplemented with additional information.

[0048] The following combination Figure 5 The working principle of the decoupled multimodal learning module is explained.

[0049] First, the multimodal features are input into a modality-invariant feature encoder with shared parameters. (Composed of convolutional layers and SILU activation function), generating mode-invariant features in the common subspace, as shown in Equation (1):

[0050] in, These are encoder parameters shared across modalities.

[0051] Multimodal features Modality-specific feature encoder with input to independent parameters (Composed of convolutional layers, C3 modules, and bottleneck layers), generating specific subspaces. Modal-specific features in As shown in formula (2):

[0052] in, These are encoder parameters that are independent for each mode.

[0053] Secondly, the central moment difference (CMD) is used to measure the visible light mode invariance characteristics. Infrared mode invariant features The distribution consistency is then used to calculate the similarity loss as shown in formula (3). .

[0054]

[0055] in For the expected value based on experience, It is the k-th order central moment.

[0056] By utilizing matrix trace constraints to mine the complementarity between modality-specific features and ensuring the independence between features, the difference loss is calculated as shown in Equation (4). .

[0057]

[0058] in It is the trace of the matrix.

[0059] By integrating the classification loss from the object detection task Regression loss and target confidence loss The common features are forced to have discriminative ability, and then the semantic loss is calculated as shown in formula (5). .

[0060]

[0061] Finally, the above losses are weighted and combined to arrive at the total loss. The calculation method is shown in formula (6).

[0062]

[0063] in It's about weighing the weights, specifically... For similarity loss The weight, For difference loss The weight, For semantic loss The weights are adjusted. Network parameters are updated via backpropagation to optimize feature decoupling. The optimized visible light invariant features are then... Compared with original infrared features Perform spatial alignment.

[0064] It is understandable that when dealing with visible light and infrared mode-invariant features... and When calculating the difference, traditional direct subtraction or feature connection methods are easily affected by complex background noise in UAV images.

[0065] Therefore, this embodiment employs a feature stitching strategy to generate variation features and uses spatial offset modeling to suppress background interference and extract effective offset information. For example... Figure 6 As shown, the spatial offset module comprises three processing units: a spatial attention unit, which automatically captures key regions (such as the transformer target region) in the modal invariant features of visible light and infrared light through a spatial attention mechanism, suppressing the feature response of background noise; a channel difference enhancement unit, which calculates cross-modal spatial channel difference features based on key region features, and highlights the intermodal variation information of the target region through channel-dimensional weighted enhancement; and a convolutional layer, which uses convolution operations to perform nonlinear transformation and fusion of the spatial channel enhancement features to generate the final spatial offset. This embodiment's spatial offset module, through its processing flow of focusing on the target via an attention mechanism, highlighting changes via channel difference enhancement, and suppressing noise via convolutional fusion, effectively solves the noise interference problem in modal feature difference calculation.

[0066] The following combination Figure 6 The working principle of the spatial offset module is explained.

[0067] First of all A spatial attention unit is input to capture representations of the target and key background regions in each modality. Within this framework, the spatially enhanced unimodal features can be calculated using formula (7). Specifically, the visible light modality-invariant features are... Infrared mode invariance characteristics Each spatial attention unit is input, and its corresponding spatially enhanced unimodal feature is calculated.

[0068]

[0069] in, For the sigmoid function, express Convolutional layer For connection operations, This represents the Adama product operation.

[0070] Further channel noise is suppressed by utilizing the channel difference enhancement unit and by parallel average pooling. With max pooling Generate multi-type spatial context descriptors; then input the descriptors into a shared multilayer perceptron network. To gain channel attention, By fusing with the channel attention map, spatial channel enhancement features are obtained. The specific calculation is shown in formula (8).

[0071]

[0072] Finally, by and By fusing with a nonlinear layer, spatial offset can be obtained. .

[0073] See Figure 7 , Figure 7 This is a schematic diagram of the offset guidance module provided in one embodiment of this application. The module employs deformable convolution technology and, based on the offset generated by the cross-modal space offset modeling module, achieves implicit offset compensation and adaptive feature alignment in the detection task, thereby more effectively correcting the feature space distribution and ensuring the stability and reliability of model training.

[0074] The following combination Figure 7 The working principle of the offset guidance module is explained.

[0075] Considering that each offset region has a different effect on the detection task, a modulation scalar learned from ϕc is added to dynamically aggregate information around the corresponding position p. Regarding the calculation of the sampled value, the corresponding value in the alignment feature is calculated based on the center sampled value, the basic offset, the fixed offset, and the implicit compensation, and the final sampled value is obtained by bilinear interpolation. The specific calculation is shown in formula (9):

[0076] Here, K represents the number of kernel weights, represents the k-th fixed offset, represents the k-th kernel weight, and represents the implicit compensation at the k-th position. Since the position might be a decimal, bilinear interpolation is used to obtain the final sampled value. Bilinear interpolation estimates the value of a non-integer coordinate position using the values ​​of the four nearest integer coordinate points. Specifically, assuming we need to estimate the value of position (x, y), where x and y are decimals, bilinear interpolation finds four surrounding integer coordinate points (x1, y1), (x1, y2), (x2, y1), and (x2, y2), and then performs a weighted average based on the values ​​of these four points.

[0077] See Figure 8 , Figure 8 This is a schematic diagram of the decoupled feature fusion module provided in one embodiment of this application. The working principle of this module includes: first, decoupling the visible light features and infrared features into mode-invariant features respectively. and modality-specific features In the process of traditional integration, direct connection and This could lead to redundancy, so it needs to be combined and optimized before fusion. and This is done to eliminate redundant information and enhance discriminative representation. Aligned visible light mode-invariant features are then incorporated. Modal invariance characteristics with infrared Combine them, and through The convolutional layers are optimized. The optimized mode-invariant features are then combined with mode-specific features from the visible and infrared light domains. and Connect the links to generate the final fused features. As shown in formula (10):

[0078] See Figure 9 , Figure 9 This is a schematic diagram of the attention enhancement module provided in one embodiment of this application. The module consists of a dilated normalized activation unit and a sigmoid activation function connected in sequence. The dilated normalized activation unit includes a dilated convolutional layer, a batch normalization layer, and a ReLU activation function connected in sequence. The dilated convolution expands the receptive field without increasing the number of parameters, and obtains multi-scale contextual information; the batch normalization layer normalizes the input data, suppresses the internal covariate shift, accelerates model convergence, and improves stability; the ReLU activation function introduces nonlinearity into the model, enhancing its ability to learn complex nonlinear mapping relationships. This embodiment adopts an improved spatial attention mechanism, which uses a learnable weight matrix to weight the feature map element by element, generating a spatial attention mask to dynamically enhance features, enabling the network to adaptively adjust its attention to different spatial locations, as shown in formula (11).

[0079]

[0080] In the feature reconstruction stage, three sets of cascaded decoders are used to restore the feature map size. The first-level decoder processes the input features through bilinear upsampling to ensure the consistency of feature map resolution in subsequent decoding processes, as shown in formula (12).

[0081]

[0082] in Indicates by the first Decoding feature maps generated by the decoder.

[0083] In a feasible embodiment, the detection head module adopts a Transformer-based detection structure, utilizing a global self-attention mechanism to directly predict the bounding boxes and confidence scores of anomaly candidate regions. Through a multi-scale feature pyramid strategy, the system can generate candidate regions of different sizes, and combined with a preset temperature threshold and texture anomaly measurement method, achieve adaptive anomaly candidate region detection. The loss function of the detection head is defined as shown in formula (13):

[0084] In formula (13), S is the heatmap of the abnormal region output by the model, R is the true residual map (generated by calculating the difference between the infrared image and the visible light image), and H and W are the height and width of the feature map, respectively. The hyperparameters are optimized through grid search. Subsequently, non-maximum suppression (NMS) is used to filter the detected candidate regions to eliminate duplicate detections, and with the help of the UAV's GPS and IMU data, the abnormal regions located in the image are accurately mapped to geographic coordinates to achieve fine positioning.

[0085] In one feasible embodiment, such as Figure 10 As shown, the execution process of using the detection head module to detect the feature map and obtain the heat detection result may include, but is not limited to, steps 1010 to 1040.

[0086] Step 1010: Analyze the feature map using the global self-attention mechanism of the detection head module to capture global contextual information of the abnormal region; Step 1020: Generate anomaly candidate regions of different sizes based on the global context information of the anomaly region using a multi-scale feature pyramid strategy; Step 1030: Based on the preset temperature threshold and texture measurement method, the abnormal candidate regions are screened, and the screened abnormal candidate regions are subjected to non-maximum suppression processing to obtain the target abnormal candidate regions. Step 1040: Map the target anomaly candidate region from image coordinates to geographic coordinates to obtain the heat detection result.

[0087] In a feasible embodiment, in step 1010, the detection head module adopts a Transformer-based structure. Its global self-attention mechanism allows the model to analyze feature maps without being limited to local information, but to simultaneously consider information from various locations on the feature map. By calculating the correlation between different locations in the feature map, the contextual information of the abnormal region within the entire image is captured. For example, when detecting abnormal transformer heating, it can not only focus on the features of the suspected heating point itself, but also combine it with relevant features such as the surrounding equipment structure and environment, thereby gaining a more comprehensive and accurate understanding of the abnormal region.

[0088] In a feasible embodiment, in step 1020, the multi-scale feature pyramid strategy operates based on the global context information of the anomalous region captured in step 210. Since anomalous regions in real-world scenarios may have different sizes, the multi-scale feature pyramid strategy constructs pyramid structures of different scales from the feature map. Anomalous candidate regions are generated at different scales, thus adapting to anomalous regions of varying sizes. For example, corresponding candidate regions can be generated for both large overheated areas and smaller local hotspots, ensuring that anomalous situations of different sizes are not overlooked.

[0089] In a feasible embodiment, in step 1030, the generated abnormal candidate regions are first screened based on a preset temperature threshold and a texture measurement method. The temperature threshold is used to determine whether the temperature of the candidate region exceeds the normal range, and the texture measurement method analyzes whether the texture features of the candidate region are abnormal. Regions that are truly likely to be abnormal are screened using these two methods. Then, non-maximum suppression (NMS) is applied. Because multiple regions may overlap and all represent the same abnormality when generating candidate regions, NMS compares the confidence scores and other indicators of the candidate regions, retaining regions with high confidence scores and suppressing duplicate regions, thereby obtaining the target abnormal candidate regions, making the detection results more accurate and removing redundant detections.

[0090] In one feasible embodiment, in step 1040, the UAV is equipped with GPS (Global Positioning System) and IMU (Inertial Measurement Unit). Using data from these devices, the target anomaly candidate region obtained in step 230 under image coordinates is accurately mapped to actual geographic coordinates. This allows the specific location of the anomaly region in the real geographic environment to be determined, providing accurate heat detection results.

[0091] In one feasible embodiment, after obtaining the overheating detection results, the specific transformer with abnormal overheating can first be located based on the geographical coordinates and equipment ledger data in the detection results. Then, the current detection results are integrated with historical detection data, and the severity of the anomaly is graded through analysis (e.g., minor, severe, critical, etc.). Finally, based on the grading results, corresponding warning information (e.g., blue, yellow, red warnings) and maintenance suggestions (e.g., continuous monitoring, time-limited verification, immediate repair, etc.) are output. Specifically, this process identifies faulty equipment by correlating detection results with equipment data, scientifically grades the degree of anomaly by combining historical data, and generates targeted warning and handling plans based on this.

[0092] See Figure 11 , Figure 11This application provides a specific example of a flowchart illustrating the overall process of transformer fault detection based on drone inspection. After determining the drone's flight path, the drone performs point-to-point flight, collecting environmental obstruction information around the power equipment using LiDAR or depth cameras. Simultaneously, visible light cameras (such as ordinary industrial cameras) and infrared thermal imagers acquire visible light and infrared images. Then, a pre-trained heat detection model is used to extract multimodal features, perform cross-modal alignment and fusion, and detect anomalies in the images, obtaining heat detection results. Subsequently, the system identifies abnormally overheated transformers based on the geographical coordinates and equipment ledger data in the detection results. It then integrates the current detection results with historical detection data to classify the severity of the anomalies. Finally, based on the classification results, it outputs corresponding warning information and maintenance suggestions, allowing personnel to determine whether there are abnormally overheated transformers in the area where the drone is operating and to perform corresponding maintenance operations. Furthermore, maintenance feedback can be used to update the fault case library and optimize the detection model, forming a closed-loop improvement process.

[0093] Reference Figure 12 , Figure 12 This is a structural diagram of a transformer fault detection system based on UAV inspection, provided in one embodiment of this application. This fault detection system can be applied to a previously mentioned transformer fault detection method based on UAV inspection. The system includes: a path planning module 1210, used to construct the transformer detection range and plan the flight path of the UAV swarm based on the detection range; a multimodal perception module 1220, used to acquire infrared and visible light data of the transformer collected by the UAV swarm as it flies along the flight path; and an anomaly detection module 1230, used to call a pre-trained heat detection model to perform detection based on the infrared and visible light data, and obtain the heat detection result. The overheating detection model includes a multi-scale feature extraction module, a decoupled multimodal learning module, a spatial offset module, an offset guidance module, a decoupled feature fusion module, an attention enhancement module, and a detection head module. The output of the multi-scale feature extraction module is connected to the input of the decoupled multimodal learning module. The output of the decoupled multimodal learning module is connected to both the spatial offset module and the offset guidance module. The output of the offset guidance module is connected to the input of the decoupled feature fusion module. The output of the decoupled feature fusion module is connected to the input of the attention enhancement module. The attention enhancement module is connected to the detection head module.

[0094] In one feasible embodiment, the multi-scale feature extraction module can extract features from multimodal data such as infrared thermal imaging images and visible light images. Infrared and visible light data can be input as corresponding images, and the final output is visible light multimodal features containing multimodal information. The decoupled multimodal learning module generates visible light modality-invariant features and infrared modality-invariant features by aligning and fusing visible light and infrared features. The spatial offset module performs spatial offset processing on the two types of modality-invariant features and outputs the spatial offset amount. The offset guidance module combines the visible light modality-invariant features, infrared modality-invariant features, and spatial offset amount, and obtains visible light modality-specific features and infrared modality-specific features through offset compensation and adaptive alignment. The decoupled feature fusion module fuses the infrared modality-invariant features, visible light modality-specific features, and infrared modality-specific features into a fused feature. The attention enhancement module performs feature enhancement on the fused feature to generate a feature map. The detection head module completes detection based on the feature map and outputs the heat detection result.

[0095] In one feasible embodiment, the multimodal sensing module includes an infrared thermal imager and an industrial camera. The infrared thermal imager is used to capture infrared radiation data from the transformer, while the industrial camera is responsible for acquiring visible light images of the transformer.

[0096] In one feasible embodiment, the multimodal perception module also includes a LiDAR (Light Detection and Ranging) or a depth camera (such as an RGB-D camera or a ToF camera). The LiDAR or depth camera, with its high-precision 3D environmental perception capabilities, can collect real-time information on the location, distance, and shape of obstacles around power equipment such as power transmission lines and substations, thereby enabling the UAV to dynamically adjust its flight path and avoid collision risks.

[0097] It should be noted that this transformer fault detection system is fully compatible with the aforementioned transformer fault detection method. The functional logic and principles of each module in the system correspond one-to-one with the technical solution of this method. Specific functions of the modules can be found in the detailed descriptions above and will not be repeated here.

[0098] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A transformer fault detection method based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: Construct the detection range of the transformer, and plan the flight path of the UAV swarm based on the detection range; The infrared and visible light data of the transformer are acquired as the UAV cluster flies along the flight path. The pre-trained heat detection model is invoked to perform detection based on the infrared data and the visible light data, and the heat detection result is obtained. The heat detection model includes a multi-scale feature extraction module, a decoupled multimodal learning module, a spatial offset module, an offset guidance module, a decoupled feature fusion module, an attention enhancement module, and a detection head module. Specifically, the multi-scale feature extraction module extracts features from the infrared and visible light data to obtain visible light multimodal features; the decoupled multimodal learning module aligns and fuses the visible light and infrared features to obtain visible light modality-invariant features and infrared modality-invariant features; the spatial offset module performs spatial offset processing on the visible light and infrared modality-invariant features to obtain a spatial offset amount; the offset guidance module performs offset compensation and adaptive alignment processing based on the visible light modality-invariant features, infrared modality-invariant features, and the spatial offset amount to obtain visible light modality-specific features and infrared modality-specific features; and the decoupled feature fusion module performs feature fusion processing on the infrared modality-invariant features, the visible light modality-specific features, the visible light modality-invariant features, and the infrared modality-specific features to obtain fused features. The attention enhancement module is used to perform feature enhancement processing on the fused features to obtain a feature map; the detection head module is used to detect the feature map to obtain the heat detection result.

2. The transformer fault detection method based on UAV inspection according to claim 1, characterized in that, The multi-scale feature extraction module includes a first feature extraction unit and a second feature extraction unit; the first feature extraction unit is used to extract features from the infrared data to obtain infrared multimodal features, and the second feature extraction unit is used to extract features from the visible light data to obtain visible light multimodal features.

3. The transformer fault detection method based on UAV inspection according to claim 1, characterized in that, The decoupled multimodal learning module includes a common subspace encoder, a specific subspace encoder, a similarity loss function, a difference loss function, and a semantic loss function. The common subspace encoder includes interconnected convolutional layers and a SILU activation function, and the specific subspace encoder includes interconnected convolutional layers and a bottleneck layer.

4. The transformer fault detection method based on UAV inspection according to claim 2, characterized in that, The spatial offset module includes a spatial attention unit, a channel difference enhancement unit, and a convolutional layer. The spatial attention unit is used to capture key region features of the visible light mode-invariant features and the infrared mode-invariant features. The channel difference enhancement unit is used to calculate spatial channel enhancement features based on the key region features. The convolutional layer is used to nonlinearly fuse the spatial channel enhancement features to obtain the spatial offset.

5. A transformer fault detection method based on UAV inspection according to claim 2, characterized in that, The attention enhancement module includes a dilated normalized activation unit and a sigmoid activation function connected in sequence. The dilated normalized activation unit includes a dilated convolutional layer, a normalized layer, and a ReLU activation function connected in sequence.

6. The transformer fault detection method based on UAV inspection according to claim 2, characterized in that, The feature map is detected using the detection head module to obtain a heat detection result, including: The global self-attention mechanism of the detection head module is used to analyze the feature map and capture global contextual information of abnormal regions; By employing a multi-scale feature pyramid strategy, anomaly candidate regions of different sizes are generated based on the global context information of the anomaly region. Based on a preset temperature threshold and texture measurement method, the abnormal candidate regions are screened, and non-maximum suppression processing is applied to the screened abnormal candidate regions to obtain the target abnormal candidate regions. The target anomaly candidate region is mapped from image coordinates to geographic coordinates to obtain the heat detection result.

7. The transformer fault detection method based on UAV inspection according to claim 1, characterized in that, The process of constructing the transformer's detection range and planning the flight path of the drone swarm based on the detection range includes: The detection area of ​​the transformer is determined based on the preset starting point and target point of the drone cluster; At the target point, an attractive potential field and an obstacle repulsive potential field are arranged and superimposed to generate the total potential field for UAV swarm path planning. Using the discrete particle swarm optimization algorithm, with the total potential field as a constraint, and combining the artificial potential field and the arc weights of the Thiessen polygon graph, the flight path of the UAV cluster in the detection area is calculated.

8. A transformer fault detection method based on UAV inspection according to claim 1, characterized in that, After obtaining the fever detection result, the method further includes: Based on the heat detection results, an abnormally overheating transformer was identified; Based on the analysis of the heat detection results and historical detection data, the abnormality level of the abnormal heat transformer is determined; Output early warning information and maintenance suggestions corresponding to the aforementioned anomaly level.

9. A transformer fault detection system based on unmanned aerial vehicle (UAV) inspection, characterized in that, The system, applied to a transformer fault detection method based on unmanned aerial vehicle (UAV) inspection as described in any one of claims 1-8, comprises: The path planning module is used to construct the detection range of the transformer and plan the flight path of the UAV swarm based on the detection range. The multimodal perception module acquires infrared and visible light data of the transformer collected by the UAV cluster as it flies along the flight path; An anomaly detection module is used to call a pre-trained heat detection model to perform detection based on the infrared data and the visible light data, and obtain heat detection results. The heat detection model includes a multi-scale feature extraction module, a decoupled multimodal learning module, a spatial offset module, an offset guidance module, a decoupled feature fusion module, an attention enhancement module, and a detection head module. The output of the multi-scale feature extraction module is connected to the input of the decoupled multimodal learning module. The output of the decoupled multimodal learning module is connected to the spatial offset module and the offset guidance module, respectively. The output of the offset guidance module is connected to the input of the decoupled feature fusion module. The output of the decoupled feature fusion module is connected to the input of the attention enhancement module. The attention enhancement module is connected to the detection head module.

10. A transformer fault detection system based on UAV inspection according to claim 9, characterized in that, The multimodal sensing module includes an infrared thermal imager and an industrial camera. The infrared thermal imager is used to capture infrared data of the transformer, and the industrial camera is used to acquire visible light images of the transformer.

Images