A transformer fault detection method and system based on unmanned aerial vehicle inspection

Abstract

The application discloses a transformer fault detection method and system based on unmanned aerial vehicle inspection, first, the detection range of the transformer is constructed, and the flight path of the unmanned aerial vehicle cluster is planned according to the detection range; then, the infrared data and visible light data of the transformer collected when the unmanned aerial vehicle cluster flies along the flight path are acquired; then, a heat detection model integrated with modules such as multi-scale feature extraction, decoupled multi-modal learning, spatial offset, offset guide, decoupled feature fusion, attention and detection head is called, detection is carried out according to the infrared data and visible light data, and a heat detection result is obtained. The embodiment of the application integrates detection range construction, path planning, multi-modal data acquisition and fusion detection and other technologies, systematically solves core problems such as difficult target identification, low path planning efficiency, poor data fusion effect and the like in unmanned aerial vehicle power inspection, and effectively improves the detection precision of abnormally heated transformers.

Description

TECHNICAL FIELD

[0001] The embodiment of the application relates to, but is not limited to, the technical field of image processing, and in particular to a transformer fault detection method and system based on unmanned aerial vehicle (UAV) inspection. BACKGROUND

[0002] With the wide application of UAV technology in power system inspection, it can efficiently obtain power equipment data by carrying multiple types of sensors. However, in the aspect of transformer detection, due to the small size of the target, complex background and variable shape, the existing detection technology cannot realize high-precision and high-efficiency fault recognition.

[0003] In the cooperative inspection of cluster UAVs, the flight path planning technology has obvious shortcomings. Although the particle swarm optimization (PSO) algorithm can alleviate the local optimal problem, the grouping time-consuming leads to slow system response; the improved artificial potential field ant colony algorithm has high requirements for the environment, and the business adaptability and task achievement rate are limited, which affects the inspection efficiency.

[0004] In terms of target detection technology, the traditional artificial design feature method has insufficient detection accuracy and robustness in complex environments. Although deep learning algorithms such as R-CNN and YOLO series have made progress, there are still problems of missed detection and misjudgment for small targets such as transformers due to low pixel ratio and blurred details. At the same time, power inspection requires high real-time detection, and both accuracy and speed need to be considered to adapt to high-speed data processing requirements.

[0005] In the field of data fusion, traditional feature-level and decision-level fusion methods cannot fully tap the potential of multi-modal data, and the detection effect of small transformer targets in complex scenes is not good.

[0006] At present, there are still technical bottlenecks in the target recognition, path planning and data fusion of UAV power inspection in transformer fault detection, and innovative methods are needed to improve the reliability and efficiency of power system inspection. SUMMARY

[0007] The following is a summary of the subject matter described in detail in this document. This summary is not intended to limit the scope of protection of the claims.

[0008] The embodiment of the application provides a transformer fault detection method and system based on UAV inspection, which solves the problems of target recognition, low path planning efficiency and poor data fusion effect, thereby effectively improving the reliability and efficiency of night abnormal heating transformer detection.

[0009] In a first aspect, the embodiments of the present application provide a transformer fault detection method based on unmanned aerial vehicle inspection, comprising: constructing a detection range of a transformer, and planning a flight path of a cluster of unmanned aerial vehicles according to the detection range; acquiring infrared data and visible light data of the transformer collected by the cluster of unmanned aerial vehicles when flying along the flight path; calling a pre-trained heat detection model to detect according to the infrared data and the visible light data to obtain a heat detection result; the heat detection model comprises a multi-scale feature extraction module, a decoupling multi-modal learning module, a spatial offset module, an offset guiding module, a decoupling feature fusion module, an attention enhancement module and a detection head module, wherein the multi-scale feature extraction module is configured to extract features from the infrared data to obtain infrared multi-modal features, and extract features from the visible light data to obtain visible light multi-modal features; the decoupling multi-modal learning module is configured to align and fuse the visible light multi-modal features and the infrared multi-modal features with each other to obtain visible light modal invariant features and infrared modal invariant features; the spatial offset module is configured to perform spatial offset processing on the visible light modal invariant features and the infrared modal invariant features to obtain a spatial offset amount; the offset guiding module is configured to perform offset compensation and adaptive alignment processing according to the visible light modal invariant features, the infrared modal invariant features and the spatial offset amount to obtain visible light modal specific features and infrared light modal specific features; the decoupling feature fusion module is configured to perform feature fusion processing on the infrared modal invariant features, the visible light modal specific features, the visible light modal invariant features and the infrared light modal specific features to obtain fusion features; the attention enhancement module is configured to perform feature enhancement processing on the fusion features to obtain a feature map; and the detection head module is configured to detect the feature map to obtain the heat detection result.

[0010] In combination with the first aspect, in an embodiment of the present application, the multi-scale feature extraction module comprises a first feature extraction unit and a second feature extraction unit; the first feature extraction unit is configured to extract features from the infrared data to obtain infrared multi-modal features, and the second feature extraction unit is configured to extract features from the visible light data to obtain visible light multi-modal features.

[0011] In combination with the first aspect, in an embodiment of the present application, the decoupling multi-modal learning module comprises a common subspace encoder, a specific subspace encoder, a similarity loss function, a difference loss function and a semantic loss function; the common subspace encoder comprises convolution layers and SILU activation functions connected with each other; and the specific subspace encoder comprises convolution layers and bottleneck layers connected with each other.

[0012] With reference to the first aspect, in an embodiment of the present application, the spatial offset module comprises a spatial attention unit, a channel difference enhancement unit and a convolution layer, the spatial attention unit is configured to capture key region features of the visible light modality invariant features and the infrared modality invariant features, the channel difference enhancement unit is configured to calculate spatial channel enhancement features according to the key region features, and the convolution layer is configured to perform nonlinear fusion on the spatial channel enhancement features to obtain the spatial offset.

[0013] With reference to the first aspect, in an embodiment of the present application, the attention enhancement module comprises a hole normalization activation unit and a sigmoid activation function connected in sequence, and the hole normalization activation unit comprises a hole convolution layer, a normalization layer and a ReLU activation function connected in sequence.

[0014] With reference to the first aspect, in an embodiment of the present application, the feature map is detected by using the detection head module to obtain a heat detection result, comprising: analyzing the feature map by using a global self-attention mechanism of the detection head module to capture global context information of an abnormal region; generating abnormal candidate regions of different sizes based on the global context information of the abnormal region by using a multi-scale feature pyramid strategy; screening the abnormal candidate regions according to a preset temperature threshold and a texture measurement method, and performing non-maximum suppression processing on the screened abnormal candidate regions to obtain a target abnormal candidate region; and mapping the target abnormal candidate region from an image coordinate to a geographic coordinate to obtain the heat detection result.

[0015] With reference to the first aspect, in an embodiment of the present application, the detection range of the transformer is constructed, and a flight path of the UAV cluster is planned according to the detection range, comprising: determining a detection area of the transformer according to a preset departure point and target point of the UAV cluster; arranging a target attractive potential field and an obstacle repulsive potential field at the position of the target point to generate a total potential field for path planning of the UAV cluster; and using a discrete particle swarm algorithm, taking the total potential field as a constraint condition, combining an artificial potential field and a Voronoi diagram arc weight, and calculating a flight path of the UAV cluster in the detection area.

[0016] With reference to the first aspect, in an embodiment of the present application, after obtaining the heat detection result, the method further comprises: determining an abnormally heated transformer based on the heat detection result; analyzing the heat detection result and historical detection data to determine an abnormal level of the abnormally heated transformer; and outputting early warning information and maintenance suggestions corresponding to the abnormal level.

[0017] In a second aspect, the embodiments of the present application provide a transformer fault detection system based on unmanned aerial vehicle inspection, which is applied to the transformer fault detection method based on unmanned aerial vehicle inspection described above. The system comprises: a path planning module, configured to construct a detection range of a transformer and plan a flight path of a cluster of unmanned aerial vehicles according to the detection range; a multi-modal perception module, configured to acquire infrared data and visible light data of the transformer collected by the cluster of unmanned aerial vehicles when flying along the flight path; and an anomaly detection module, configured to call a pre-trained heat detection model to perform detection according to the infrared data and the visible light data to obtain a heat detection result. The heat detection model comprises a multi-scale feature extraction module, a decoupled multi-modal learning module, a spatial offset module, an offset guiding module, a decoupled feature fusion module, an attention enhancement module, and a detection head module. An output end of the multi-scale feature extraction module is connected with an input end of the decoupled multi-modal learning module. Output ends of the decoupled multi-modal learning module are respectively connected with the spatial offset module and the offset guiding module. An output end of the offset guiding module is connected with an input end of the decoupled feature fusion module. An output end of the decoupled feature fusion module is connected with an input end of the attention enhancement module. The attention enhancement module is connected with the detection head module.

[0018] In combination with the second aspect, in an embodiment of the present application, the multi-modal perception module comprises an infrared thermal imager and an industrial camera. The infrared thermal imager is configured to capture infrared data of the transformer. The industrial camera is configured to acquire visible light images of the transformer.

[0019] The embodiments of the present application first construct a transformer detection range and plan a flight path of a cluster of unmanned aerial vehicles, realize efficient and reasonable path planning, reduce the risk of inspection blind area and improve the inspection efficiency. Then, the cluster of unmanned aerial vehicles collects infrared and visible light multi-modal data of the transformer, and fully plays the complementary nature of the data. Further, a heat detection model pre-trained and integrated with modules such as multi-scale feature extraction, decoupled multi-modal learning, spatial offset, offset guiding, decoupled feature fusion, attention, and detection head is used to process the data. Through processes such as feature extraction, alignment fusion, spatial offset processing, offset compensation, feature fusion, and enhancement, the modules break through the bottleneck of insufficient detection accuracy and robustness of traditional technology for small transformers. At the same time, the model optimizes the data processing process through a modular architecture, balances the detection accuracy and speed, and provides timely and reliable data support for fault diagnosis. In summary, the embodiments of the present application integrate technologies such as detection range construction, path planning, multi-modal data acquisition, and fusion detection, and systematically solve core problems such as difficult target recognition, low path planning efficiency, and poor data fusion effect in unmanned aerial vehicle power inspection, effectively improving the detection accuracy of abnormally heated transformers in complex environments (such as night inspection). BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of a transformer fault detection method based on unmanned aerial vehicle inspection provided by an embodiment of the present application;

[0021] Figure 2 is a flowchart of step 110 in the method provided by an embodiment of the present application; Figure 1

[0022] Figure 3 is a structure diagram of a fever detection model provided by an embodiment of the present application;

[0023] Figure 4 is a structure diagram of a pyramid feature extractor provided by an embodiment of the present application;

[0024] Figure 5 is a principle structure diagram of a decoupled multi-modal learning module provided by an embodiment of the present application;

[0025] Figure 6 is a principle structure diagram of a spatial offset module provided by an embodiment of the present application;

[0026] Figure 7 is a principle structure diagram of an offset guiding module provided by an embodiment of the present application;

[0027] Figure 8 is a principle structure diagram of a decoupled feature fusion module provided by an embodiment of the present application;

[0028] Figure 9 is a principle structure diagram of an attention enhancement module provided by an embodiment of the present application;

[0029] Figure 10 is a flowchart of obtaining a fever detection result provided by an embodiment of the present application;

[0030] Figure 11 is a whole flowchart of a transformer fault detection method based on unmanned aerial vehicle inspection provided by an embodiment of the present application;

[0031] Figure 12 is a structure diagram of a transformer fault detection system based on unmanned aerial vehicle inspection provided by an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0033] ​It is to be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that in the flowchart. The terms "first", "second", and the like in the description and claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a particular order or sequence. It is to be understood that the structures, proportions, sizes, etc. shown in the drawings of the present specification are only used to illustrate the content disclosed in the specification, so that those skilled in the art can understand and read, and are not used to limit the conditions that can be implemented by the present application, so they do not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, without affecting the effect and purpose that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" used in the specification are only for the purpose of clear understanding of the description, and are not intended to limit the scope of the application that can be implemented, and the change or adjustment of the relative relationship, without substantially changing the technical content, is also considered as the scope of the application that can be implemented.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing the embodiments of the present application only and is not intended to limit the present application.

[0035] With the development of unmanned aerial vehicle (UAV) technology, its application in the field of power system inspection is increasingly widespread. By carrying multiple types of sensors such as visible light, infrared thermal imaging, or laser radar, UAVs can efficiently acquire high-definition images and three-dimensional data of power transmission lines, substations, and distribution network equipment, significantly improving the intelligent level of power inspection. However, in the power system inspection task, transformers have small target size, complex background environment, and variable form structure, which poses technical bottlenecks for existing detection methods, making it difficult to achieve high-precision and high-efficiency fault identification.

[0036] In the aspect of cluster UAV cooperative inspection, in order to deal with complex power inspection tasks, a single UAV often cannot complete the task independently. In order to ensure safety and efficiency, cluster UAVs have been widely applied and promoted, but there are still technical difficulties in the flight path planning of cluster UAVs: although the particle swarm optimization (PSO) algorithm can alleviate the local optimal problem through particle feature classification, the grouping process is time-consuming, which leads to system response lag; although the ant colony algorithm based on improved artificial potential field can optimize path planning, it has strict requirements for application environment, and has limitations in adaptability and task completion rate in business scenarios, which affects the efficiency and quality of transformer inspection.

[0037] In the field of target detection technology, the traditional target detection method based on manually designed features (such as Haar and HOG) has serious deficiencies in detection accuracy and robustness when the background is complex, the light changes, and the target shape changes. Although the deep learning target detection algorithm (such as the R-CNN series and the YOLO series) has made significant progress in unmanned aerial vehicle image detection, there are still problems of missed detection and false detection in the detection of small targets such as transformers, because the target pixel ratio is low and the detailed features are blurred. At the same time, the detection real-time requirement of power inspection is very high, and how to balance the detection accuracy and speed to adapt to the massive data processing demand under the high-speed flight of unmanned aerial vehicles has become a key problem to be solved.

[0038] In the aspect of data fusion technology, the traditional feature-level and decision-level multi-modal fusion method cannot fully tap the complementary potential of multi-modal data when dealing with small transformer target detection in complex scenarios, resulting in limited detection performance.

[0039] In summary, the existing unmanned aerial vehicle power inspection technology still has significant deficiencies in target recognition accuracy, path planning efficiency, and multi-modal data fusion effect in transformer fault detection, and a new detection method is urgently needed to break through the above technical bottlenecks and improve the safety and reliability of power system inspection.

[0040] Therefore, the embodiments of the present application provide a transformer fault detection method based on unmanned aerial vehicle inspection and a transformer fault detection system. The method eliminates the inspection blind area and improves the inspection efficiency by pre-constructing the transformer detection range and optimizing the flight path planning of the unmanned aerial vehicle cluster. At the same time, the infrared and visible light multi-modal data are collected by the unmanned aerial vehicle cluster, and the heat detection model composed of multi-scale feature extraction and decoupled multi-modal learning modules is relied on to deeply fuse the data and enhance the feature capture capability, solving the problem of insufficient detection accuracy and robustness of traditional technology in complex environments for small transformers. In addition, the balance between detection accuracy and processing speed is achieved through a modular architecture, providing efficient support for power equipment fault diagnosis. This scheme systematically solves the path planning, target recognition, and data fusion core technical bottlenecks of unmanned aerial vehicle power inspection in transformer fault detection, effectively improving the detection accuracy and reliability of abnormal heat transformers at night.

[0041] The embodiments of the present application will be further described below with reference to the accompanying drawings.

[0042] Reference Figure 1 , Figure 1 is a flowchart of a transformer fault detection method based on unmanned aerial vehicle inspection provided by an embodiment of the present application. The flowchart can specifically include but is not limited to steps 110 to 130.

[0043] Step 110: Construct the detection range of the transformer, and plan the flight path of the unmanned aerial vehicle cluster according to the detection range;

[0044] Step 120: Obtain the infrared data and visible light data of the transformer collected by the UAV cluster when flying along the flight path;

[0045] Step 130: Call the pre-trained heat detection model to detect according to the infrared data and visible light data to obtain the heat detection result.

[0046] The steps 110 to 130 will be described in detail below.

[0047] In a feasible embodiment, the detection range of the transformer refers to the spatial area range that needs to be detected for the transformer equipment in the power system. By specifying the target detection area of the UAV cluster, it can avoid missed detection or invalid inspection due to ambiguous range. The flight path of the UAV cluster refers to the optimal flight trajectory set planned for the UAV cluster according to the transformer detection range and the inspection task requirements, aiming to ensure efficient and comprehensive completion of the detection task. It should be noted that the key points of the flight path include: complete coverage of the transformer detection range to ensure no blind area of inspection; avoid redundant flight, shorten the inspection time, and improve the task execution efficiency; avoid obstacles (such as high-voltage lines, buildings), no-fly zones, etc. The paths of multiple UAVs need to be coordinated, such as parallel detection in different areas or relay detection in sequence to avoid collision or repeated work. For example: if the transformer detection range is a cylindrical area with a radius of 10 meters, the flight path of the UAV cluster can be set as a spiral ascending trajectory around the cylinder, or a layered surrounding trajectory to ensure that each UAV collects data on the transformer at different heights and angles.

[0048] In a feasible embodiment, as shown in Figure 2 Step 110 constructs the detection range of the transformer, and plans the execution process of the flight path of the UAV cluster according to the detection range, which can include but is not limited to steps 210 to 230.

[0049] Step 210: Determine the detection area of the transformer according to the preset departure point and target point of the UAV cluster;

[0050] Step 220: Arrange the target attractive potential field and obstacle repulsive potential field at the position of the target point, and superimpose to generate the total potential field of the UAV cluster path planning;

[0051] Step 230: Use the discrete particle swarm algorithm to calculate the flight path of the UAV cluster in the detection area with the total potential field as the constraint condition, combining the artificial potential field and the arc weight of the Thiessen polygon map.

[0052] In a feasible embodiment, step 210 aims to explicitly define the spatial range of the transformer to be inspected by the UAV cluster, providing a geographical boundary for subsequent path planning. The starting point refers to the takeoff point or current position of the UAV cluster, and the target point refers to the location of the abnormally heated transformer to be detected (which can be determined based on preliminary fault warning methods, etc.).

[0053] In a feasible embodiment, when determining the detection area of the transformer, a spatial area (such as a cylindrical or rectangular area with a radius R) can be delineated with the target point as the center, combined with the transformer size, safety distance, and inspection requirements. If there are multiple transformers to be detected, the detection area may expand to a collection area of multiple target points (such as through Voronoi diagram to divide the detection range of adjacent transformers, ensuring that the weak coverage area is covered).

[0054] In a feasible embodiment, step 220 aims to simulate the movement trend of the UAV being attracted by the target and repelled by the obstacles through the physical field model, guiding it to fly along a reasonable path. It can be understood that the principle of artificial potential field method is to analogize the movement of UAV in space as the force movement of charged particles in electric field, by setting target attraction potential field (generating attractive force to guide UAV to target) at target point and obstacle repulsion potential field (generating repulsive force to make UAV avoid obstacles) at obstacle position, superimposing the two potential fields to form total potential field; UAV is affected by virtual resultant force in total potential field, moves along the potential field gradient direction (i.e. the direction of resultant force), thereby realizing path planning from starting point to target point, this method simulates the interaction of physical field, makes UAV tend to target while avoiding obstacles, and is suitable for path planning in dynamic environment. For example, in dark environment, UAV cannot rely on visual identification of obstacles, potential field model can plan obstacle avoidance path in advance through pre-set obstacle data (such as GIS map). For adjacent transformers, repulsive potential field can avoid UAV from being too close to other equipment during detection, ensuring safety.

[0055] In a feasible embodiment, step 230 aims to generate an optimal flight path covering the detection area by intelligent optimization algorithm, taking into account the efficiency and coverage integrity under the constraint of the total potential field. The principle of the discrete particle swarm optimization algorithm (D-PSO) is to abstract the UAV cluster as a particle, each particle representing a potential flight path, and to search for the optimal path (particle position) through iteration. Among them, the particle must move within the total potential field range, and the path must meet the attraction and repulsion rules of the potential field (such as avoiding the repulsion peak area). The role of the Voronoi diagram is to divide the detection area of multiple transformers to be detected into multiple Voronoi units, each unit corresponding to the responsibility detection area of a transformer, avoiding repeated detection by UAVs within the cluster. The arc weight refers to the edge (arc) of the Voronoi diagram representing the boundary of adjacent detection areas, and the weight can represent the cost of the path (such as flight distance, obstacle density). The algorithm can ensure that the UAV flies to the adjacent transformer with the minimum cost after completing the current transformer detection by optimizing the arc weight. The particle swarm continuously adjusts the path in iteration to make the total flight distance shortest, the number of obstacle avoidance least, and cover all Voronoi units (i.e. no dead angle in the detection area). The final output path is a discrete sequence of waypoints, and the UAV flies according to the sequence to complete the inspection task from the starting point to the target point and the adjacent transformer.

[0056] It can be understood that the visible light data can be represented in the form of a visible light image, which can provide rich texture and shape information, while the infrared data can be represented in the form of an infrared thermal imaging image, which can reflect the thermal characteristics of the target. The fusion of the two modal data can complement each other's advantages under different environmental conditions and improve the performance of target detection.

[0057] In a feasible embodiment, in order to obtain high-quality infrared and visible light data of the transformer, a high-resolution infrared thermal imager (such as LIR T640) and a high-definition industrial camera (such as Sony ILX-550) can be carried on the UAV to respectively collect infrared radiation signals and visible light images of the surface of the transformer. Among them, the infrared thermal imager can detect the temperature distribution of the surface of the transformer, accurately capturing temperature differences caused by internal faults, uneven load, etc. The industrial camera can identify physical damage (such as cracks, oil stains, corrosion, etc.) on the surface of the transformer through high-resolution visible light images. In order to ensure the synchronous collection of multi-modal data, a hardware trigger synchronization mechanism is adopted, which activates the exposure process of the infrared thermal imager and the industrial camera at the same time through an external trigger signal, ensuring the time consistency of the two modal images.

[0058] In a feasible embodiment, the UAV can carry a LiDAR or a depth camera (such as an RGB-D camera or a ToF camera), collect the environmental blocking information around the power equipment (such as a power transmission line or a substation) in real time through high-precision three-dimensional environmental perception capability, and dynamically adjust the flight path to avoid collision risks.

[0059] Referring to Figure 3 , Figure 3 is a structure diagram of a heat detection model provided in an embodiment of the present application. The model includes a plurality of scale feature extraction modules, a decoupling multi-modal learning module, a spatial offset module, an offset guiding module, a decoupling feature fusion module, an attention enhancement module, and a detection head module. The functions of these modules are introduced as follows: the scale feature extraction module can extract features from multi-modal data such as infrared thermal imaging images and visible light images, among which the infrared data is input in the form of infrared thermal imaging images and the visible light data is input in the form of visible light images, and finally visible light multi-modal features containing multi-modal information are output. The decoupling multi-modal learning module generates visible light modal invariant features and infrared modal invariant features by aligning and fusing the visible light features and the infrared features; the spatial offset module performs spatial offset processing on the two types of modal invariant features and outputs a spatial offset; the offset guiding module is used to combine the visible light modal invariant features, the infrared modal invariant features, and the spatial offset, and through offset compensation and adaptive alignment, obtain visible light modal specific features and infrared modal specific features; the decoupling feature fusion module fuses the infrared modal invariant features, the visible light modal specific features, the visible light modal invariant features, and the infrared modal specific features to form fusion features; the attention enhancement module implements feature enhancement on the fusion features to generate a feature map; and the detection head module completes detection based on the feature map and outputs a heat detection result.

[0060] In a feasible embodiment, as shown in Figure 3 , the scale feature extraction module includes a first feature extraction unit and a second feature extraction unit. The first feature extraction unit is used to extract features from infrared data to obtain infrared multi-modal features, and the second feature extraction unit is used to extract features from visible light data to obtain visible light multi-modal features.

[0061] In a feasible embodiment, the first feature extraction unit and the second feature extraction unit can adopt independent basic pyramid feature extractors. The visible light image and the infrared image input into the heat detection model are respectively input into the corresponding independent basic pyramid feature extractors to extract multi-modal features , wherein the first feature extraction unit extracts infrared multi-modal features and the second feature extraction unit extracts visible light multi-modal features. The pyramid feature extractor is as shown inFigure 4 As shown, a multi-layer convolutional structure is used, and the output features are bounded random samples. .

[0062] In one feasible embodiment, such as Figure 5 As shown, the decoupled multimodal learning module constructs a common subspace encoder (C) consisting of convolutional layers and SILU activation functions, and a specific subspace encoder (S) containing convolutional layers and a C3 module (containing three convolutional layers and a bottleneck layer). It combines a multi-task loss function system, including similarity loss (minimizing modality-invariant feature distance to learn the common subspace), difference loss (maximizing modality-specific feature differences to capture complementary information), and semantic loss (guiding feature alignment to true semantics), to achieve efficient alignment and fusion of visible light and infrared image features. Among the similarity loss, difference loss, and semantic loss, similarity loss can be used to minimize visible light modality-invariant features. Infrared mode invariant features The distance between them is used to learn the common subspace C; secondly, the difference loss is used to highlight... and The differences between them have been supplemented with additional information.

[0063] The following is combined with Figure 5 The working principle of the decoupled multimodal learning module is explained.

[0064] First, the multimodal features are input into a modality-invariant feature encoder with shared parameters. (Composed of convolutional layers and SILU activation function), generating mode-invariant features in the common subspace, as shown in Equation (1):

[0065]

[0066] in, These are encoder parameters shared across modalities.

[0067] Multimodal features Modality-specific feature encoder with input to independent parameters (Composed of convolutional layers, C3 modules, and bottleneck layers), generating specific subspaces. Modal-specific features in As shown in formula (2):

[0068]

[0069] in, These are encoder parameters that are independent for each mode.

[0070] Secondly, the central moment difference (CMD) is used to measure the visible light mode invariance characteristics. Infrared mode invariant features The distribution consistency is then used to calculate the similarity loss as shown in formula (3). .

[0071]

[0072] in For the expected value based on experience, It is the k-th order central moment.

[0073] By utilizing matrix trace constraints to mine the complementarity between modality-specific features and ensuring the independence between features, the difference loss is calculated as shown in Equation (4). .

[0074]

[0075] in It is the trace of the matrix.

[0076] By integrating the classification loss from the object detection task Regression loss and target confidence loss The common features are forced to have discriminative ability, and then the semantic loss is calculated as shown in formula (5). .

[0077]

[0078] Finally, the above losses are weighted and combined to arrive at the total loss. The calculation method is shown in formula (6).

[0079]

[0080] in It's about weighing the weights, specifically... For similarity loss The weight, For difference loss The weight, For semantic loss The weights are adjusted. Network parameters are updated via backpropagation to optimize feature decoupling. The optimized visible light invariant features are then... Compared with original infrared features Perform spatial alignment.

[0081] It is understandable that when dealing with visible light and infrared mode-invariant features... and When calculating the difference, traditional direct subtraction or feature connection methods are easily affected by complex background noise in UAV images.

[0082] To this end, the embodiment adopts a feature stitching strategy to generate change features, and suppresses background interference and extracts effective offset information through spatial offset modeling. As shown in Figure 6 The spatial offset module includes three processing units: a spatial attention unit: through the spatial attention mechanism, the key areas (such as transformer target areas) in the invariant features of the visible light and infrared modalities are automatically captured, and the feature response of the background noise is suppressed; a channel difference enhancement unit: based on the key area features, the spatial channel difference features across modalities are calculated, and the inter-modal change information of the target area is highlighted through weighted enhancement in the channel dimension; a convolution layer: the spatial channel enhanced features are nonlinearly transformed and fused through convolution operation to generate the final spatial offset. The spatial offset module of the embodiment can effectively solve the noise interference problem in the calculation of modal feature differences through the processing flow of focusing on the target through the attention mechanism, highlighting the changes through channel difference enhancement, and suppressing noise through convolution fusion.

[0083] The working principle of the spatial offset module will be described below. Figure 6

[0084] Firstly, the input a spatial attention unit to capture the representation of the target and key background area in each modality. In this framework, the spatially enhanced unimodal features can be calculated according to formula (7). Specifically, the visible light modality invariant features and the infrared modality invariant features are input into a spatial attention unit respectively, and the corresponding spatially enhanced unimodal features are calculated.

[0085]

[0086] wherein, is a sigmoid function, represents a convolution layer, is a connection operation, represents a Hadamard product operation.

[0087] Further, the channel difference enhancement unit is used to suppress channel noise, and parallel average pooling and maximum pooling are used to generate multi-type spatial context descriptors; then the descriptors are input into a shared multi-layer perceptron network to obtain channel attention, and the is fused with the channel attention map to obtain the spatial channel enhanced features , and the specific calculation is shown in formula (8).

[0088]

[0089] ​Finally, by fusing the and with the nonlinear layer, the spatial offset can be obtained.

[0090] Referring to Figure 7 , Figure 7 is a principle structure diagram of an offset guiding module provided in an embodiment of the present application. The module adopts a deformable convolution technology, and based on the offset generated by the cross-modal spatial offset modeling module, realizes implicit offset compensation and adaptive feature alignment in a detection task, so as to more effectively correct the feature space distribution and guarantee the stability and reliability of model training.

[0091] The working principle of the offset guiding module will be described below in combination with Figure 7 .

[0092] Considering that each offset region has different effects on the detection task, a modulation scalar learned from c is added to dynamically aggregate the information around the corresponding position p. Regarding the calculation of the sampling value, according to the center sampling value, the basic offset, the fixed offset and the implicit compensation, the corresponding value in the aligned feature is calculated, and the bilinear interpolation is adopted to obtain the final sampling value. The specific calculation is shown in formula (9):

[0093]

[0094] wherein K represents the number of kernel weights, represents the kth fixed offset, represents the kth kernel weight, and represents the implicit compensation of the kth position. Because the position can be a decimal number, the bilinear interpolation is adopted to obtain the final sampling value. The bilinear interpolation estimates the value of a non-integer coordinate position through the values of the surrounding four nearest integer coordinate points. Specifically, assuming that the value of position (x, y) needs to be estimated, wherein x and y are decimals. The bilinear interpolation will find the surrounding four integer coordinate points (x1, y1), (x1, y2), (x2, y1) and (x2, y2), and then perform a weighted average according to the values of the four points.

[0095] Referring to Figure 8 , Figure 8 is a principle structure diagram of a decoupled feature fusion module provided in an embodiment of the present application. The working principle of the module includes: first, decoupling the visible light feature and the infrared feature into modal invariant features and modal specific features respectively. In the traditional fusion process, directly connecting and may cause redundancy, so before fusion, it is necessary to combine and optimize and to eliminate redundant information and enhance discriminative representation. The aligned visible light modal invariant features and infrared modal invariant features are combined and optimized by convolutional layers. The optimized modal invariant features are concatenated with the visible light and infrared modal specific features and to generate the final fusion features as shown in equation (10):

[0096]

[0097] Referring to Figure 9 , Figure 9 is a schematic diagram of an attention-enhanced module according to an embodiment of the present application. The module is composed of a cavity normalization activation unit and a sigmoid activation function connected in sequence, wherein the cavity normalization activation unit comprises a cavity convolutional layer, a batch normalization layer and a ReLU activation function connected in sequence. The cavity convolution expands the receptive field without increasing the number of parameters, obtaining multi-scale context information; the batch normalization layer normalizes the input data, suppresses the internal covariate shift, accelerates the model convergence and improves the stability; and the ReLU activation function introduces nonlinearity into the model, enhancing its learning ability for complex nonlinear mapping relationships. The improved spatial attention mechanism adopted in this embodiment generates a spatial attention mask by element-wise weighting of the feature map through a learnable weight matrix, dynamically enhancing the features so that the network can adaptively adjust the attention to different spatial positions, as shown in equation (11).

[0098]

[0099] In the feature reconstruction phase, three groups of cascaded decoders are used to restore the feature map size, wherein the first level decoder processes the input features by bilinear upsampling to ensure the consistency of the feature map resolution in the subsequent decoding process, as shown in equation (12).

[0100]

[0101] wherein represents the decoded feature map generated by the first decoder.

[0102] In a feasible embodiment, the detection head module adopts a detection structure based on the Transformer, which directly predicts the bounding box and confidence of the abnormal candidate region using the global self-attention mechanism. Through the multi-scale feature pyramid strategy, the system can generate candidates for abnormal regions of different sizes, and combined with the preset temperature threshold and texture anomaly measurement method, adaptive abnormal candidate region detection is realized. The loss function of the detection head is defined as shown in equation (13):

[0103]

[0104] In formula (13), S is an abnormal area thermal map of a model output, R is a real residual map (generated by calculating the difference between an infrared image and a visible light image), H and W are the height and width of a feature map respectively. The hyperparameters are optimized by grid search. Subsequently, non-maximum suppression (NMS) is used to screen the detected candidate regions to eliminate repeated detection, and the abnormal area located in the image is accurately mapped to a geographic coordinate with the help of the GPS and IMU of the unmanned aerial vehicle, so as to realize fine positioning.

[0105] In a feasible embodiment, as shown in FIG. 10, the execution process of detecting the feature map by using the detection head module to obtain the heat detection result can include but is not limited to steps 1010 to 1040. Figure 10

[0106] Step 1010: analyzing the feature map by using the global self-attention mechanism of the detection head module to capture global context information of the abnormal area;

[0107] Step 1020: generating abnormal candidate areas of different sizes based on the global context information of the abnormal area by using a multi-scale feature pyramid strategy;

[0108] Step 1030: screening the abnormal candidate areas according to a preset temperature threshold and a texture measurement method, and performing non-maximum suppression processing on the screened abnormal candidate areas to obtain a target abnormal candidate area;

[0109] Step 1040: mapping the target abnormal candidate area from an image coordinate to a geographic coordinate to obtain a heat detection result.

[0110] In a feasible embodiment, in step 1010, the detection head module adopts a structure based on Transformer, in which the global self-attention mechanism can enable the model to analyze the feature map without being limited to local information, but can consider the information of each position on the feature map at the same time. By calculating the correlation degree between different positions in the feature map, the context information of the abnormal area in the entire image is captured. For example, when detecting the heat abnormality of a transformer, not only the features of the suspected heat point itself can be focused on, but also the related features such as the surrounding device structure and environment can be combined, so that the situation of the abnormal area can be more comprehensively and accurately understood.

[0111] ​In an embodiment, in step 1020, the multi-scale feature pyramid strategy is based on the global context information of the abnormal area captured in step 210. Since the abnormal area in the actual scene can have different sizes, the feature map is constructed into a pyramid structure of different scales through the multi-scale feature pyramid strategy. Abnormal candidate regions are generated at different scales, which can adapt to abnormal areas of different sizes. For example, both large-scale overheating areas and small local hot spots can generate corresponding candidate regions, ensuring that no abnormal situation of different sizes is missed.

[0112] In an embodiment, in step 1030, the generated abnormal candidate regions are first screened according to the preset temperature threshold and texture measurement method. The temperature threshold is used to determine whether the temperature of the candidate region is out of the normal range, and the texture measurement method is used to analyze whether the texture features of the candidate region are abnormal. Through these two ways, the area that may truly exist abnormality is screened out. Then, non-maximum suppression (NMS) processing is adopted. Because multiple regions may overlap with each other and represent the same abnormality when generating candidate regions, NMS retains the region with high confidence by comparing the confidence and other indicators of the candidate regions, and suppresses the repeated regions, so as to obtain the target abnormal candidate region, so that the detection result is more accurate and redundant detection is removed.

[0113] In an embodiment, in step 1040, the unmanned aerial vehicle is equipped with GPS (Global Positioning System) and IMU (Inertial Measurement Unit), and the target abnormal candidate region in the image coordinates obtained in step 230 is accurately mapped to the actual geographic coordinates by using the data of these devices. In this way, the specific position of the abnormal area in the real geographic environment can be determined, and an accurate heat detection result can be given.

[0114] In an embodiment, after obtaining the heat detection result, the specific transformer with abnormal heat can be first located based on the geographic coordinates in the detection result and the equipment account data; then the current detection result and the historical detection data are fused, and the severity of the abnormality is classified (for example, into slight, severe, critical, etc.) by analysis; finally, according to the classification result, the corresponding level of warning information (such as blue, yellow, red warning) and maintenance suggestion (such as continuous monitoring, limited period of verification, immediate repair, etc.) are output. Specifically, this process locks the faulty equipment through the association of the detection result and the equipment data, realizes scientific classification of the severity of the abnormality in combination with the historical data, and generates targeted warning and disposal schemes based on this.

[0115] Referring to Figure 11 , Figure 11is a whole flow chart of transformer fault detection based on unmanned aerial vehicle (UAV) inspection provided by an embodiment of the present application. After the flight path of the UAV is determined, the UAV performs point-by-point flight, collects the blocking information around the power equipment by using a laser radar or a depth camera, and simultaneously collects visible light images and infrared images by using a visible light camera (such as a common industrial camera) and an infrared thermal imager. Then, a pre-trained heat detection model is used to perform multi-modal feature extraction, cross-modal alignment fusion and anomaly detection on the images, to obtain a heat detection result. Subsequently, the system determines an abnormally heated transformer based on the geographic coordinates in the detection result and equipment account data, and then fuses the current detection result and historical detection data to analyze the severity classification of the abnormality. Finally, the system outputs warning information and maintenance suggestions of the corresponding level according to the classification result, to help the staff to determine whether there is an abnormally heated transformer in the area where the UAV performs work and to perform corresponding maintenance operations. The operation and maintenance feedback can be used to update the fault case library and optimize the detection model, to form a closed-loop improvement process.

[0116] Reference Figure 12 , Figure 12 is a system structure diagram of a transformer fault detection system based on UAV inspection provided by an embodiment of the present application. The fault detection system can be applied to the above-mentioned transformer fault detection method based on UAV inspection. The system comprises: a path planning module 1210, configured to construct a detection range of a transformer, and plan a flight path of a UAV cluster according to the detection range; a multi-modal perception module 1220, configured to acquire infrared data and visible light data of the transformer collected by the UAV cluster when flying along the flight path; and an anomaly detection module 1230, configured to call a pre-trained heat detection model to perform detection according to the infrared data and the visible light data, to obtain a heat detection result. The heat detection model comprises a multi-scale feature extraction module, a decoupled multi-modal learning module, a spatial offset module, an offset guide module, a decoupled feature fusion module, an attention enhancement module and a detection head module. The output end of the multi-scale feature extraction module is connected with the input end of the decoupled multi-modal learning module. The output end of the decoupled multi-modal learning module is connected with the spatial offset module and the offset guide module respectively. The output end of the offset guide module is connected with the input end of the decoupled feature fusion module. The output end of the decoupled feature fusion module is connected with the input end of the attention enhancement module. The attention enhancement module is connected with the detection head module.

[0117] In an implementable embodiment, the multiscale feature extraction module can perform feature extraction on multi-modal data such as infrared thermal imaging images and visible light images, wherein the infrared data and the visible light data can be input in the form of corresponding images, and finally output visible light multi-modal features containing multi-modal information. The decoupled multi-modal learning module generates visible light modal invariant features and infrared modal invariant features by aligning and fusing the visible light features and the infrared features; the spatial offset module performs spatial offset processing on the two types of modal invariant features, and outputs spatial offset; the offset guiding module combines the visible light modal invariant features, the infrared modal invariant features, and the spatial offset, and obtains visible light modal specific features and infrared modal specific features through offset compensation and adaptive alignment; the decoupled feature fusion module fuses the infrared modal invariant features, the visible light modal specific features, the visible light modal invariant features, and the infrared modal specific features into fusion features; the attention enhancement module performs feature enhancement on the fusion features to generate feature maps; and the detection head module completes detection based on the feature maps to output the heat detection result.

[0118] In an implementable embodiment, the multi-modal perception module includes an infrared thermal imager and an industrial camera. The infrared thermal imager is used to capture infrared radiation data of the transformer, and the industrial camera is responsible for collecting visible light images of the transformer.

[0119] In an implementable embodiment, the multi-modal perception module further includes a LiDAR or a depth camera (such as an RGB-D camera or a ToF camera). The LiDAR or the depth camera can capture the position, distance, and shape information of obstacles around power equipment such as power transmission lines and substations in real time by virtue of high-precision three-dimensional environment perception capability, thereby supporting dynamic adjustment of flight paths of the unmanned aerial vehicle to avoid collision risks.

[0120] It should be noted that the transformer fault detection system is fully compatible with the aforementioned transformer fault detection method, and the function implementation logic and principles of the modules of the system are one-to-one corresponding to the technical solutions of the method. The specific functions of the modules can be referred to the foregoing detailed description, and will not be described here again.

[0121] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be apparent to those of ordinary skill in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Accordingly, the application will not be limited to the embodiments shown herein but is intended to cover any alternatives, modifications, equivalents and the like falling within the scope of the principles and novel features disclosed herein.

Claims

1. A transformer fault detection method based on unmanned aerial vehicle inspection, characterized in that, The method comprises the steps of: constructing a detection range of a transformer, and planning a flight path of a UAV cluster according to the detection range; acquiring infrared data and visible light data of the transformer collected when the UAV cluster flies along the flight path; calling a pre-trained heat detection model to detect according to the infrared data and the visible light data, to obtain a heat detection result; The heat detection model comprises a multi-scale feature extraction module, a decoupling multi-modal learning module, a spatial offset module, an offset guiding module, a decoupling feature fusion module, an attention enhancement module and a detection head module. The multi-scale feature extraction module is used for feature extraction of the infrared data to obtain infrared multi-modal features, and feature extraction of the visible light data to obtain visible light multi-modal features. The decoupling multi-modal learning module is used for mutual alignment and fusion of the visible light multi-modal features and the infrared multi-modal features to obtain visible light modal invariant features and infrared modal invariant features. The spatial offset module is used for spatial offset processing of the visible light modal invariant features and the infrared modal invariant features to obtain a spatial offset amount. The offset guiding module is used for offset compensation and adaptive alignment processing according to the visible light modal invariant features, the infrared modal invariant features and the spatial offset amount to obtain visible light modal specific features and infrared light modal specific features. The decoupling feature fusion module is used for feature fusion processing of the infrared modal invariant features, the visible light modal specific features, the visible light modal invariant features and the infrared light modal specific features to obtain fusion features. The attention enhancement module is used for feature enhancement processing of the fusion features to obtain a feature map. The detection head module is used for detection of the feature map to obtain a heat detection result.

2. The transformer fault detection method based on unmanned aerial vehicle inspection according to claim 1, characterized in that, The multi-scale feature extraction module comprises a first feature extraction unit and a second feature extraction unit. The first feature extraction unit is used for feature extraction of the infrared data to obtain infrared multi-modal features. The second feature extraction unit is used for feature extraction of the visible light data to obtain visible light multi-modal features. 3.The transformer fault detection method based on UAV inspection of claim 1, wherein, The decoupling multi-modal learning module comprises a common subspace encoder, a specific subspace encoder, a similarity loss function, a difference loss function and a semantic loss function. The common subspace encoder comprises mutually connected convolution layers and SILU activation functions. The specific subspace encoder comprises mutually connected convolution layers and bottleneck layers.

4. The transformer fault detection method based on unmanned aerial vehicle inspection according to claim 2, characterized in that, The spatial offset module comprises a spatial attention unit, a channel difference enhancement unit and a convolution layer. The spatial attention unit is used for capturing key region features of the visible light modal invariant features and the infrared modal invariant features. The channel difference enhancement unit is used for calculating spatial channel enhancement features according to the key region features. The convolution layer is used for nonlinear fusion of the spatial channel enhancement features to obtain a spatial offset amount.

5. The transformer fault detection method based on UAV inspection according to claim 2, characterized in that, The attention enhancement module comprises a cavity normalization activation unit and a sigmoid activation function connected in sequence, and the cavity normalization activation unit comprises a cavity convolution layer, a normalization layer and a ReLU activation function connected in sequence.

6. The transformer fault detection method based on UAV inspection according to claim 2, characterized in that, The detection head module is used to detect the feature map to obtain a heat detection result, including: The global self-attention mechanism of the detection head module is used to analyze the feature map and capture global context information of the abnormal area; Through a multi-scale feature pyramid strategy, different sizes of abnormal candidate areas are generated based on the global context information of the abnormal area; According to a preset temperature threshold and a texture measurement method, the abnormal candidate areas are screened, and a non-maximum suppression process is performed on the screened abnormal candidate areas to obtain a target abnormal candidate area; The target abnormal candidate area is mapped from an image coordinate to a geographic coordinate to obtain a heat detection result.

7. The transformer fault detection method based on UAV inspection according to claim 1, characterized in that, The detection range of the transformer is constructed, and the flight path of the UAV cluster is planned according to the detection range, including: According to the preset starting point and target point of the UAV cluster, the detection area of the transformer is determined; A target attractive potential field and an obstacle repulsive potential field are arranged at the position of the target point to generate a total potential field for path planning of the UAV cluster; The flight path of the UAV cluster in the detection area is calculated by using a discrete particle swarm algorithm, taking the total potential field as a constraint condition, combining an artificial potential field with a Voronoi diagram arc weight. 8.The transformer fault detection method based on UAV inspection according to claim 1, wherein, After obtaining the heat detection result, the method further includes: determining an abnormal heat transformer based on the heat detection result; analyzing the heat detection result and historical detection data to determine an abnormal level of the abnormal heat transformer; outputting warning information and maintenance recommendations corresponding to the abnormal level.

9. A transformer fault detection system based on UAV inspection, characterized in that, The system is applied to a transformer fault detection method based on UAV inspection according to any one of claims 1-8, and the system comprises: a path planning module for constructing a detection range of a transformer and planning a flight path of a UAV cluster according to the detection range; a multi-modal perception module for acquiring infrared data and visible light data of the transformer collected by the UAV cluster when flying along the flight path; an abnormality detection module for calling a pre-trained heat detection model to detect the infrared data and the visible light data to obtain a heat detection result; the heat detection model comprises a multi-scale feature extraction module, a decoupled multi-modal learning module, a spatial offset module, an offset guide module, a decoupled feature fusion module, an attention enhancement module and a detection head module, an output end of the multi-scale feature extraction module is connected with an input end of the decoupled multi-modal learning module, output ends of the decoupled multi-modal learning module are connected with the spatial offset module and the offset guide module respectively, an output end of the offset guide module is connected with an input end of the decoupled feature fusion module, an output end of the decoupled feature fusion module is connected with an input end of the attention enhancement module, and the attention enhancement module is connected with the detection head module.

10. The transformer fault detection system based on UAV inspection according to claim 9, wherein, The multi-modal perception module includes an infrared thermal imager for capturing infrared data of the transformer and an industrial camera for capturing visible light images of the transformer. The multi-modal perception module includes an infrared thermal imager for capturing infrared data of the transformer and an industrial camera for capturing visible light images of the transformer.

Images