Power equipment infrared thermal defect diagnosis method based on cross-modal visual information migration

Abstract

This invention relates to a method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer. The specific steps include: S1, acquiring visible light images and infrared thermal images of the target power equipment and their corresponding temperature matrices; S2, extracting edge features from the visible light images and infrared thermal images respectively, detecting key points and calculating feature descriptors for the key points, performing cross-modal key point matching, and obtaining a set of matching point pairs; S3, obtaining a registered visible light image aligned with the viewpoint of the infrared thermal image based on the corrected set of matching point pairs; S4, performing instance segmentation on the registered visible light image to obtain segmentation prediction maps corresponding to each power equipment instance; scaling all segmentation prediction maps to the same size as the infrared thermal image, and extracting pixels corresponding to the power equipment regions as segmentation masks; S5, using the segmentation masks to extract temperature data within the segmentation mask regions from the temperature matrix, calculating the maximum temperature difference, and determining whether each device has a heating defect.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology for power equipment, specifically to a method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer. Background Technology

[0002] During long-term operation, power line equipment is prone to abnormal heating due to poor contact, overload, insulation aging, and other reasons. Timely detection and identification of heating defects in equipment is an important means to ensure the safe operation of the power system. Currently, using drones equipped with infrared thermal imagers to inspect power lines has become the mainstream solution in the industry. By collecting infrared thermal images of the equipment and performing temperature analysis, it is possible to effectively identify whether there are abnormal temperature differences in the equipment.

[0003] In the existing patent "CN112288761B", a method, apparatus and readable storage medium for detecting abnormal heating of power equipment, although the heating detection is improved by increasing the accuracy of feature point matching, it still uses low-quality infrared images as the basis for segmentation. It lacks a systematic solution to the huge resolution difference and modal gap between infrared and visible light images, and it is difficult to avoid background interference, resulting in inaccurate segmentation and a high misjudgment rate of heating defects.

[0004] In existing technologies, the typical process for identifying overheating defects in power equipment is as follows: First, the infrared image is segmented into equipment regions using an instance segmentation model to obtain a mask for the equipment; then, the mask is superimposed on the temperature matrix corresponding to the infrared image to extract temperature data within the mask region; finally, the maximum temperature difference within the region is calculated and compared with the threshold specified by industry standards to determine whether the equipment has an overheating defect.

[0005] However, the above methods have a significant technical problem: infrared thermal images typically have low resolution (e.g., 640×512 or 1280×1024), and infrared images only contain information from a single temperature channel, often resulting in insufficient contrast between the device and the background. Under these conditions, the segmentation accuracy of the instance segmentation model is difficult to guarantee, and it is easy to incorrectly segment background areas (such as sky, ground, vegetation, etc.) as device areas. Since the temperature of the background area is usually significantly lower or higher than the device temperature, once background pixels are included in the device mask, it will cause serious deviations in the temperature difference calculation results, generating a large number of misjudgments and severely affecting the accuracy of heat defect identification.

[0006] Therefore, improving the accuracy of segmentation of power equipment in infrared images, thereby enhancing the accuracy of identifying heating defects, is a technical problem that urgently needs to be solved. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention proposes a method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer.

[0008] The technical solution of the present invention is as follows: This invention proposes a method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer, the specific steps of which include: S1. Acquire visible light images and infrared thermal images of the target power equipment and their corresponding temperature matrices; S2. Extract edge features from visible light images and infrared thermal images respectively to obtain visible light edge maps and infrared edge maps; detect key points on the two edge maps respectively and calculate the feature descriptors of the key points, perform cross-modal key point matching, obtain a set of matching point pairs, and use the RANSAC filtering algorithm to remove incorrect matching point pairs; S3. Based on the corrected set of matching point pairs, calculate the projective transformation matrix from the visible light image to the infrared thermal image coordinate system; use the projective transformation matrix to perform a geometric transformation on the visible light image to obtain a registered visible light image aligned with the viewpoint of the infrared thermal image. S4. Perform instance segmentation on the registered visible light image to obtain the segmentation prediction map corresponding to each power equipment instance; scale all segmentation prediction maps to the same size as the infrared thermal image, and extract the pixels corresponding to the power equipment area as segmentation masks. S5. Using the segmentation mask of each power equipment instance, extract the temperature data within the segmentation mask area from the temperature matrix, calculate the maximum temperature difference of each power equipment instance, and determine whether each device has a heat generation defect based on the preset temperature difference threshold.

[0009] In a preferred embodiment, the feature descriptor is a phase-consistency-based structural descriptor or a Log-Gabor histogram descriptor.

[0010] In a preferred embodiment, the step of detecting key points and calculating feature descriptors of key points on two edge maps respectively, and performing cross-modal key point matching, is implemented by a pre-trained cross-modal feature extraction deep learning network, and the matching of corresponding key points is achieved based on the matching between feature descriptors.

[0011] In a preferred embodiment, the projective transformation matrix is ​​a homography transformation matrix, which is solved based on the set of matching point pairs using a direct linear transformation algorithm or the least squares method.

[0012] As a preferred embodiment, bilinear interpolation or bicubic interpolation is used for resampling when performing geometric transformations on visible light images.

[0013] In a preferred embodiment, a hierarchical registration strategy combining downsampling coarse registration and region of interest fine registration is adopted in steps S2 and S3. This hierarchical registration strategy specifically includes three stages: a downsampling coarse registration stage, a region locking stage, and a fine registration stage. In the downsampling coarse registration stage, the visible light image is downsampled to a resolution similar to that of the infrared thermal image. Edge detection, key point matching, and RANSAC filtering are then performed between the downsampled visible light image and the infrared thermal image to obtain a coarse transformation matrix. During the region locking stage, the field of view of the infrared thermal image is back-projected onto the visible light original image coordinate system using the coarse transformation matrix, and the corresponding region of interest sub-image is cropped out. In the fine registration stage, edge detection, key point matching, and RANSAC filtering are performed again between the region of interest sub-image and the infrared thermal image to obtain the fine transformation matrix. The local coordinates are then converted into global coordinates and used to solve the projective transformation matrix.

[0014] As a preferred embodiment, the instance segmentation employs a deep learning-based instance segmentation model, including Mask R-CNN, SOLOv2, and YOLACT.

[0015] In a preferred embodiment, in the step of segmenting the registered visible light image to obtain the segmentation prediction map corresponding to each power equipment instance, nearest neighbor interpolation is used to scale the segmentation prediction map to the same size as the infrared thermal image.

[0016] In a preferred embodiment, the step of extracting temperature data within the segmentation mask region from the temperature matrix using the segmentation mask of each power equipment instance, calculating the maximum temperature difference of each power equipment instance, and determining whether each device has a heat generation defect based on a preset temperature difference threshold specifically includes: The segmentation mask of each power equipment instance is applied to the temperature matrix, and the temperature values ​​corresponding to all pixels in each mask area are extracted to form a temperature dataset. For each device instance, calculate the difference between the highest and lowest temperature values ​​in the temperature dataset, which is taken as the maximum temperature difference for that device instance. The maximum temperature difference is compared with the temperature difference threshold set according to the power industry standard. If the maximum temperature difference is greater than or equal to the temperature difference threshold, the device is determined to have a heating defect.

[0017] As a preferred embodiment, the method further includes step S6, which involves identifying the type of power equipment by using the registered visible light image after determining whether each device has a heat defect, matching the corresponding temperature difference judgment threshold according to the equipment type, and realizing the determination of the heat defect level by type.

[0018] The present invention has the following beneficial effects: 1. This invention utilizes a visible light image-guided infrared image segmentation technique, effectively solving the problem of insufficient segmentation accuracy caused by the low resolution and limited information dimensions of infrared images. Registering a visible light image not only preserves the color and texture information of the RGB three channels but also allows for flexible setting of the output resolution for segmentation. This fully leverages the information advantages of visible light images to obtain more accurate device boundaries. Furthermore, by scaling and mapping to the infrared coordinate system, segmentation accuracy is significantly improved.

[0019] 2. This invention adopts a cross-modal feature matching method based on edge structure, which overcomes the problem that traditional feature matching methods fail due to the large difference in appearance between visible light and infrared images. It also proposes a hierarchical registration strategy that combines downsampling coarse registration with region of interest fine registration. By determining the corresponding region through coarse matching and improving the registration accuracy through fine matching, this invention effectively solves the matching difficulties caused by the large difference in resolution and inconsistent field of view between the two modal images.

[0020] 3. This invention significantly improves the accuracy of segmentation mask through cross-modal feature point extraction and matching and visible light image registration steps, so that the highest and lowest temperatures in the mask area can truly reflect the temperature distribution of the device body. The maximum can be directly calculated to accurately determine the heating defect, avoiding temperature difference calculation deviation and misjudgment caused by background pixel mixing. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0024] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0025] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0026] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0027] Example 1: See Figure 1 A method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer, the specific steps of which include: Step S1: Acquire visible light images and infrared thermal images of the target power equipment and their corresponding temperature matrices; Step S1 specifically involves dual-modal image acquisition. The target power line is inspected and photographed using a dual-light fusion imaging module mounted on the UAV. This module includes a visible light lens and an infrared thermal imaging lens, both mounted on the same pan-tilt unit with nearly parallel optical axes and minimal spacing between their shooting positions. At the same trigger moment, the visible light lens acquires a high-resolution visible light image. An infrared thermal imaging camera acquires a low-resolution infrared thermal image. and its corresponding temperature matrix The resolution of the visible light image is higher than that of the infrared thermal image, and the field of view of the infrared thermal image is contained within the field of view of the visible light image.

[0028] In one specific embodiment, a visible light image The resolution is 4000×3000 pixels, infrared thermal image. The resolution is 640×512 pixels. The field of view of an infrared thermal image is smaller than that of a visible light image; the imaging area of ​​an infrared thermal image corresponds to a sub-region near the center of the visible light image. Temperature matrix. With infrared thermal image They are all the same size, and each element records the absolute temperature value of the corresponding pixel.

[0029] Step S2: Extract edge features from visible light image and infrared thermal image respectively to obtain visible light edge map and infrared edge map; detect key points on the two edge maps respectively and calculate the feature descriptors of key points, perform cross-modal key point matching, obtain a set of matching point pairs, and use RANSAC filtering algorithm to remove incorrect matching point pairs; Step S2 specifically involves cross-modal feature point extraction and matching. Due to significant differences in grayscale distribution and texture features between visible light images and infrared thermal images, the success rate of directly using traditional grayscale gradient-based feature descriptors (such as SIFT and ORB) for matching is low. Furthermore, the resolution of visible light images is much higher than that of infrared thermal images (e.g., 4000×3000 vs. 640×512), and the infrared thermal image only covers a small area in the center of the visible light image. These two images differ greatly in scale and field of view. Directly performing global feature matching on the original resolution image would result in problems such as an excessively large search space, inconsistent scales, and a high rate of incorrect matches. Therefore, this application employs a hierarchical registration strategy combining downsampling coarse registration and region of interest fine registration, combined with a cross-modal feature matching strategy based on edge structures.

[0030] S2.1 Downsampling and coarse registration stage.

[0031] First, for visible light images Downsampling is performed to construct an image pyramid with decreasing resolution. Let the original resolution of the visible light image be... × The infrared thermal image resolution is × Then calculate the downsampling factor r:

[0032] Downsample the visible light image to a resolution of ( / r)×( / r), to obtain low-resolution visible light images In one specific embodiment, r ≈ 5, the visible light image is downsampled from 4000×3000 to approximately 800×600, bringing it to a resolution level similar to that of the 640×512 infrared thermal image.

[0033] For low-resolution visible light images and infrared thermal images Edge detection is performed separately. Taking the Canny operator as an example, the edge detection process includes: Gaussian filtering the image for noise reduction, where the Gaussian kernel function is:

[0034] in, Let x be the standard deviation of the Gaussian kernel, x be the horizontal offset of a pixel relative to the kernel center, and y be the vertical offset of a pixel relative to the kernel center. The horizontal gradient is then calculated using the Sobel operator after filtering. and vertical gradient Synthesize the gradient magnitude M and gradient direction :

[0035] After nonmaximum suppression and dual thresholding, a low-resolution visible light edge map is obtained. and infrared edge map Keypoints are detected and feature descriptors are calculated on two edge maps. The feature descriptors can be Log-Gabor histogram descriptors, and their calculation process is as follows: a multi-scale, multi-directional Log-Gabor filter bank is applied to the local region surrounding the keypoint to obtain the local phase response; the phase response is statistically plotted into a histogram according to direction and scale, and then concatenated into a feature vector d∈ n Where n is the descriptor dimension, which is equal to the product of the number of scales, the number of directions, and the number of histogram intervals in each filter bank. For the set of real numbers, n Let represent an n-dimensional real vector space.

[0036] For the two sets of descriptor subsets, the visible light edge map descriptor subset { } and infrared thermal image descriptor subset { Matching is performed on the descriptor of each visible light keypoint. Search for Euclidean distance in the infrared descriptor subset Minimal matching:

[0037] The RANSAC filtering algorithm is used to remove erroneous matches, resulting in a coarse set of matching point pairs and a coarse transformation matrix. .

[0038] S2.2 Area Lockdown Phase.

[0039] Using coarse transformation matrix Infrared thermal image The coordinates of the four corner points are back-projected onto the original visible light image. In the coordinate system, let the four corner points of the infrared image be... (k=1,2,3,4), then the corresponding visible light coordinates for:

[0040] by{ The bounding rectangle of the rectangle is expanded outwards by a preset margin. (e.g., 5% of the side length), from the original visible light image Crop the area of ​​interest from the middle The ROI local coordinate system preserves all the details of the original resolution, and its scale difference from the infrared image is reduced to approximately 2 to 4 times.

[0041] S2.3 Fine registration stage.

[0042] exist and infrared thermal images Edge detection, keypoint extraction, descriptor calculation, and matching are performed again between the two points. Because the content of each point now highly overlaps and the scale difference is significantly reduced, the matching quality is greatly improved. After RANSAC removes incorrect matches, a set of perfectly matched point pairs is obtained. ,in, For a fine-grained visible light edge map description subset, is a subset of detailed infrared thermal image descriptors, where N is the number of matching pairs.

[0043] Transform the visible light coordinates of the precisely matched point pairs from the ROI local coordinate system back to the original image's global coordinate system. Let the coordinates of the top-left corner of the ROI in the original image be ( Then the visible light global coordinates of the precisely matched point pair are... for:

[0044] Final set of matching points This is the input for step S3.

[0045] Step S3: Based on the corrected set of matching point pairs, calculate the projective transformation matrix from the visible light image to the infrared thermal image coordinate system; use the projective transformation matrix to perform a geometric transformation on the visible light image to obtain a registered visible light image aligned with the viewpoint of the infrared thermal image. Step S3 specifically involves image registration and transformation, based on a set of finely matched point pairs. Solve for the original image of visible light. To infrared thermal image The projective transformation matrix H of the coordinate system. The projective transformation matrix H is a 3×3 homography matrix, and its transformation relationship is as follows:

[0046] in,( , ) represents the pixel coordinates in the visible light image. , Let ) represent the corresponding pixel coordinates in the infrared thermal image, and s be the scale factor for the homogeneous coordinates. Expand the elements of the homography matrix H:

[0047] Each pair of matching points ( , Given two linear equations, N pairs of matching points can construct an overdetermined linear system Ah = 0, where A is a 2N×8 coefficient matrix and h is a vector consisting of the 8 unknown parameters of H. A direct linear transformation algorithm is used, and by performing singular value decomposition on A, the right singular vector corresponding to the smallest singular value is taken as the least-squares solution of h.

[0048] In a preferred embodiment of this application, when the electrical equipment can be approximated as being on the same plane, homography transformation is used for registration. This transformation has eight degrees of freedom and can accurately describe translation, rotation, scaling, and perspective distortion. In another embodiment, if the equipment is distributed on different depth planes, affine transformation (six degrees of freedom) can be used to achieve approximate registration. The affine transformation matrix has a special form of H, where... = = 0.

[0049] Using the projective transformation matrix H, the visible light image is... Perform geometric transformations and resampling. Specifically, for registered visible light images... The location of each target pixel in ( , The original image in visible light is calculated using the inverse matrix of H. The corresponding source coordinates ( , ):

[0050] Due to the calculated source coordinates ( , The value is usually a non-integer value and requires interpolation. When using bilinear interpolation, let ( , The integer part of ) is ( , The decimal part is (). , ) = ( - , - If ), then the interpolation result is:

[0051] The above interpolation is calculated independently for each of the three RGB channels. After transformation and interpolation, the result is obtained that corresponds to the infrared thermal image. Viewpoint aligned registered visible light images RGB color and texture information are fully preserved.

[0052] In this application, a visible light image is registered. The output resolution is determined by the projective transformation matrix H and the visible light original image. The resolution is determined jointly. Specifically, the original visible light image... After the four corner points are transformed by H, the bounding rectangle of the transformed corner points is calculated. The width and height of this bounding rectangle are the natural output resolution of the registered visible light image. Because the resolution of the visible light image is much higher than that of the infrared thermal image, and H contains a significant scaling component, and The actual value depends on the specific parameters of H. The higher the resolution of the registered visible light image, the clearer the edge details of the device, which is beneficial for obtaining higher precision segmentation results in subsequent steps.

[0053] Step S4: Perform instance segmentation on the registered visible light image to obtain the segmentation prediction map corresponding to each power equipment instance; scale all segmentation prediction maps to the same size as the infrared thermal image, and extract the pixels corresponding to the power equipment area as segmentation masks. Step S4 specifically involves: device segmentation for registering the visible light image. This includes segmenting the registered visible light image... Perform instance segmentation to obtain segmentation prediction maps for each of the N1 power device instances. In this model, a pixel value of 1 represents the power equipment area, and a pixel value of 0 represents the background area. The instance segmentation can employ a deep learning-based instance segmentation model, including but not limited to Mask R-CNN, SOLOv2, and YOLACT.

[0054] Registered visible light images retain rich RGB three-channel color and texture information. Electrical equipment (metal components, ceramic insulators, wires, etc.) and the background (sky, vegetation, ground, etc.) have significant color and texture differences in visible light images. The higher the output resolution of the registered visible light image, the clearer the edge details of the equipment, and the more local texture and color information the segmentation model can acquire, resulting in higher segmentation accuracy.

[0055] Segmentation prediction graphs for each power equipment instance (size is) (Scaled to the size of the infrared thermal image using nearest neighbor interpolation, at the same size as the registered visible light image) ), to obtain the scaled prediction map .from Extract the regions with a pixel value of 1 as the segmentation mask for the power equipment. For each pixel location (i,j) in the scaled prediction image, its value is determined by the following formula:

[0056] Since the segmentation mask is a binary image, nearest neighbor interpolation will not introduce grayscale transitions, and the scaled mask boundaries remain clear.

[0057] Final segmentation mask This is a binary image of the same size as the infrared thermogram, where areas with a pixel value of 1 represent power equipment areas and areas with a pixel value of 0 represent background areas. Because this invention precisely aligns the visible light image to the infrared coordinate system through cross-modal registration and fully utilizes the information advantages of the visible light image for segmentation, this mask can be directly applied to subsequent infrared temperature analysis.

[0058] Step S5: Using the segmentation mask of each power equipment instance, extract the temperature data within the segmentation mask area from the temperature matrix, calculate the maximum temperature difference of each power equipment instance, and determine whether each device has a heat generation defect based on the preset temperature difference threshold.

[0059] Step S5 specifically involves temperature matrix analysis and defect determination. For each device instance, a segmented mask is used. Applied to temperature matrix Extract the temperature dataset within the mask area of ​​this instance. :

[0060] in and These represent the height and width of the infrared thermal image, respectively.

[0061] Calculate the temperature difference within the mask area for each device instance. :

[0062] in This represents the highest temperature within the mask area. This represents the lowest temperature within the masked area. Since thermal defects in power equipment typically manifest as point-like or small-area abnormally high temperatures due to poor local contact, insulation aging, etc., This often corresponds to potential defects and hot spots. The difference between the two reflects the temperature level at which the equipment is operating normally. It can directly characterize the severity of defects.

[0063] The temperature difference With respect to the preset temperature difference threshold The comparison is performed, and each device instance is judged separately: .

[0064] The temperature difference threshold The temperature difference criteria for different equipment types can be set according to relevant standards in the power industry. Since this invention has effectively eliminated the interference of background pixels through cross-modal registration and visible light guided segmentation, the temperature data in the mask area can truly reflect the temperature distribution of the equipment body. Therefore, directly using the maximum temperature difference as the criterion has sufficient accuracy and reliability.

[0065] In a preferred embodiment, the registered visible light image can be further used to identify the equipment type (e.g., to determine if the equipment is an insulator, wire joint, transformer bushing, etc.), and then the corresponding temperature difference judgment threshold can be automatically matched according to the equipment type to achieve a more refined defect level judgment.

[0066] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer, characterized in that, The specific steps include: S1. Acquire visible light images and infrared thermal images of the target power equipment and their corresponding temperature matrices; S2. Extract edge features from visible light images and infrared thermal images respectively to obtain visible light edge maps and infrared edge maps; detect key points on the two edge maps respectively and calculate the feature descriptors of the key points, perform cross-modal key point matching, obtain a set of matching point pairs, and use the RANSAC filtering algorithm to remove incorrect matching point pairs; S3. Based on the corrected set of matching point pairs, calculate the projective transformation matrix from the visible light image to the infrared thermal image coordinate system; use the projective transformation matrix to perform a geometric transformation on the visible light image to obtain a registered visible light image aligned with the viewpoint of the infrared thermal image. S4. Perform instance segmentation on the registered visible light image to obtain the segmentation prediction map corresponding to each power equipment instance; scale all segmentation prediction maps to the same size as the infrared thermal image, and extract the pixels corresponding to the power equipment area as segmentation masks. S5. Using the segmentation mask of each power equipment instance, extract the temperature data within the segmentation mask area from the temperature matrix, calculate the maximum temperature difference of each power equipment instance, and determine whether each device has a heat generation defect based on the preset temperature difference threshold.

2. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, The feature descriptor is a phase-consistency-based structural descriptor or a Log-Gabor histogram descriptor.

3. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, The step of detecting key points on two edge maps and calculating feature descriptors for the key points, and performing cross-modal key point matching, is implemented by extracting and matching feature descriptors through a pre-trained cross-modal feature extraction deep learning network. Matching of corresponding key points is achieved based on the matching between feature descriptors.

4. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, The projective transformation matrix is ​​a homography transformation matrix, which is solved based on the set of matching point pairs using a direct linear transformation algorithm or the least squares method.

5. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, When performing geometric transformations on visible light images, bilinear interpolation or bicubic interpolation is used for resampling.

6. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, In steps S2 and S3, a hierarchical registration strategy combining downsampling coarse registration and region of interest fine registration is adopted. This hierarchical registration strategy specifically includes three stages: a downsampling coarse registration stage, a region locking stage, and a fine registration stage. In the downsampling coarse registration stage, the visible light image is downsampled to a resolution similar to that of the infrared thermal image. Edge detection, key point matching, and RANSAC filtering are then performed between the downsampled visible light image and the infrared thermal image to obtain a coarse transformation matrix. During the region locking stage, the field of view of the infrared thermal image is back-projected onto the visible light original image coordinate system using the coarse transformation matrix, and the corresponding region of interest sub-image is cropped out. In the fine registration stage, edge detection, key point matching, and RANSAC filtering are performed again between the region of interest sub-image and the infrared thermal image to obtain the fine transformation matrix. The local coordinates are then converted into global coordinates and used to solve the projective transformation matrix.

7. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, The instance segmentation employs a deep learning-based instance segmentation model, including Mask R-CNN, SOLOv2, and YOLACT.

8. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, In the step of performing instance segmentation on the registered visible light image to obtain the segmentation prediction map corresponding to each power equipment instance, nearest neighbor interpolation is used to scale the segmentation prediction map to the same size as the infrared thermal image.

9. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, The specific steps of extracting temperature data within the segmented mask region from the temperature matrix using the segmentation mask of each power equipment instance, calculating the maximum temperature difference of each power equipment instance, and determining whether each device has a heat generation defect based on a preset temperature difference threshold are as follows: The segmentation mask of each power equipment instance is applied to the temperature matrix, and the temperature values ​​corresponding to all pixels in each mask area are extracted to form a temperature dataset. For each device instance, calculate the difference between the highest and lowest temperature values ​​in the temperature dataset, which is taken as the maximum temperature difference for that device instance. The maximum temperature difference is compared with the temperature difference threshold set according to the power industry standard. If the maximum temperature difference is greater than or equal to the temperature difference threshold, the device is determined to have a heating defect.

10. The method for diagnosing infrared thermal defects in power equipment based on cross-modal visual information transfer according to claim 1, characterized in that, It also includes step S6, which, after determining whether each device has a heat-generating defect, uses the registered visible light image to identify the type of power equipment, matches the corresponding temperature difference judgment threshold according to the equipment type, and realizes the classification of heat-generating defect level.

Images