Power equipment infrared and visible light image registration method, system, device and medium

Abstract

The application discloses a kind of electric power equipment infrared and visible light image registration method, system, equipment and medium, belong to electric power system field, the method is: in electric power equipment infrared and visible light image registration, first based on SuperPoint network respectively extracts infrared, visible light feature point;Again input LightGlue, OmniGlue network obtains two kinds of matching pair set;According to the light intensity of visible light, the weight factor of LightGlue matching pair set is calculated, the confidence threshold of two kinds of matching pair set is dynamically calculated;Threshold filtering and merging obtain initial matching pair set, and the target matching pair set is screened by improved RANSAC algorithm, and the visible light image is transformed to realize registration by calculating affine transformation matrix and transforming.The therefore, by implementing the application, it can realize the high-precision registration of electric power equipment image under the difference of extreme illumination and multi-modal.

Description

Technical Field

[0001] This invention relates to the field of power systems, and more particularly to a method, system, device, and medium for registering infrared and visible light images of power equipment. Background Technology

[0002] In the intelligent inspection of power equipment in substations, drones are usually used to carry photoelectric imaging systems equipped with both infrared and visible light acquisition devices to acquire images of substation equipment. At the same time, the infrared and visible light images acquired by the same device often have different scaling ratios and relative translation and rotation issues. Therefore, the two images need to be registered before subsequent detection, segmentation, temperature measurement and other tasks can be carried out.

[0003] Currently, common image registration methods include grayscale or template-based methods, domain transformation-based methods, and feature-based methods. Feature-based methods belong to the field of artificial intelligence and typically involve four steps: feature extraction, feature matching, calculation of the transformation matrix, and image transformation. Due to nonlinear spectral radiation variations, infrared and visible light images often exhibit varying degrees of grayscale differences, resulting in low similarity between corresponding locations, more mismatches than correct matches, making it difficult to achieve accurate and effective matching using traditional feature extraction and matching methods. Furthermore, infrared images typically have low resolution and blurred edges, and their texture information is less rich than that of visible light images. Especially when images are captured in extreme environments such as excessively dim or intense light, the resulting visible light images often exhibit overexposure or underexposure, and the edge information of electrical equipment and components in infrared images may also be blurred due to thermal effects, making them difficult to distinguish from the background. This further hinders the correct matching of feature points in infrared images with those in visible light images. Summary of the Invention

[0004] This invention provides a method, system, device, and medium for registering infrared and visible light images of power equipment, which can solve the problems of low matching reliability caused by inherent differences in multimodal images and low registration accuracy caused by extreme lighting conditions exacerbating registration difficulties.

[0005] This invention provides a method for registering infrared and visible light images of power equipment, comprising: Feature points are extracted from infrared and visible light images using the SuperPoint network to obtain infrared and visible light feature points respectively. The infrared feature points and the visible light feature points are input into the LightGlue network for matching to obtain a LightGlue matching pair set, and the infrared feature points and the visible light feature points are input into the OmniGlue network to obtain an OmniGlue matching pair set; Based on the weight factors of the LightGlue matching pair set, the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set are dynamically calculated respectively; wherein, the weight factors of the LightGlue matching pair set are calculated based on the illumination intensity of the visible light image; The LightGlue matching pair set is filtered based on the LightGlue confidence threshold, and the OmniGlue matching pair set is filtered based on the OmniGlue confidence threshold and then merged to obtain the initial matching pair set. The RANSAC algorithm is used to filter the initial matching pair set to obtain the target matching pair set. An affine transformation matrix is ​​calculated based on the target matching pair set, and the visible light image is transformed based on the affine transformation matrix to achieve the registration of the infrared image to be registered with the visible light image to be registered.

[0006] This invention employs the SuperPoint network to accurately extract feature points from infrared and visible light images. Simultaneously, it utilizes a dual-network matching approach with LightGlue (high reliability) and OmniGlue (high quantity), avoiding the problems of insufficient matching points or numerous false matches associated with a single network. This provides a sufficient and high-quality initial matching pair foundation for subsequent registration. Illumination scores are calculated based on the light intensity of the visible light image. LightGlue weight factors are dynamically generated through nonlinear mapping, and the confidence thresholds of the two types of matching pairs are adjusted accordingly. This allows the matching strategy to adaptively optimize according to illumination conditions (normal / abnormal)—prioritizing the retention of high-reliability LightGlue matching pairs under extreme illumination, while considering the quantity of OmniGlue matching points under normal illumination. This significantly reduces the false match rate caused by extreme illumination, achieving adaptive illumination adjustment and solving the problem of unreliable matching under extreme illumination. The quality of the initial matching pairs is optimized by dynamic threshold filtering, and then high-precision target matching pairs are obtained by RANSAC algorithm. The affine transformation matrix calculated accordingly can more accurately realize the spatial alignment of infrared and visible light images, and ultimately improve the accuracy of multimodal image registration of power equipment and the robustness under complex lighting scenarios, thereby improving registration accuracy and robustness.

[0007] Furthermore, the step of dynamically calculating the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set based on the weight factors of the LightGlue matching pair set is specifically as follows: An illumination score is calculated based on the illumination intensity of the visible light image, and a weighting factor is generated based on the illumination score through a nonlinear mapping function. Obtain the initial LightGlue confidence threshold and the LightGlue weight factor adjustment coefficient. Adjust the initial LightGlue confidence threshold according to the LightGlue weight factor adjustment coefficient and the weight factor to obtain the LightGlue confidence threshold. Obtain the initial OmniGlue confidence threshold and the OmniGlue weight factor adjustment coefficient. Adjust the initial OmniGlue confidence threshold according to the OmniGlue weight factor adjustment coefficient and the weight factor to obtain the OmniGlue confidence threshold.

[0008] This adjustment logic, consisting of an initial threshold, adjustment coefficient, and weighting factor, allows the confidence thresholds of LightGlue and OmniGlue to dynamically change with the lighting score. When lighting is abnormal, the LightGlue threshold decreases (retaining more high-reliability matching points), while the OmniGlue threshold increases (eliminating more low-reliability false matches). When lighting is normal, the thresholds return to a reasonable range, balancing the quantity and quality of matching points. This two-step method—"obtaining the initial threshold → adjusting it using the adjustment coefficient and weighting factor"—avoids complex threshold optimization models. It allows those skilled in the art to quickly achieve adaptive threshold adjustment by experimentally setting the initial threshold (e.g., the optimal threshold under good lighting) and adjustment coefficient, simplifying the threshold adjustment logic and improving feasibility.

[0009] Furthermore, the calculation of the illumination score based on the illumination intensity of the visible light image is specifically as follows: The visible light image is converted from the RGB color space to the HSV color space to obtain the luminance component; The initial average brightness of the visible light image is calculated based on the brightness components, and the initial average brightness is normalized to obtain the average brightness. The standard deviation of the brightness of the visible light image is calculated based on the average brightness value. The proportion of the first brightness region is obtained by counting the number of pixels in the visible light image whose brightness value is greater than the first brightness threshold. The proportion of the second brightness region is obtained by counting the number of pixels in the visible light image whose brightness value is less than the second brightness threshold. The first brightness threshold is greater than the second brightness threshold. The illumination score is calculated based on the mean brightness, the standard deviation of brightness, the proportion of the first brightness region, and the proportion of the second brightness region.

[0010] This method extracts independent luminance components (V channels) through HSV color space conversion, and then calculates the illumination score from four dimensions: "overall luminance (mean), luminance distribution uniformity (standard deviation), and the proportion of extreme luminance areas (ratio of first / second luminance areas)." Compared to a single luminance mean, this method can more comprehensively identify extreme lighting scenes such as overexposure and underexposure (e.g., a small standard deviation indicates low contrast, and a high proportion of overexposure / underexposure areas indicates severe extreme lighting), achieving a multi-dimensional representation of the lighting state and avoiding misjudgment based on a single indicator. Through normalization processing (e.g., normalizing the luminance mean to [0,1]), visible light images of different resolutions and luminance ranges have a unified illumination score standard, ensuring that the weighting factors and confidence thresholds generated based on the illumination score can accurately adapt to actual lighting conditions, avoiding matching strategy deviations caused by inaccurate illumination quantization.

[0011] Furthermore, the average brightness value is specifically: ; in, This represents the total number of pixels in the image. For the first The brightness value of each pixel, with the value ranging from 0 to a certain value. Between, divided by This will normalize the result to the [0,1] interval; The standard deviation of the brightness is: ; in, Used to characterize the discreteness of the brightness distribution of the visible light image and reflect the global contrast; The proportion of the first brightness region is: ; in, (⋅) is an indicator function; its value is 1 when its internal condition is true, and 0 otherwise. This indicates that the brightness value is greater than the first brightness threshold. And less than the brightness value The number of pixels; The proportion of the second brightness region is: ; in, This indicates that the brightness value is less than the second preset threshold. And the number of pixels with values ​​greater than zero; The illumination score is: .

[0012] This lighting score formula highlights the core impact of overexposure / underexposure on contrast reduction in matching, while also taking into account the proportion of extreme areas and overall brightness. This ensures that the lighting score is highly correlated with the degree of interference of actual lighting on matching, providing precise numerical support for dynamic adjustments.

[0013] Furthermore, the LightGlue confidence threshold is specifically as follows: ; in, The initial LightGlue confidence threshold, , This is the adjustment coefficient for the LightGlue weighting factor. ; It is a weighting factor, when When the value is 0.5, the illumination is normal. ;when When the value is 1, the lighting is abnormal. , that is, all LightGlue matching points are retained; The confidence threshold for OmniGlue is: ; in, Initial OmniGlue confidence threshold, ; The OmniGlue weighting factor adjustment coefficient. ,when When the value is 0.5, the illumination is normal. The initial OmniGlue confidence threshold; when When the value is 1, the lighting is abnormal. .

[0014] This ensures a continuous linear relationship between the weighting factor and the threshold, avoiding quality fluctuations in matching pairs caused by sudden threshold changes. The experimental values ​​for the initial threshold and adjustment coefficient are clearly defined. These parameters have been verified through power equipment image scenarios and can accurately adapt to matching requirements under extreme lighting conditions, filtering out more than 90% of OmniGlue mismatches.

[0015] Furthermore, the step of using the RANSAC algorithm to filter the initial set of matching pairs to obtain the target set of matching pairs specifically involves: The fusion attention weights for each matching pair are determined and normalized to obtain the probability distribution of the initial matching pair set; In each iteration of the RANSAC algorithm, a preset number of matching pairs are randomly sampled and estimated according to the cumulative probability distribution function corresponding to the probability distribution to obtain the homography transformation matrix of the current iteration. Matching pairs that do not meet the preset conditions in the initial matching pair set are then removed according to the homography transformation matrix to obtain the target matching pair set.

[0016] This approach, by fusing attention weights and normalizing the probability distribution, and then sampling based on the cumulative distribution function (CDF), significantly increases the probability of selecting high-reliability matching pairs (those with large fused attention weights). This avoids the problem of "frequent selection of low-reliability points and numerous invalid iterations" caused by traditional RANSAC random sampling, reducing the number of iterations and improving screening efficiency. Based on the homography transformation matrix obtained from sampling, outliers that do not meet the requirement of "coordinate difference after transformation less than a threshold" are removed. Combined with the high-reliability initial sampling points, this allows for more accurate screening of target matching pairs that meet geometric constraints, reducing the interference of mismatches on subsequent affine transformation matrix calculations and further improving registration accuracy.

[0017] Furthermore, the fusion attention weights are calculated based on the feature space attention weights and the spatial distribution attention weights, specifically as follows: For each matching pair in the initial matching pair set, the feature cosine similarity is calculated; Based on the feature similarity between each feature point in the visible light image and each nearest neighbor feature point in the set of k nearest neighbor feature points, the similarity variance is calculated to obtain the local consistency weight. The feature space attention weight is calculated based on the feature cosine similarity and the local consistency weight. The spatial density heatmap is obtained by traversing each feature point in the visible light image, and the spatial heatmap is then Gaussian smoothed and normalized to obtain the spatial attention weights. The fusion attention weight is calculated based on the feature space attention weight and the spatial distribution attention weight.

[0018] This approach evaluates the feature reliability of matching pairs by combining "feature cosine similarity (descriptor matching degree) + local consistency weight (nearest neighbor similarity variance)," avoiding misjudgments caused by single descriptor similarity (e.g., matching pairs in repetitive texture regions are downweighted due to low local consistency), ensuring that the weights reflect the matching quality at the feature level. The spatial density heatmap (Gaussian smoothing + normalization) evaluates the positional reliability of matching pairs. Matching pairs in high-density regions (concentrated matching points) are weighted higher due to high redundancy and strong reliability, while those in low-density regions (isolated matching points) are downweighted due to susceptibility to noise interference, thus supplementing the evaluation deficiencies of the feature dimension from a spatial perspective. The fusion of feature space and spatial distribution attention weights highlights the core influence of feature matching while also considering the auxiliary judgment of spatial distribution, enabling the fused attention weights to comprehensively and accurately characterize the reliability of matching pairs and providing precise weight support for RANSAC sampling.

[0019] Another embodiment of the present invention provides an infrared and visible light image registration system for power equipment, comprising: a feature extraction module, a feature matching module, a confidence calculation module, a confidence filtering module, and an image registration module; The feature extraction module is used to extract feature points from infrared images and visible light images based on the SuperPoint network, respectively, to obtain infrared feature points and visible light feature points. The feature matching module is used to input the infrared feature points and the visible light feature points into the LightGlue network for matching to obtain a LightGlue matching pair set, and input the infrared feature points and the visible light feature points into the OmniGlue network to obtain an OmniGlue matching pair set; The confidence calculation module is used to dynamically calculate the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set based on the weight factors of the LightGlue matching pair set; wherein, the weight factors of the LightGlue matching pair set are calculated based on the illumination intensity of the visible light image; The confidence filtering module is used to filter the LightGlue matching pair set based on the LightGlue confidence threshold, and to filter and merge the OmniGlue matching pair set based on the OmniGlue confidence threshold to obtain an initial matching pair set. The image registration module is used to filter the initial matching pair set using the RANSAC algorithm to obtain a target matching pair set, calculate an affine transformation matrix based on the target matching pair set, and transform the visible light image based on the affine transformation matrix to achieve registration between the infrared image to be registered and the visible light image to be registered.

[0020] This invention employs the SuperPoint network to accurately extract feature points from infrared and visible light images. Simultaneously, it utilizes a dual-network matching approach with LightGlue (high reliability) and OmniGlue (high quantity), avoiding the problems of insufficient matching points or numerous false matches associated with a single network. This provides a sufficient and high-quality initial matching pair foundation for subsequent registration. Illumination scores are calculated based on the light intensity of the visible light image. LightGlue weight factors are dynamically generated through nonlinear mapping, and the confidence thresholds of the two types of matching pairs are adjusted accordingly. This allows the matching strategy to adaptively optimize according to illumination conditions (normal / abnormal)—prioritizing the retention of high-reliability LightGlue matching pairs under extreme illumination, while considering the quantity of OmniGlue matching points under normal illumination. This significantly reduces the false match rate caused by extreme illumination, achieving adaptive illumination adjustment and solving the problem of unreliable matching under extreme illumination. The quality of the initial matching pairs is optimized by dynamic threshold filtering, and then high-precision target matching pairs are obtained by RANSAC algorithm. The affine transformation matrix calculated accordingly can more accurately realize the spatial alignment of infrared and visible light images, and ultimately improve the accuracy of multimodal image registration of power equipment and the robustness under complex lighting scenarios, thereby improving registration accuracy and robustness.

[0021] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the infrared and visible light image registration method for power equipment of the present invention.

[0022] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the power equipment infrared and visible light image registration method of the present invention. Attached Figure Description

[0023] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic flowchart of an infrared and visible light image registration method for power equipment provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a registration result provided by an embodiment of the present invention; Figure 3This is a schematic diagram of the structure of an infrared and visible light image registration system for power equipment provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0027] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0030] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0031] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0032] See Figure 1 To address the problems of low matching reliability due to inherent differences in multimodal images and low registration accuracy caused by registration difficulties exacerbated by extreme lighting conditions in existing technologies, an embodiment of the present invention provides a method for registering infrared and visible light images of power equipment, comprising steps S101-S105, specifically: Among them, SuperPoint network, namely self-supervised feature point detection network, is a feature point detection and description network based on self-supervised learning; LightGlue network, namely lightweight feature matching network, is a lightweight feature point matching network; OmniGlue network, namely general feature matching network, is a feature matching network suitable for multimodal images; and MagicPoint network, namely feature point detection and description network.

[0033] Step S101: Extract feature points from infrared and visible light images respectively based on the SuperPoint network to obtain infrared feature points and visible light feature points.

[0034] In this embodiment, the SuperPoint network is trained to extract feature points from two infrared and visible light images to be registered. Specifically, the infrared and visible light image training datasets, obtained by randomly generating basic geometric shapes, are used for feature extraction training to obtain the MagicPoint feature detector network as a pre-trained model. The MagicPoint feature detector network is then used to automatically label the constructed infrared and visible light image training datasets, obtaining pseudo-real labels for the feature points in each dataset. The MagicPoint detector network is then trained again using the infrared and visible light image training datasets and the obtained pseudo-real labels for the feature points. This training process is iterated two or more times to obtain the final pseudo-real labels for the feature points in the infrared and visible light images and the MagicPoint detector network. The SuperPoint feature detector network is then trained using the final pseudo-real labels for the feature points in the infrared and visible light images and the MagicPoint detector network to obtain a SuperPoint model suitable for infrared and visible light images. Finally, the SuperPoint model is used to extract feature points from the infrared and visible light images to be registered.

[0035] Step S102: Input the infrared feature points and the visible light feature points into the LightGlue network for matching to obtain a LightGlue matching pair set, and input the infrared feature points and the visible light feature points into the OmniGlue network to obtain an OmniGlue matching pair set.

[0036] In this embodiment, the extracted two types of feature points are input into the LightGlue network for coarse matching to obtain LightGlue matching pairs; OmniGlue coarse matching is then performed on the extracted two types of feature points to obtain OmniGlue matching pairs. Specifically, a lightweight LightGlue network structure and a pre-trained model are used to perform feature matching on the input infrared feature points and visible light feature points, and the network output is the LightGlue matching pair. Since the OmniGlue network model structure and parameter count are large, to prevent memory overflow during the feature matching process, the large-sized visible light image is scaled. For example, for an NVIDIA GeForce GTX 1080Ti 11GB graphics card, the maximum input is an image of 960*960 pixels. The infrared image to be registered and the scaled visible light image are input into the OmniGlue model to obtain the matching pair coordinates of the infrared image and the visible light image, respectively. The matching pair coordinates of the visible light image are divided by the scaling factor to obtain the OmniGlue matching point coordinates at the original visible light size.

[0037] Step S103: Based on the weight factors of the LightGlue matching pair set, dynamically calculate the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set, respectively; wherein, the weight factors of the LightGlue matching pair set are calculated based on the illumination intensity of the visible light image.

[0038] In this embodiment, the illumination intensity of the visible light image is quantified to construct a comprehensive illumination score; a nonlinear dynamic mapping relationship between illumination intensity and feature matching weight factors is established, and dynamic weight factors are obtained based on the comprehensive illumination score; based on the dynamic weight factors, the confidence thresholds of the two types of matching pairs are calculated.

[0039] Step S104: Filter the LightGlue matching pair set based on the LightGlue confidence threshold, and filter the OmniGlue matching pair set based on the OmniGlue confidence threshold, then merge them to obtain the initial matching pair set.

[0040] In this embodiment, the two types of matching pairs are filtered based on their respective confidence thresholds, and the filtered matching pairs are then vertically stacked row by row to obtain a merged initial set of matching pairs. Specifically, the LightGlue matching pairs and OmniGlue matching pairs are filtered separately using different confidence thresholds obtained in the previous step. Let the filtered LightGlue matching pair set be... The filtered OmniGlue matching pair set is The initial set of matched pairs after merging is: This can be represented in matrix form, i.e.: ; in, Represent a The set of LightGlue matrix matching pairs, Represent a The set of OmniGlue matching pairs, This indicates that a vertical stacking operation is performed row-wise in the 0th dimension, ultimately resulting in the merged initial set of matching pairs. .

[0041] Step S105: The initial matching pair set is filtered using the RANSAC algorithm to obtain the target matching pair set. An affine transformation matrix is ​​calculated based on the target matching pair set, and the visible light image is transformed based on the affine transformation matrix to achieve the registration of the infrared image to be registered with the visible light image to be registered.

[0042] In this embodiment, fusion attention weights are generated for the merged matching pairs. The RANSAC algorithm for fusion attention weights is used iteratively to filter the initial set of merged matching pairs to obtain refined matching pairs, i.e., the target set of matching pairs. The affine transformation matrix parameters are calculated based on the refined matching pairs. The specific steps are as follows: Affine transformation matrix The general form is: ; in, and They are respectively shaft and Scale parameters along the axis. For rotation parameters, and They are respectively shaft and Translation parameters along the axis. When used in matrix operations, they are generally converted to the following form: ; Let the set of exact matching pairs in infrared and visible light images be represented in matrix form as follows: and Both are The matrix, where This represents the number of matching pairs.

[0043] Let the pair of interior points that have a matching relationship be respectively and Then, a system of linear equations can be constructed based on this pair of interior points: ; ; Input the coordinates of all perfectly matched point pairs and transform the system of linear equations into matrix form. The affine transformation matrix can then be obtained. .

[0044] An affine transformation matrix is ​​used to transform a visible light image to the infrared image coordinate system. Specifically, the coordinates of the input visible light image are multiplied by the affine transformation matrix to obtain the position of the visible light image in the infrared image coordinate system. Merging the two achieves the registration of the infrared and visible light images. Figure 2 The figure shown is a schematic diagram of a registration result provided by an embodiment of the present invention.

[0045] It should be noted that LightGlue's matching method is based on geometric constraints, making it more robust to changes in lighting. In environments with abnormal lighting, LightGlue obtains more reliable feature matching point pairs. On the other hand, OmniGlue's matching method is based on appearance features, making it more susceptible to interference from lighting factors. In environments with abnormal lighting, OmniGlue obtains feature matching point pairs with a higher error rate.

[0046] As an example of an embodiment of the present invention, the step of dynamically calculating the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set based on the weight factors of the LightGlue matching pair set specifically involves: calculating an illumination score based on the illumination intensity of the visible light image, and generating the weight factors based on the illumination score through a nonlinear mapping function; obtaining an initial LightGlue confidence threshold and a LightGlue weight factor adjustment coefficient, and adjusting the initial LightGlue confidence threshold according to the LightGlue weight factor adjustment coefficient and the weight factors to obtain the LightGlue confidence threshold; obtaining an initial OmniGlue confidence threshold and an OmniGlue weight factor adjustment coefficient, and adjusting the initial OmniGlue confidence threshold according to the OmniGlue weight factor adjustment coefficient and the weight factors to obtain the OmniGlue confidence threshold.

[0047] In this embodiment, during multimodal image registration, the lighting conditions in the image acquisition environment are directly reflected in the light intensity of the visible light image, and these conditions also directly affect the edge sharpness of the infrared image. When the ambient light is too strong, overexposure is likely to occur in the visible light image, while the infrared image often exhibits target edge thermal diffusion due to the thermal saturation effect of the infrared sensor. When the ambient light is insufficient, underexposure is common in the visible light image, while the infrared image often suffers from background noise interference due to low thermal contrast. The significant differences in feature point descriptor characteristics between the visible light and infrared images make them prone to mismatches, affecting subsequent tasks. To differentiate between different lighting conditions, a light intensity quantization module is constructed based on the visible light image to calculate a light score to assess whether extreme lighting conditions occur. Based on the calculated comprehensive light score, a weighting factor is set for the LightGlue matching point pairs to dynamically adjust their proportion under different lighting conditions. For example, an improved sigmoid function is used to construct a weighting factor based on the comprehensive light score. Weighting factors for LightGlue The nonlinear mapping relationship is used to obtain the weight factors corresponding to the LightGlue matching pair set. Based on the dynamic weight factors, the confidence thresholds for the two types of matching pairs are calculated: when the lighting conditions are abnormal, the confidence threshold for LightGlue should be decreased, and the confidence threshold for OmniGlue should be increased; while when the lighting conditions are normal, the threshold can be appropriately decreased to retain more matching points. Specifically, the weight factors obtained in the previous step... Subsequently, a dynamic relationship between the weighting factor and the confidence threshold was established: in the process of calculating the confidence threshold of LightGlue matching point pairs, the basic confidence threshold of LightGlue under good lighting conditions was used as the initial LightGlue confidence threshold, and empirical values ​​were taken based on experiments. And obtain the LightGlue weighting factor adjustment coefficient, and take the empirical value based on the experiment. When weight factor When the value is 0.5, the illumination is normal. ;when When the value is 1, the light intensity is severely abnormal. This means that all LightGlue matching points are preserved. The baseline confidence threshold for OmniGlue under good lighting conditions is used as the initial OmniGlue confidence threshold, with empirical values ​​determined based on experiments. , This is the weighting factor adjustment coefficient, with empirical values ​​taken from experiments. .when When the value is 0.5, the illumination is normal. Based on confidence threshold; when When the value is 1, the light intensity is severely abnormal. At this point, most OmniGlue matching point pairs are filtered out, and only a portion of point pairs with higher confidence are retained.

[0048] As an example of an embodiment of the present invention, the LightGlue confidence threshold is specifically as follows: ; in, The initial LightGlue confidence threshold, , This is the adjustment coefficient for the LightGlue weighting factor. ;when When the value is 0.5, the illumination is normal. ;when When the value is 1, the lighting is abnormal. , that is, all LightGlue matching points are retained; The confidence threshold for OmniGlue is: ; in, Initial OmniGlue confidence threshold, ; The OmniGlue weighting factor adjustment coefficient. ,when When the value is 0.5, the illumination is normal. The initial OmniGlue confidence threshold; when When the value is 1, the lighting is abnormal. .

[0049] As an example of an embodiment of the present invention, the calculation of the illumination score based on the illumination intensity of the visible light image specifically involves: The visible light image is converted from the RGB color space to the HSV color space to obtain a luminance component. An initial mean luminance of the visible light image is calculated based on the luminance component, and the initial mean luminance is normalized to obtain a luminance mean. The standard deviation of the luminance of the visible light image is calculated based on the mean luminance. The number of pixels in the visible light image with luminance values ​​greater than a first luminance threshold is counted to obtain a first luminance region proportion, and the number of pixels in the visible light image with luminance values ​​less than a second luminance threshold is counted to obtain a second luminance region proportion, wherein the first luminance threshold is greater than the second luminance threshold. The illumination score is calculated based on the mean luminance, the standard deviation luminance, the first luminance region proportion, and the second luminance region proportion.

[0050] As an example of an embodiment of the present invention, the average brightness value is specifically: ; in, This represents the total number of pixels in the image. For the first The brightness value of each pixel, with the value ranging from 0 to a certain value. Between, divided by This will normalize the result to the [0,1] interval; The standard deviation of the brightness is: ; in, Used to characterize the discreteness of the brightness distribution of the visible light image and reflect the global contrast; The proportion of the first brightness region is: ; in, (⋅) is an indicator function; its value is 1 when its internal condition is true, and 0 otherwise. This indicates that the brightness value is greater than the first brightness threshold. And less than the brightness value The number of pixels; The proportion of the second brightness region is: ; in, This indicates that the brightness value is less than the second preset threshold. And the number of pixels with values ​​greater than zero; The illumination score is as follows: .

[0051] In this embodiment, since the input visible light image is usually an RGB image, the HSV color space decoupling function is first used. Convert the visible light image from the RGB color space to the HSV color space and analyze the visible light image. Brightness and color information: ; Among them, components Each channel independently represents the brightness information of the image.

[0052] According to the luminance component The average brightness of the image can be calculated. And perform normalization: ; in, This represents the total number of pixels in the image. For the first The brightness value of each pixel (ranging from 0 to 255) is normalized to the [0,1] interval by dividing by 255. The average brightness value indicates whether the overall image is too bright or too dark.

[0053] According to the luminance component Calculate the standard deviation of the image's brightness. : ; It can show the dispersion of image brightness distribution and reflect the global contrast of the image. The smaller the value, the lower the contrast, and the more severe the overexposure or underexposure.

[0054] Calculate the number and proportion of pixels under extreme lighting conditions. Define overexposed areas as the set of pixels with a brightness value greater than 220, and underexposed areas as the set of pixels with a brightness value less than 35. The proportion of overexposed areas... and underexposed area ratio Defined as the proportion of the corresponding region to the total number of pixels in the image, calculated as follows: , ; in (⋅) is an indicator function. Its value is 1 when its internal condition is true, and 0 otherwise. This indicates the number of pixels with a brightness value greater than 220. This indicates the number of pixels with a brightness value less than 35. Both 220 and 35 are empirical thresholds obtained from experiments on visible light images of outdoor substation power equipment under various lighting conditions, and can be adjusted appropriately according to actual circumstances.

[0055] Based on the above four indicators, the comprehensive illumination score is calculated. : ; in, The larger the value, the lower the contrast, and the higher the likelihood of overexposure / underexposure; The term represents the sum of the proportions of overexposed and underexposed areas to the total number of pixels in the image, i.e., the total proportion of pixels under extreme lighting conditions; when the average brightness... When the value is close to 0 (all black) or 1 (all white), The larger values ​​reflect extreme brightness. The weighting coefficients (0.5, 0.3, and 0.2) are empirical values ​​obtained from experiments. Experiments show that this comprehensive score has a high correlation with actual lighting conditions and is highly reliable.

[0056] Overall Light Score The higher the value, the more severe the abnormal lighting conditions. Based on this, a lighting intensity grading standard can be established: when... When the image lighting conditions are normal, it is considered that the image is under normal illumination. At that time, it was considered that the image showed a moderate illumination anomaly; when At that time, it was believed that the image showed severe lighting abnormalities.

[0057] A nonlinear dynamic mapping relationship between illumination intensity and feature matching weight factors is established. Specifically, based on the comprehensive illumination score calculated in the previous step, weight factors are set for LightGlue matching point pairs to dynamically adjust their proportion under different illumination conditions. An improved sigmoid function is used to construct a mapping relationship from the comprehensive illumination score... Weighting factors for LightGlue Nonlinear mapping relationship: ; The constant value of 0.5 ensures that the function's range is [0.5, 1.0], preventing LightGlue weights from becoming too low. (Parameters) Control the steepness of the function curve. Controls the center point of the function's response. When When increasing, the exponent term in the denominator Decrease, thus making The value increases. This function has the characteristics of smoothness, monotonicity, and continuity, which can continuously map the comprehensive illumination score to the weights, avoiding abrupt changes in the weights and thus ensuring the stability of the matching results.

[0058] For example, when That is, under normal lighting conditions, The value is 0.5; when hour, A value of 0.75 indicates that when the light score reaches 0.4, the LightGlue weight is 75%, achieving the goal of rapidly increasing the weight value from 0.5 to around 1 under moderately abnormal lighting conditions; when That is, when there is severe abnormality in light intensity, A value of 1 significantly increases the weight of LightGlue.

[0059] By using the above method, the weights of the matching algorithm are dynamically adjusted according to the light intensity score. This ensures the reliability of the matching points while guaranteeing the number of matching points, which is beneficial to improving the accuracy of subsequent transformations.

[0060] As an example of an embodiment of the present invention, the step of using the RANSAC algorithm to filter the initial set of matching pairs to obtain the target set of matching pairs specifically involves: The fusion attention weights for each matching pair are determined and normalized to obtain the probability distribution of the initial matching pair set. In each iteration of the RANSAC algorithm, a preset number of matching pairs are randomly sampled according to the cumulative probability distribution function corresponding to the probability distribution to estimate the homography transformation matrix of the current iteration. Matching pairs that do not meet the preset conditions in the initial matching pair set are removed according to the homography transformation matrix to obtain the target matching pair set.

[0061] In this embodiment, the merged matching pairs have high confidence, but some background points or incorrect matching pairs may still exist. To further filter the merged matching pairs and establish a new unified evaluation method for matching points, feature space attention weights and spatial distribution attention weights are generated for each matching pair, ultimately forming fused attention weights. Unlike traditional uniform random sampling, this method uses probabilistic sampling based on fused weights, that is, iteratively using the RANSAC algorithm of fused attention weights to filter the merged matching pair set to obtain refined matching pairs. Specifically, the fused attention weight vectors of each matching point pair are first normalized to make them a probability distribution: ; in, This represents the probability of selecting a matching point pair.

[0062] To perform weighted random sampling, a cumulative distribution function (CDF) for the normalized weight vector is constructed: ; in Indicates the preceding The cumulative probability of a pair of matching points, such as This probabilistic sampling mechanism based on dynamic weights ensures that high-weight points (such as matching point pairs with a fusion attention weight close to 1) have a higher sampling probability, while low-weight points (such as matching point pairs with a fusion attention weight less than 0.3) have a lower sampling probability. In this way, highly reliable matching point pairs are more likely to be selected for the calculation of the homography transformation matrix.

[0063] In each iteration of RANSAC, four points are sampled according to the above probability distribution to estimate the homography transformation matrix. Specifically, in each RANSAC iteration, sampling is required. A pair of matching points (for the calculation of the homography transformation matrix, usually...) First, generate 1 independent, uniformly and identically distributed random number For each random number Find the corresponding matching point pair index through CDF. ,satisfy The sampled 1 matching point pair (index is ) This is used to compute the candidate homography transformation matrix for the current iteration. Secondly, the obtained homography transformation matrix is ​​used. The remaining feature points are transformed to find the correct matching pairs, specifically: a pair of matching feature point coordinates. and In the middle, one of the points Perform a transformation to obtain the new coordinate points. ,calculate and Differences between ,like Less than the threshold If so, it is considered to satisfy Treat each point as an interior point and store the matching pair in the interior point set. In the middle; if Greater than the threshold If it is not satisfied, then it is considered not to meet the requirements. This pair of matching points is considered an exterior point, and the exterior points are discarded. Finally, the interior point set is calculated. number of inner points Repeat the first two steps within the set number of iterations. The result is obtained by comparison, showing the maximum value. set of values This is the set of target matching pairs required.

[0064] As an example of an embodiment of the present invention, the fused attention weight is calculated based on the feature space attention weight and the spatial distribution attention weight, specifically as follows: For each matching pair in the initial matching pair set, the feature cosine similarity is calculated; based on the feature similarity between each feature point in the visible light image and each nearest neighbor feature point in the k-nearest neighbor feature point set, the similarity variance is calculated to obtain the local consistency weight; the feature spatial attention weight is calculated based on the feature cosine similarity and the local consistency weight; each feature point in the visible light image is traversed to obtain a spatial density heatmap, and the spatial heatmap density is Gaussian smoothed and normalized to obtain the spatial attention weight; the fusion attention weight is calculated based on the feature spatial attention weight and the spatial distribution attention weight.

[0065] In this embodiment, during the calculation of the attention weights in the feature space, for each pair of matches... and ,in These are feature points in a visible light image. For feature points in an infrared image, the feature cosine similarity is first calculated. The calculation formula is as follows: ; in, and They are respectively and The corresponding feature descriptor. It reflects the degree of matching of the feature points of the matching pair in the feature structure of the description subspace. The higher the value, the higher the degree of matching and the stronger the reliability.

[0066] And calculate local feature consistency; specifically, in the image feature space, find... of Neighbors ( Feature point set ,calculate The feature similarity between a feature and each of its nearest neighbor feature points is calculated using the following formula: ; The similarity variance of all nearest neighbors is calculated based on the consistency of local features. The local consistency weight is obtained, and the calculation formula is as follows: ; in, It reflects the degree of similarity in the distribution of feature points in the vicinity. The larger the value, the more likely the feature is located in an area of ​​approximately repeating texture, resulting in lower matching reliability and a greater likelihood of confusion.

[0067] Finally, based on the obtained feature cosine similarity and local feature consistency weights, the feature space attention weights are calculated by weighted summation, and the calculation formula is expressed as follows: ; in =0.7, =0.3 is the weighted empirical coefficient.

[0068] In calculating spatial distribution attention weights, a spatial density heatmap is created for the image. This heatmap displays the spatial distribution of matching points on the image. High-density areas have high matching redundancy and more reliable results, while low-density areas have weaker matching reliability. The specific creation method is as follows: Initialize a matrix of all zeros ( The width of the image (High), for each matching point coordinate The value at the corresponding coordinate on the heatmap is incremented by 1; after traversing all the coordinates of the matching points, the spatial density heatmap of the matching points on the image is obtained. The resulting heatmap Gaussian smoothing and normalization are performed to obtain the spatial attention weights for each matching point: ; By combining the spatial attention and spatial distribution attention weights of the two types mentioned above, the final fused attention weights used for subsequent filtering can be obtained: ; in, , These are the weighting coefficients.

[0069] This invention proposes an image registration method specifically for infrared and visible light images of power equipment. First, a comprehensive illumination scoring mechanism is established to measure illumination intensity, and based on this, a feature matching weight factor that dynamically changes with illumination intensity is calculated, thereby obtaining the confidence threshold for illumination-adaptive matching point pairs. Second, an unsupervised attention mechanism is constructed, which fully utilizes the feature space distribution characteristics of matching point pairs to generate attention weights for each matching point pair, and integrates these weights into the RANSAC screening process to perform weighted random probability sampling, thereby improving the iterative efficiency and accuracy of the feature matching pair screening process. The specific steps of this invention include: training a SuperPoint network to extract feature points from two infrared and visible light images to be registered; inputting the extracted two types of feature points into a LightGlue network for coarse matching to obtain LightGlue matching pairs; inputting the extracted two types of feature points into an OmniGlue network for coarse matching to obtain OmniGlue matching pairs; quantifying the illumination intensity of the visible light image and constructing a comprehensive illumination score; establishing a nonlinear dynamic mapping relationship between illumination intensity and feature matching weight factors; calculating the confidence thresholds of each of the two types of matching pairs based on the dynamic weight factors; filtering the two types of matching pairs based on their respective confidence thresholds and merging the filtered two types of matching pairs; generating unsupervised fusion attention weights for the merged matching pairs; iterating using the RANSAC algorithm of fusion attention weights to filter the merged matching pair set to obtain refined matching pairs; calculating the affine transformation matrix parameters based on the refined matching pairs; and transforming the visible light image to the infrared image coordinate system using the affine transformation matrix. The method of this invention takes into account the matching difficulties under extreme lighting conditions. It improves the feature matching process by using dynamic weight mapping to adaptively adjust the confidence threshold of matching point pairs, and improves the RANSAC feature selection process by using an unsupervised attention mechanism based on feature space distribution features. This enables more accurate feature matching of infrared and visible light images of substation equipment and improves the accuracy of infrared and visible light image registration of substation equipment.

[0070] like Figure 3 As shown, based on the above method embodiments, corresponding system embodiments are provided; An embodiment of the present invention provides an infrared and visible light image registration system 300 for power equipment, comprising: a feature extraction module 301, a feature matching module 302, a confidence calculation module 303, a confidence filtering module 304, and an image registration module 305; The feature extraction module 301 is used to extract feature points from infrared images and visible light images based on the SuperPoint network, respectively, to obtain infrared feature points and visible light feature points. The feature matching module 302 is used to input the infrared feature points and the visible light feature points into the LightGlue network for matching to obtain a LightGlue matching pair set, and input the infrared feature points and the visible light feature points into the OmniGlue network to obtain an OmniGlue matching pair set; The confidence calculation module 303 is used to dynamically calculate the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set according to the weight factors of the LightGlue matching pair set; wherein, the weight factors of the LightGlue matching pair set are calculated based on the illumination intensity of the visible light image; The confidence filtering module 304 is used to filter the LightGlue matching pair set based on the LightGlue confidence threshold, and to filter and merge the OmniGlue matching pair set based on the OmniGlue confidence threshold to obtain an initial matching pair set. The image registration module 305 is used to filter the initial matching pair set using the RANSAC algorithm to obtain a target matching pair set, calculate an affine transformation matrix based on the target matching pair set, and transform the visible light image based on the affine transformation matrix to achieve registration between the infrared image to be registered and the visible light image to be registered.

[0071] It is understood that the above system embodiments correspond to the method embodiments of the present invention, and can implement the infrared and visible light image registration method for power equipment provided by any of the above method embodiments of the present invention.

[0072] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0073] For ease of description and brevity, the system embodiments of the present invention include all the implementation methods described in the above embodiments of the UAV-based information recognition method, and will not be repeated here.

[0074] Based on the above embodiments of the infrared and visible light image registration method for power equipment, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the infrared and visible light image registration method for power equipment according to any embodiment of the present invention.

[0075] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0076] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0077] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0078] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the infrared and visible light image registration method for power equipment described in any of the above-described method embodiments of the present invention.

[0079] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0080] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for registering infrared and visible light images of power equipment, characterized in that, include: Feature points are extracted from infrared and visible light images using the SuperPoint network to obtain infrared and visible light feature points respectively. The infrared feature points and the visible light feature points are input into the LightGlue network for matching to obtain a LightGlue matching pair set, and the infrared feature points and the visible light feature points are input into the OmniGlue network to obtain an OmniGlue matching pair set; Based on the weight factors of the LightGlue matching pair set, the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set are dynamically calculated respectively; wherein, the weight factors of the LightGlue matching pair set are calculated based on the illumination intensity of the visible light image; The LightGlue matching pair set is filtered based on the LightGlue confidence threshold, and the OmniGlue matching pair set is filtered based on the OmniGlue confidence threshold and then merged to obtain the initial matching pair set. The RANSAC algorithm is used to filter the initial matching pair set to obtain the target matching pair set. An affine transformation matrix is ​​calculated based on the target matching pair set, and the visible light image is transformed based on the affine transformation matrix to achieve the registration of the infrared image to be registered with the visible light image to be registered.

2. The infrared and visible light image registration method for power equipment as described in claim 1, characterized in that, The step of dynamically calculating the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set based on the weight factors of the LightGlue matching pair set is as follows: An illumination score is calculated based on the illumination intensity of the visible light image, and a weighting factor is generated based on the illumination score through a nonlinear mapping function. Obtain the initial LightGlue confidence threshold and the LightGlue weight factor adjustment coefficient. Adjust the initial LightGlue confidence threshold according to the LightGlue weight factor adjustment coefficient and the weight factor to obtain the LightGlue confidence threshold. Obtain the initial OmniGlue confidence threshold and the OmniGlue weight factor adjustment coefficient. Adjust the initial OmniGlue confidence threshold according to the OmniGlue weight factor adjustment coefficient and the weight factor to obtain the OmniGlue confidence threshold.

3. The infrared and visible light image registration method for power equipment as described in claim 2, characterized in that, The illumination score is calculated based on the illumination intensity of the visible light image, specifically as follows: The visible light image is converted from the RGB color space to the HSV color space to obtain the luminance component; The initial average brightness of the visible light image is calculated based on the brightness components, and the initial average brightness is normalized to obtain the average brightness. The standard deviation of the brightness of the visible light image is calculated based on the average brightness value. The proportion of the first brightness region is obtained by counting the number of pixels in the visible light image whose brightness value is greater than the first brightness threshold. The proportion of the second brightness region is obtained by counting the number of pixels in the visible light image whose brightness value is less than the second brightness threshold. The first brightness threshold is greater than the second brightness threshold. The illumination score is calculated based on the mean brightness, the standard deviation of brightness, the proportion of the first brightness region, and the proportion of the second brightness region.

4. The infrared and visible light image registration method for power equipment as described in claim 3, characterized in that, The average brightness value is specifically: ; in, This represents the total number of pixels in the image. For the first The brightness value of each pixel, with the value ranging from 0 to a certain value. Between, divided by This will normalize the result to the [0,1] interval; The standard deviation of the brightness is: ; in, Used to characterize the discreteness of the brightness distribution of the visible light image and reflect the global contrast; The proportion of the first brightness region is: ; in, (⋅) is an indicator function; its value is 1 when its internal condition is true, and 0 otherwise. This indicates that the brightness value is greater than the first brightness threshold. And less than the brightness value The number of pixels; The proportion of the second brightness region is: ; in, This indicates that the brightness value is less than the second preset threshold. And the number of pixels with values ​​greater than zero; The illumination score is: 。 5. The infrared and visible light image registration method for power equipment as described in claim 2, characterized in that, The LightGlue confidence threshold is specifically as follows: ; in, The initial LightGlue confidence threshold, , This is the adjustment coefficient for the LightGlue weighting factor. ; It is a weighting factor, when When the value is 0.5, the illumination is normal. ;when When the value is 1, the lighting is abnormal. , that is, all LightGlue matching points are retained; The confidence threshold for OmniGlue is: ; in, Initial OmniGlue confidence threshold, ; The OmniGlue weighting factor adjustment coefficient. ;when When the value is 0.5, the illumination is normal. The initial OmniGlue confidence threshold; when When the value is 1, the lighting is abnormal. .

6. The infrared and visible light image registration method for power equipment as described in claim 1, characterized in that, The initial matching pair set is filtered using the RANSAC algorithm to obtain the target matching pair set, specifically as follows: The fusion attention weights for each matching pair are determined and normalized to obtain the probability distribution of the initial matching pair set; In each iteration of the RANSAC algorithm, a preset number of matching pairs are randomly sampled and estimated according to the cumulative probability distribution function corresponding to the probability distribution to obtain the homography transformation matrix of the current iteration. Matching pairs that do not meet the preset conditions in the initial matching pair set are then removed according to the homography transformation matrix to obtain the target matching pair set.

7. The infrared and visible light image registration method for power equipment as described in claim 6, characterized in that, The fusion attention weights are calculated based on the feature space attention weights and the spatial distribution attention weights, specifically: For each matching pair in the initial matching pair set, the feature cosine similarity is calculated; Based on the feature similarity between each feature point in the visible light image and each nearest neighbor feature point in the set of k nearest neighbor feature points, the similarity variance is calculated to obtain the local consistency weight. The feature space attention weight is calculated based on the feature cosine similarity and the local consistency weight. The spatial density heatmap is obtained by traversing each feature point in the visible light image, and the spatial heatmap is then Gaussian smoothed and normalized to obtain the spatial attention weights. The fusion attention weight is calculated based on the feature space attention weight and the spatial distribution attention weight.

8. An infrared and visible light image registration system for power equipment, characterized in that, include: Feature extraction module, feature matching module, confidence calculation module, confidence filtering module, and image registration module; The feature extraction module is used to extract feature points from infrared images and visible light images based on the SuperPoint network, respectively, to obtain infrared feature points and visible light feature points. The feature matching module is used to input the infrared feature points and the visible light feature points into the LightGlue network for matching to obtain a LightGlue matching pair set, and input the infrared feature points and the visible light feature points into the OmniGlue network to obtain an OmniGlue matching pair set; The confidence calculation module is used to dynamically calculate the LightGlue confidence threshold corresponding to the LightGlue matching pair set and the OmniGlue confidence threshold corresponding to the OmniGlue matching pair set based on the weight factors of the LightGlue matching pair set; wherein, the weight factors of the LightGlue matching pair set are calculated based on the illumination intensity of the visible light image; The confidence filtering module is used to filter the LightGlue matching pair set based on the LightGlue confidence threshold, and to filter and merge the OmniGlue matching pair set based on the OmniGlue confidence threshold to obtain an initial matching pair set. The image registration module is used to filter the initial matching pair set using the RANSAC algorithm to obtain a target matching pair set, calculate an affine transformation matrix based on the target matching pair set, and transform the visible light image based on the affine transformation matrix to achieve registration between the infrared image to be registered and the visible light image to be registered.

9. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the infrared and visible light image registration method for power equipment as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the infrared and visible light image registration method for power equipment as described in any one of claims 1-7.

Images