Infrared-visible light registration method, device and equipment for distribution network equipment and storage medium

Abstract

The invention discloses an infrared-visible light registration method, device and equipment for distribution network equipment and a storage medium, which are used for solving the problem that the positions of equipment characteristics in two images are not matched due to the fact that the two images cannot be naturally aligned generally because the imaging principles, visual angles and resolutions of the infrared image and the visible light image are different. And difficulty is brought to fault positioning and diagnosis. The method comprises the following steps: acquiring an infrared image and a visible light image of the distribution network equipment; extracting a first feature of the infrared image and a second feature of the visible light image; performing coarse registration on the first feature and the second feature to obtain a coarse registration infrared image; extracting a third feature of the coarse registration infrared image and a fourth feature of the visible light image; generating a point matching pair of the third feature and the fourth feature; generating a regular grid on the visible light image; calculating the offset of each grid point on the regular grid through the point matching pair; and adjusting the visible light image according to the offset to obtain a registered visible light image.

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and in particular to an infrared-visible light registration method, device, equipment and storage medium for distribution network equipment. Background Art

[0002] As a vital component of the power system, the operating status of distribution network equipment is directly related to the safe and stable operation and power supply quality of the entire power grid. Failures in these devices can not only cause localized power outages but can also trigger chain reactions, impacting power supply across a wider area. Therefore, accurately assessing and monitoring the operating status of distribution network equipment in real time is of great practical significance. During equipment operation, many potential defects and faults often first manifest as changes in thermal conditions. For example, poor contact can lead to localized overheating, and aging insulation can cause abnormal temperature rises. These thermal characteristics are important indicators of equipment health. Based on this principle, thermal imaging technology (infrared detection) has become an important means of monitoring the status of distribution network equipment. Using infrared thermal imagers, maintenance personnel can obtain temperature distribution information on equipment in a non-contact manner, enabling timely detection of abnormalities such as overheating. However, relying solely on infrared images can sometimes make it difficult to accurately locate faults, especially in complex scenarios. To enhance the detail in infrared images and improve the accuracy of fault diagnosis, visible light images are often acquired simultaneously. Visible light images provide clear information about the device's appearance, structural features, and control locations, facilitating better understanding and analysis of thermal signatures in infrared images. However, due to differences in imaging principles, viewing angles, and resolutions between infrared and visible light cameras, the two images often cannot be naturally aligned. This misalignment results in mismatched positions of device features in the two images, making fault location and diagnosis difficult. Summary of the Invention

[0003] The present invention provides an infrared-visible light registration method, apparatus, device and storage medium for distribution network equipment, which are used to solve the problem that there are differences in imaging principles, viewing angles and resolutions between infrared images and visible light images, resulting in the two images usually being unable to be naturally aligned, causing the positions of device features in the two images to not match, which brings difficulties to fault location and diagnosis.

[0004] The present invention provides an infrared-visible light registration method for distribution network equipment, comprising:

[0005] Acquire infrared and visible light images of distribution network equipment;

[0006] extracting a first feature of the infrared image and a second feature of the visible light image;

[0007] performing coarse registration on the first feature and the second feature to obtain a coarsely registered infrared image;

[0008] extracting a third feature of the coarsely registered infrared image and a fourth feature of the visible light image;

[0009] generating a point matching pair of the third feature and the fourth feature;

[0010] Generate a regular grid on the visible light image;

[0011] Calculating the offset of each grid point on the regular grid through the point matching pairs;

[0012] The visible light image is adjusted according to the offset to obtain a registered visible light image.

[0013] Optionally, the step of performing coarse registration on the first feature and the second feature to obtain a coarsely registered infrared image includes:

[0014] Matching the first feature and the second feature using a LightGlue model to obtain a point matching result;

[0015] Using the point matching results, generating a homography matrix;

[0016] The infrared image is adjusted using the homography matrix to obtain a coarsely registered infrared image.

[0017] Optionally, the step of generating a point matching pair of the third feature and the fourth feature includes:

[0018] Generating a first high-dimensional feature map of the third feature and a second high-dimensional feature map of the fourth feature;

[0019] Generate point matching pairs according to the first high-dimensional feature map and the second high-dimensional feature map.

[0020] Optionally, the third feature includes a third descriptor and a third key point; the fourth feature includes a fourth descriptor and a fourth key point; and the step of generating a first high-dimensional feature map of the third feature and a second high-dimensional feature map of the fourth feature includes:

[0021] Acquiring a coarsely registered infrared image length and a coarsely registered infrared image width of the coarsely registered infrared image;

[0022] Generate a first matrix of 257 dimensions using the coarsely registered infrared image length, the coarsely registered infrared image width, the third descriptor, and the third key point;

[0023] Acquiring a visible light image length and a visible light image width of the visible light image;

[0024] Generate a second matrix of 257 dimensions using the visible light image length, the visible light image width, the fourth descriptor, and the fourth key point;

[0025] extracting a first high-dimensional feature from the first matrix;

[0026] A second high-dimensional feature is extracted from the second matrix.

[0027] The present invention also provides an infrared-visible light registration device for distribution network equipment, comprising:

[0028] Infrared image and visible light image acquisition module, used to acquire infrared images and visible light images of distribution network equipment;

[0029] A first feature and a second feature extraction module, configured to extract a first feature of the infrared image and a second feature of the visible light image;

[0030] a coarse registration module, configured to perform coarse registration on the first feature and the second feature to obtain a coarsely registered infrared image;

[0031] a third feature and a fourth feature extraction module, configured to extract the third feature of the coarsely registered infrared image and the fourth feature of the visible light image;

[0032] a point matching pair generating module, configured to generate a point matching pair of the third feature and the fourth feature;

[0033] A regular grid generation module, used for generating a regular grid on a visible light image;

[0034] an offset calculation module, configured to calculate the offset of each grid point on the regular grid using the point matching pairs;

[0035] A registration module is configured to adjust the visible light image according to the offset to obtain a registered visible light image.

[0036] Optionally, the coarse registration module includes:

[0037] A matching submodule, configured to match the first feature and the second feature using a LightGlue model to obtain a point matching result;

[0038] A homography matrix generation submodule, configured to generate a homography matrix using the point matching results;

[0039] The coarse registration submodule is used to adjust the infrared image using the homography matrix to obtain a coarsely registered infrared image.

[0040] Optionally, the point matching pair generation module includes:

[0041] A first high-dimensional feature map and a second high-dimensional feature map generating submodule, configured to generate a first high-dimensional feature map of the third feature and a second high-dimensional feature map of the fourth feature;

[0042] A point matching pair generation submodule is used to generate point matching pairs according to the first high-dimensional feature map and the second high-dimensional feature map.

[0043] Optionally, the third feature includes a third descriptor and a third key point; the fourth feature includes a fourth descriptor and a fourth key point; and the first high-dimensional feature map and the second high-dimensional feature map generation submodule include:

[0044] A first length and width acquisition unit is used to acquire a coarse registration infrared image length and a coarse registration infrared image width of the coarse registration infrared image;

[0045] A first matrix generating unit, configured to generate a first matrix of 257 dimensions by using the length of the coarsely registered infrared image, the width of the coarsely registered infrared image, the third descriptor, and the third key point;

[0046] a second length and width acquiring unit, configured to acquire a visible light image length and a visible light image width of the visible light image;

[0047] A second matrix generating unit, configured to generate a second matrix of 257 dimensions using the visible light image length, the visible light image width, the fourth descriptor, and the fourth key point;

[0048] A first high-dimensional feature map extraction unit, configured to extract a first high-dimensional feature map from the first matrix;

[0049] A second high-dimensional feature map extraction unit is used to extract a second high-dimensional feature map from the second matrix.

[0050] The present invention further provides an electronic device, comprising a processor and a memory:

[0051] The memory is used to store program code and transmit the program code to the processor;

[0052] The processor is used to execute the infrared-visible light registration method for network distribution equipment as described in any one of the above items according to the instructions in the program code.

[0053] The present invention also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the infrared-visible light registration method for distribution network equipment as described in any one of the above items.

[0054] It can be seen from the above technical solution that the present invention has the following advantages: the present invention provides an infrared-visible light registration method for distribution network equipment, and specifically discloses: obtaining an infrared image and a visible light image of the distribution network equipment; extracting a first feature of the infrared image and a second feature of the visible light image; coarsely aligning the first feature and the second feature to obtain a coarsely registered infrared image; extracting a third feature of the coarsely registered infrared image and a fourth feature of the visible light image; generating a point matching pair of the third feature and the fourth feature; generating a regular grid on the visible light image; calculating the offset of each grid point on the regular grid through the point matching pair; adjusting the visible light image according to the offset to obtain a registered visible light image.

[0055] The present invention performs coarse registration of features of infrared images and visible light images to obtain a coarsely registered image; then performs point matching on the coarsely registered image and the visible light image to calculate an offset based on the point matching pairs, and then adjusts the visible light image based on the offset, so that the adjusted visible light image and the coarsely registered infrared image can be finely registered, thereby improving the accuracy of subsequent fault location and diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 A flowchart of the steps of an infrared-visible light registration method for network distribution equipment provided by an embodiment of the present invention;

[0058] Figure 2 A flowchart of a method for infrared-visible light registration of network distribution equipment provided in another embodiment of the present invention;

[0059] Figure 3 An architecture diagram of an infrared-visible light registration method for network distribution equipment provided by an embodiment of the present invention;

[0060] Figure 4 This is a structural block diagram of an infrared-visible light alignment device for network distribution equipment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Embodiments of the present invention provide a method, apparatus, device, and storage medium for infrared-visible light registration of distribution network equipment, which are used to address the technical problem that differences in imaging principles, viewing angles, and resolutions between infrared and visible light images result in the two images often being unable to be naturally aligned, resulting in mismatched positions of device features in the two images, and bringing difficulties to fault location and diagnosis.

[0062] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0063] See also Figure 1 , Figure 1 A flowchart of the steps of an infrared-visible light registration method for network distribution equipment provided by an embodiment of the present invention.

[0064] The present invention provides a method for infrared-visible light registration of network distribution equipment, which may specifically include the following steps:

[0065] Step 101: Acquire infrared images and visible light images of distribution network equipment;

[0066] Infrared images use infrared radiation (typically with wavelengths between 0.75μm and 1000μm) to capture the thermal radiation or reflective properties of an object. Unlike visible light images, infrared imaging does not rely on ambient lighting. Instead, it presents information by detecting temperature distribution or differences in infrared reflectivity on an object's surface.

[0067] Visible light imaging is achieved by capturing electromagnetic waves with wavelengths between 380nm and 750nm (i.e., light perceptible to the human eye). It is the most common imaging method in daily life and is widely used in photography, surveillance, autonomous driving, medical imaging, and other fields.

[0068] In an embodiment of the present invention, infrared images and visible light images of the network distribution equipment may be collected to facilitate subsequent registration.

[0069] Step 102, extracting a first feature of the infrared image and a second feature of the visible light image;

[0070] In the embodiment of the present invention, the first feature of the infrared image and the second feature of the visible light image may be extracted for subsequent coarse registration.

[0071] The first feature may include a first key point and a first descriptor; the second feature may include a second key point and a second descriptor.

[0072] Key points are locations in an image that have significant features and can be used for identification, matching, or analysis. They typically have the following characteristics:

[0073] Unique features (e.g., corners, edges, textured areas);

[0074] Repeatability (detectable under different viewing angles or lighting);

[0075] Stability (robust to rotation, scaling, and lighting changes).

[0076] A descriptor is a mathematical representation used to quantify the characteristics of a keypoint (or image region) so that it can be efficiently matched and compared by computers. It is usually a fixed-length vector that describes information such as the pixel distribution, texture, and gradient around the keypoint.

[0077] Step 103, performing coarse registration on the first feature and the second feature to obtain a coarsely registered infrared image;

[0078] After the first feature of the infrared image and the second feature of the visible light image are collected, the first feature and the second feature may be roughly registered to obtain a roughly registered infrared image.

[0079] In one example, SuperPoint and LightGlue may be used to implement coarse registration of an infrared image and a visible light image to obtain a coarsely registered infrared image.

[0080] SuperPoint is an end-to-end feature point detection and descriptor generation model based on deep learning. It achieves higher robustness and accuracy than traditional methods (such as SIFT and ORB) in complex scenes through self-supervised learning. It is widely used in tasks such as visual SLAM, image registration, and 3D reconstruction.

[0081] LightGlue is a deep neural network for matching sparse local features in image pairs.

[0082] Step 104, extracting the third feature of the roughly registered infrared image and the fourth feature of the visible light image;

[0083] In the embodiment of the present invention, the third feature of the coarsely registered infrared image includes a third key point and a third descriptor; and the fourth feature of the visible light image includes a fourth key point and a fourth descriptor.

[0084] The extraction process of the third feature and the fourth feature can refer to the extraction process of the first feature and the second feature, and will not be repeated here.

[0085] Step 105: Generate a point matching pair of the third feature and the fourth feature;

[0086] After obtaining the third feature and the fourth feature, the third feature and the fourth feature may be matched to obtain a point matching pair.

[0087] Step 106, generating a regular grid on the visible light image;

[0088] Step 107, calculating the offset of each grid point on the regular grid by point matching;

[0089] Step 108 : Adjust the visible light image according to the offset to obtain a registered visible light image.

[0090] A regular grid is a grid with equal spacing between edges. ,in and are the number of grids. The length and width of each grid are typically set to 10×10, and the grid point g ij The initial coordinates are (x ij ,y ij ).

[0091] After generating point matching pairs of the coarsely registered infrared image and the visible light image, a regular grid can be generated on the visible light image. Then, the offset of each grid point on the regular grid is calculated based on the point matching pairs. The visible light image is adjusted according to the offset to obtain a registered visible light image that is registered with the coarsely registered infrared image.

[0092] The present invention performs coarse registration of features of infrared images and visible light images to obtain a coarsely registered image; then performs point matching on the coarsely registered image and the visible light image to calculate an offset based on the point matching pairs, and then adjusts the visible light image based on the offset, so that the adjusted visible light image and the coarsely registered infrared image can be finely registered, thereby improving the accuracy of subsequent fault location and diagnosis.

[0093] See also Figure 2 and Figure 3 , Figure 2 A flowchart of a method for infrared-visible light registration of network distribution equipment provided by another embodiment of the present invention is provided. Figure 3 This is an architecture diagram of a method for infrared-visible light registration of network distribution equipment provided by an embodiment of the present invention. Specifically, the following steps may be included:

[0094] Step 201: Acquire infrared images and visible light images of distribution network equipment;

[0095] Step 202: extracting a first feature of the infrared image and a second feature of the visible light image;

[0096] Steps 201-202 are the same as steps 101-102. For details, please refer to the description of steps 101-102, which will not be repeated here.

[0097] Step 203, matching the first feature and the second feature using the LightGlue model to obtain a point matching result;

[0098] In the embodiment of the present invention, the first feature includes a first key point and a first description point, and the second feature includes a second key point and a second description point; the infrared image I of the input distribution network device ir and visible light image I vis , the SuperPoint model can be used to extract the keypoints and descriptors of infrared images and visible light images respectively. The first keypoint of the infrared image is: , the first descriptor of the infrared image is: , the first key point of visible light is: , the first descriptor of visible light is: .

[0099]

[0100]

[0101] The first feature and the second feature may be roughly matched to obtain a point matching result.

[0102] In one example, the LightGlue model can be used to quantize features of infrared and visible light images ( , ) Predict a matching matrix , where the element s in row p and column q in S is pq represents the probability that the p-th point in the infrared image matches the q-th element in the visible light image. Finally, the nearest neighbor matching is used to determine the matching pair based on these scores. As a point matching result:

[0103]

[0104] in, are the horizontal and vertical coordinates corresponding to the Dth matching point in the infrared image, They can be the horizontal coordinate and vertical coordinate corresponding to the D-th matching point in the visible light image respectively.

[0105] Step 204, using the point matching results to generate a homography matrix;

[0106] The homography matrix is ​​a 3×3 matrix that is usually used to describe the projective transformation between two planes (2D to 2D mapping).

[0107] In the embodiment of the present invention, the point matching result can be used to generate the homography matrix H by the RANSAC algorithm, where H∈R 3×3 :

[0108] H = RANSAC (S match )

[0109] The RANSAC (RANdom Sampling Consensus) algorithm is a robust model fitting algorithm used to estimate mathematical model parameters from datasets containing a large number of outliers (noise / erroneous data). It is widely used in computer vision for tasks such as homography matrix estimation, point cloud registration, and line / plane fitting. RANSAC randomly selects four points from a matching pair and uses them to calculate a candidate homography matrix H. The candidate H is then used to calculate the reprojection error for all matching points:

[0110]

[0111] Then count the points whose error is less than the threshold, i.e., the inliers. Repeat the above steps K times, and keep H with the most inliers.

[0112] Step 205, adjusting the infrared image using a homography matrix to obtain a coarsely registered infrared image;

[0113] In an embodiment of the present invention, a homography transformation may be applied to the infrared image to obtain a coarsely registered infrared image:

[0114]

[0115] in, For coarse registration of infrared images.

[0116] Step 206, extracting the third feature of the roughly registered infrared image and the fourth feature of the visible light image;

[0117] In the embodiment of the present invention, SuperPoint can be used to roughly register infrared images. Extract the third key point and the third descriptor; use SuperPoint to analyze the visible light image I vis Extract the fourth key point and the fourth descriptor. Among them, the third key point is: , the third descriptor is: , the fourth key point is: , the fourth descriptor is: , where the descriptor has 256 dimensions.

[0118] Step 207, generating a point matching pair of the third feature and the fourth feature;

[0119] After obtaining the third feature and the fourth feature, the third feature and the fourth feature may be matched to obtain a point matching pair.

[0120] In one example, the step of generating a point matching pair of the third feature and the fourth feature includes:

[0121] S71, generating a first high-dimensional feature map of the third feature and a second high-dimensional feature map of the fourth feature;

[0122] Step S71 includes:

[0123] S711, obtaining a coarsely registered infrared image length and a coarsely registered infrared image width of the coarsely registered infrared image;

[0124] S712, generating a first matrix of 257 dimensions using the coarsely registered infrared image length, the coarsely registered infrared image width, the third descriptor, and the third key point;

[0125] S713, obtaining a visible light image length and a visible light image width of the visible light image;

[0126] S714, generating a second matrix of 257 dimensions using the visible light image length, the visible light image width, the fourth descriptor, and the fourth key point;

[0127] S715, extracting a first high-dimensional feature map from the first matrix;

[0128] S716: Extract a second high-dimensional feature map from the second matrix.

[0129] In the specific implementation, the CNN network can be used to generate two H1×W1×257 matrices T ir (first matrix), T vis (Second matrix), where the length of the coarsely registered infrared image and the visible light image are both H1 and W1, and 257 refers to 257 dimensions. The first dimension is 0 or 1, with 1 indicating that the pixel has a keypoint and 0 indicating that the pixel does not have a keypoint. Dimensions 2-257 are the descriptors corresponding to the keypoints.

[0130] FeatureBooster is a deep learning network for enhancing local feature descriptors. Its core goal is to improve the robustness of feature matching in complex scenes (such as illumination changes, occlusion, and low texture) by learning the contextual information and geometric consistency of feature descriptors.

[0131] S72: Generate point matching pairs based on the first high-dimensional feature map and the second high-dimensional feature map.

[0132] The precise point matching pairs generated by the first high-dimensional feature map, the second high-dimensional feature map and the FeatureBooster network are:

[0133]

[0134] in, are the horizontal and vertical coordinates corresponding to the Dth matching point in the coarse registration infrared image, They can be the horizontal coordinate and vertical coordinate corresponding to the D-th matching point in the visible light image respectively.

[0135] Specifically, FeatureBooster transforms T ir 、T vis Use MLP to project it into high-dimensional space, and then use the Transformer structure contained in it to ir 、T vis Perform feature interaction to generate the final feature T ir ', T vis '. Finally, based on the similarity between the two features, the nearest neighbor search is performed to obtain the final S match '.

[0136] Step 208, generating a regular grid on the visible light image;

[0137] A regular grid is a grid with equal spacing between edges. , where H and W are the height and width of the grid respectively. A typical setting is 10×10, with a grid point g ij The initial coordinates are (x ij ,y ij ).

[0138] Step 209, calculating the offset of each grid point on the regular grid by point matching;

[0139] Step 210 : Adjust the visible light image according to the offset to obtain a registered visible light image.

[0140] according to Calculate the offset of each grid point Δgij=(Δx ij ,Δy ij ), the offset coordinates are:

[0141]

[0142] The regular grid is offset according to the offset coordinates, and the visible light image is adjusted to achieve alignment of the visible light and infrared images, and a registered visible light image can be obtained.

[0143] The present invention performs coarse registration of features of infrared images and visible light images to obtain a coarsely registered image; then performs point matching on the coarsely registered image and the visible light image to calculate an offset based on the point matching pairs, and then adjusts the visible light image based on the offset, so that the adjusted visible light image and the coarsely registered infrared image can be finely registered, thereby improving the accuracy of subsequent fault location and diagnosis.

[0144] See also Figure 4 , Figure 4 This is a structural block diagram of an infrared-visible light alignment device for network distribution equipment provided by an embodiment of the present invention.

[0145] An embodiment of the present invention provides an infrared-visible light registration device for network distribution equipment, comprising:

[0146] Infrared image and visible light image acquisition module 401, used to acquire infrared images and visible light images of distribution network equipment;

[0147] A first feature and a second feature extraction module 402 for extracting a first feature of the infrared image and a second feature of the visible light image;

[0148] A coarse registration module 403 is configured to perform coarse registration on the first feature and the second feature to obtain a coarsely registered infrared image;

[0149] a third feature and a fourth feature extraction module 404 for extracting the third feature of the coarsely registered infrared image and the fourth feature of the visible light image;

[0150] A point matching pair generating module 405 is used to generate a point matching pair of the third feature and the fourth feature;

[0151] A regular grid generating module 406 is used to generate a regular grid on the visible light image;

[0152] An offset calculation module 407 is used to calculate the offset of each grid point on the regular grid through point matching;

[0153] The registration module 408 is configured to adjust the visible light image according to the offset to obtain a registered visible light image.

[0154] In this embodiment of the present invention, the coarse registration module 403 includes:

[0155] The matching submodule is used to match the first feature and the second feature through the LightGlue model to obtain a point matching result;

[0156] The homography matrix generation submodule is used to generate the homography matrix using the point matching results;

[0157] The coarse registration submodule is used to adjust the infrared image using the homography matrix to obtain a coarsely registered infrared image.

[0158] In this embodiment of the present invention, the point matching pair generation module 405 includes:

[0159] A first high-dimensional feature map and a second high-dimensional feature map generating submodule, configured to generate a first high-dimensional feature map of a third feature and a second high-dimensional feature map of a fourth feature;

[0160] The point matching pair generation submodule is used to generate point matching pairs according to the first high-dimensional feature map and the second high-dimensional feature map.

[0161] In an embodiment of the present invention, the third feature includes a third descriptor and a third key point; the fourth feature includes a fourth descriptor and a fourth key point; and the first high-dimensional feature map and the second high-dimensional feature map generation submodule include:

[0162] A first length and width acquisition unit is used to acquire a coarse registration infrared image length and a coarse registration infrared image width of the coarse registration infrared image;

[0163] A first matrix generating unit is used to generate a first matrix of 257 dimensions by using the length of the coarsely registered infrared image, the width of the coarsely registered infrared image, the third descriptor and the third key point;

[0164] a second length and width acquiring unit, configured to acquire a visible light image length and a visible light image width of the visible light image;

[0165] A second matrix generating unit is used to generate a second matrix of 257 dimensions using the visible light image length, the visible light image width, the fourth descriptor and the fourth key point;

[0166] A first high-dimensional feature map extraction unit, configured to extract a first high-dimensional feature map from the first matrix;

[0167] The second high-dimensional feature map extraction unit is used to extract a second high-dimensional feature map from the second matrix.

[0168] An embodiment of the present invention further provides an electronic device, the device including a processor and a memory:

[0169] The memory is used to store program codes and transmit the program codes to the processor;

[0170] The processor is used to execute the infrared-visible light registration method for network distribution equipment according to the instructions in the program code.

[0171] An embodiment of the present invention further provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the infrared-visible light registration method for network distribution equipment according to an embodiment of the present invention.

[0172] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0173] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0174] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0178] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0180] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0181] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for infrared-visible light registration of distribution network equipment, characterized in that: include: Acquire infrared and visible light images of distribution network equipment; extracting a first feature of the infrared image and a second feature of the visible light image; performing coarse registration on the first feature and the second feature to obtain a coarsely registered infrared image; extracting a third feature of the coarsely registered infrared image and a fourth feature of the visible light image; generating a point matching pair of the third feature and the fourth feature; Generate a regular grid on the visible light image; Calculating the offset of each grid point on the regular grid through the point matching pairs; The visible light image is adjusted according to the offset to obtain a registered visible light image.

2. The method according to claim 1, characterized in that The step of coarsely registering the first feature and the second feature to obtain a coarsely registered infrared image includes: Matching the first feature and the second feature using a LightGlue model to obtain a point matching result; Using the point matching results, generating a homography matrix; The infrared image is adjusted using the homography matrix to obtain a coarsely registered infrared image.

3. The method according to claim 1, characterized in that The step of generating a point matching pair of the third feature and the fourth feature includes: Generating a first high-dimensional feature map of the third feature and a second high-dimensional feature map of the fourth feature; Generate point matching pairs according to the first high-dimensional feature map and the second high-dimensional feature map.

4. The method according to claim 3, characterized in that The third feature includes a third descriptor and a third key point; the fourth feature includes a fourth descriptor and a fourth key point; and the step of generating a first high-dimensional feature map of the third feature and a second high-dimensional feature map of the fourth feature includes: Acquiring a coarsely registered infrared image length and a coarsely registered infrared image width of the coarsely registered infrared image; Generate a first matrix of 257 dimensions using the coarsely registered infrared image length, the coarsely registered infrared image width, the third descriptor, and the third key point; Acquiring a visible light image length and a visible light image width of the visible light image; Generate a second matrix of 257 dimensions using the visible light image length, the visible light image width, the fourth descriptor, and the fourth key point; extracting a first high-dimensional feature from the first matrix; A second high-dimensional feature is extracted from the second matrix.

5. An infrared-visible light registration device for distribution network equipment, characterized in that: include: Infrared image and visible light image acquisition module, used to acquire infrared images and visible light images of distribution network equipment; A first feature and a second feature extraction module, configured to extract a first feature of the infrared image and a second feature of the visible light image; a coarse registration module, configured to perform coarse registration on the first feature and the second feature to obtain a coarsely registered infrared image; a third feature and a fourth feature extraction module, configured to extract the third feature of the coarsely registered infrared image and the fourth feature of the visible light image; a point matching pair generating module, configured to generate a point matching pair of the third feature and the fourth feature; A regular grid generation module, used for generating a regular grid on a visible light image; an offset calculation module, configured to calculate the offset of each grid point on the regular grid using the point matching pairs; A registration module is configured to adjust the visible light image according to the offset to obtain a registered visible light image.

6. The device according to claim 5, characterized in that The coarse registration module includes: A matching submodule, configured to match the first feature and the second feature using a LightGlue model to obtain a point matching result; A homography matrix generation submodule, configured to generate a homography matrix using the point matching results; The coarse registration submodule is used to adjust the infrared image using the homography matrix to obtain a coarsely registered infrared image.

7. The device according to claim 5, characterized in that The point matching pair generation module includes: A first high-dimensional feature map and a second high-dimensional feature map generating submodule, configured to generate a first high-dimensional feature map of the third feature and a second high-dimensional feature map of the fourth feature; A point matching pair generation submodule is used to generate point matching pairs according to the first high-dimensional feature map and the second high-dimensional feature map.

8. The device according to claim 7, characterized in that The third feature includes a third descriptor and a third key point; the fourth feature includes a fourth descriptor and a fourth key point; the first high-dimensional feature map and the second high-dimensional feature map generation submodule include: A first length and width acquisition unit is used to acquire a coarse registration infrared image length and a coarse registration infrared image width of the coarse registration infrared image; A first matrix generating unit, configured to generate a first matrix of 257 dimensions by using the length of the coarsely registered infrared image, the width of the coarsely registered infrared image, the third descriptor, and the third key point; a second length and width acquiring unit, configured to acquire a visible light image length and a visible light image width of the visible light image; A second matrix generating unit, configured to generate a second matrix of 257 dimensions using the visible light image length, the visible light image width, the fourth descriptor, and the fourth key point; A first high-dimensional feature map extraction unit, configured to extract a first high-dimensional feature map from the first matrix; A second high-dimensional feature map extraction unit is used to extract a second high-dimensional feature map from the second matrix.

9. An electronic device, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the infrared-visible light registration method for network distribution equipment according to any one of claims 1 to 4 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the infrared-visible light registration method for network distribution equipment according to any one of claims 1 to 4.

Images