Insulator dual-optical image cross-modality registration method and system

Abstract

The application discloses a kind of insulator double light image cross-modal registration method and system, wherein method includes obtaining the double light synchronous image of target insulator;Double light synchronous image is input into cross-modal registration model and is registered to obtain cross-modal registration result;Wherein, the process of registration processing includes: based on insulator geometric prior information, feature extraction is carried out to visible light image and infrared image respectively, to obtain multiscale visible light modal feature and infrared modal feature;Cross-modal attention interaction processing is carried out to visible light modal feature and infrared modal feature between the same scale, to generate multiscale cross-modal fusion feature;Cross-modal fusion feature is carried out to the fusion processing between multiple scales, to generate global fusion feature;Based on global fusion feature, spatial transformation parameter is predicted, and infrared image is geometrically transformed according to spatial transformation parameter, to obtain registration result.The method of the application improves the accuracy of double light image cross-modal registration.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for cross-modal registration of two-light images of insulators. Background Technology

[0002] In power system operation and maintenance, accurate and timely detection of zero-value insulators is a core element in ensuring the safe and stable operation of transmission lines. Non-contact detection technology, with its outstanding advantages of operational safety and high detection efficiency, has become the mainstream technical approach for insulator condition monitoring. Its core logic is to obtain the temperature distribution information of the insulator through infrared thermal imaging, and combine this with the geometric contours, texture details, and other appearance features presented by visible light images to achieve accurate location of zero-value defects.

[0003] Existing dual-light image registration for insulators relies on registration methods based on single-mode features, such as traditional algorithms like SIFT and ORB. However, visible light images and infrared thermal imaging differ fundamentally in their imaging principles, grayscale distribution, and feature representation. Visible light images focus on the structural morphology and texture details of the insulator, while infrared images emphasize temperature field distribution and can only present a blurry thermal radiation outline. This makes it difficult for existing single-mode registration methods to establish an effective correspondence between the two modes, resulting in insufficient registration accuracy. They can only achieve region-level alignment and cannot meet the pixel-level precise matching requirements for zero-value insulator temperature measurement. Summary of the Invention

[0004] This invention provides a method and system for cross-modal registration of two-light images of insulators, in order to solve the technical problem of how to improve existing cross-modal registration methods for two-light images and achieve the effect of improving the accuracy of cross-modal registration of two-light images.

[0005] To address the aforementioned technical problems, this invention provides a method for cross-modal registration of dual-light images of insulators, comprising: Acquire a dual-light synchronized image of the target insulator, wherein the dual-light synchronized image includes a visible light image and an infrared image; The dual-light synchronized image is input into a pre-trained cross-modal registration model for registration processing to obtain the cross-modal registration result of the dual-light synchronized image; The registration process includes: Based on the insulator's geometric prior information, modal adaptation feature extraction is performed on the visible light image and the infrared image respectively to obtain multi-scale visible light modal features and infrared modal features; The visible light modal features and the infrared modal features are subjected to cross-modal attention interaction processing at the same scale to generate multi-scale cross-modal fusion features; The cross-modal fusion features are subjected to multi-scale fusion processing to generate global fusion features; Based on the global fusion features, spatial transformation parameters are predicted, and the infrared image is geometrically transformed according to the spatial transformation parameters to obtain the registration result of the dual-light synchronized image.

[0006] As one preferred embodiment, the prior geometric information of the insulator includes the periodic distribution characteristics and axisymmetric structural characteristics of the target insulator; The modal adaptation feature extraction is performed on the visible light image and the infrared image based on the insulator geometric prior information to obtain multi-scale visible light modal features and infrared modal features, including: Based on the periodic distribution characteristics and the axisymmetric structural characteristics, a feature guidance map of the target insulator is generated; The feature guidance map is fused with the visible light image and the infrared image respectively to obtain a first visible light image and a first infrared image; Based on a parameter-independent dual-branch feature extraction structure, multi-scale feature extraction is performed on the first visible light image and the first infrared image to obtain visible light modal features and infrared modal features, wherein the visible light modal features and the infrared modal features respectively include at least fine-scale edge features, local structural features and global contour features.

[0007] As one preferred embodiment, the step of performing cross-modal attention interaction processing on the visible light modal features and the infrared modal features at the same scale to generate multi-scale cross-modal fusion features includes: The visible light mode feature and the infrared mode feature are respectively subjected to feature enhancement processing to obtain the first visible light mode feature and the first infrared mode feature; Pixel-level correlation analysis is performed on the first visible light modal features and the first infrared modal features based on the cross-attention mechanism, and a correlation weight matrix between modalities is generated based on the analysis results; Based on the correlation weight matrix, the first visible light modal features and the first infrared modal features of the same scale are fused to generate multi-scale cross-modal fusion features.

[0008] As one preferred embodiment, the step of performing multi-scale fusion processing on the cross-modal fusion features to generate global fusion features includes: The cross-modal fusion features at each scale are subjected to feature saliency analysis, and weight coefficients corresponding to the cross-modal fusion features at the corresponding scale are generated based on the analysis results; wherein, the feature saliency analysis is used to quantify the contribution of features at each scale to the registration results; Based on the bidirectional fusion path and the weight coefficients, the multi-scale cross-modal fusion features are weighted and aggregated to obtain the output features corresponding to the bidirectional fusion path respectively. The bidirectional fusion path includes an upsampling fusion branch that iteratively fuses from global contour features to fine-scale edge features, and a downsampling fusion branch that iteratively fuses from the fine-scale edge features to the global contour features. The first output feature of the upsampling fusion branch and the second output feature of the downsampling fusion branch are concatenated to generate the global fusion feature.

[0009] As one preferred embodiment, before inputting the dual-light synchronized image into a pre-trained cross-modal registration model for registration processing, the method further includes: Data enhancement processing is performed on the dual-light synchronized image to obtain a first dual-light synchronized image; wherein, the data enhancement processing is designed to enhance the texture details of the visible light image and optimize the contrast of the infrared image; A cross-modal registration dataset is constructed based on the first dual-light synchronized image, wherein each sample in the dataset contains a pair of visible light images and infrared images of insulators; The initial cross-modal registration model is trained and optimized based on the cross-modal registration dataset to obtain the pre-trained cross-modal registration model.

[0010] Another aspect of the present invention provides an insulator dual-light image cross-modal registration system, comprising: The acquisition module is used to acquire a dual-light synchronous image of the target insulator, wherein the dual-light synchronous image includes a visible light image and an infrared image; The registration module is used to input the dual-light synchronized image into a pre-trained cross-modal registration model for registration processing, so as to obtain the cross-modal registration result of the dual-light synchronized image; The registration process includes: The extraction unit is used to perform mode-adaptive feature extraction on the visible light image and the infrared image based on the insulator geometric prior information, respectively, to obtain multi-scale visible light mode features and infrared mode features; A cross-modal fusion unit is used to perform cross-modal attention interaction processing on the visible light modal features and the infrared modal features at the same scale to generate multi-scale cross-modal fusion features; A multi-scale fusion unit is used to perform multi-scale fusion processing on the cross-modal fusion features to generate global fusion features; The generation unit is used to predict spatial transformation parameters based on the global fusion features, perform geometric transformation on the infrared image according to the spatial transformation parameters, and generate the registration result of the dual-light synchronized image.

[0011] As one preferred embodiment, the prior geometric information of the insulator includes the periodic distribution characteristics and axisymmetric structural characteristics of the target insulator; The extraction unit is specifically used for: Based on the periodic distribution characteristics and the axisymmetric structural characteristics, a feature guidance map of the target insulator is generated; The feature guidance map is fused with the visible light image and the infrared image respectively to obtain a first visible light image and a first infrared image; Based on a parameter-independent dual-branch feature extraction structure, multi-scale feature extraction is performed on the first visible light image and the first infrared image to obtain visible light modal features and infrared modal features, wherein the visible light modal features and the infrared modal features respectively include at least fine-scale edge features, local structural features and global contour features.

[0012] As one preferred embodiment, the cross-modal fusion unit is specifically used for: The visible light mode feature and the infrared mode feature are respectively subjected to feature enhancement processing to obtain the first visible light mode feature and the first infrared mode feature; Pixel-level correlation analysis is performed on the first visible light modal features and the first infrared modal features based on the cross-attention mechanism, and a correlation weight matrix between modalities is generated based on the analysis results; Based on the correlation weight matrix, the first visible light modal features and the first infrared modal features of the same scale are fused to generate multi-scale cross-modal fusion features.

[0013] As one preferred embodiment, the multi-scale fusion unit is specifically used for: The cross-modal fusion features at each scale are subjected to feature saliency analysis, and weight coefficients corresponding to the cross-modal fusion features at the corresponding scale are generated based on the analysis results; wherein, the feature saliency analysis is used to quantify the contribution of features at each scale to the registration results; Based on the bidirectional fusion path and the weight coefficients, the multi-scale cross-modal fusion features are weighted and aggregated to obtain the output features corresponding to the bidirectional fusion path respectively. The bidirectional fusion path includes an upsampling fusion branch that iteratively fuses from global contour features to fine-scale edge features, and a downsampling fusion branch that iteratively fuses from the fine-scale edge features to the global contour features. The first output feature of the upsampling fusion branch and the second output feature of the downsampling fusion branch are concatenated to generate the global fusion feature.

[0014] As one preferred embodiment, the registration module is further configured to: Data enhancement processing is performed on the dual-light synchronized image to obtain a first dual-light synchronized image; wherein, the data enhancement processing is designed to enhance the texture details of the visible light image and optimize the contrast of the infrared image; A cross-modal registration dataset is constructed based on the first dual-light synchronized image, wherein each sample in the dataset contains a pair of visible light images and infrared images of insulators; The initial cross-modal registration model is trained and optimized based on the cross-modal registration dataset to obtain the pre-trained cross-modal registration model.

[0015] Compared with the prior art, the beneficial effects of the present invention are at least one of the following: 1) This invention significantly improves the accuracy and anti-interference capability of dual-light image registration through precise data acquisition and targeted feature extraction. On the one hand, the acquisition of synchronous dual-light images ensures the spatiotemporal consistency of visible light and infrared images, laying a data foundation for cross-modal registration and avoiding registration deviations caused by asynchronous acquisition. On the other hand, feature extraction for modal adaptation based on the insulator's geometric prior information can accurately focus on the core area of ​​the insulator, effectively filtering background interference from conductors, vegetation, metal supports, etc., while acquiring multi-scale modal features. This preserves the overall structural information of the insulator while capturing local detailed features. In addition, cross-modal attention interaction processing at the same scale can strengthen the effective correlation between the two light modes, weaken the feature misalignment problem caused by modal differences, and make the fused features more reflective of the essential properties of the insulator, greatly reducing registration errors.

[0016] 2) This invention further enhances the applicability and reliability of registration results through multi-scale fusion and precise geometric transformation, providing high-quality data support for subsequent detection tasks. Multi-scale fusion processing integrates cross-modal fusion features at different levels into global fusion features, taking into account both the macroscopic structure and microscopic details of the insulator, while enhancing the model's adaptability to complex situations such as insulator tilt and scale changes. By predicting spatial transformation parameters through global fusion features and performing geometric transformations on infrared images, precise spatial alignment of dual-light images can be achieved, perfectly adapting to the actual posture of the insulator string and solving the problem of insulator posture changes that are difficult to handle in traditional registration. High-quality registration results ensure accurate matching of dual-light information in subsequent tasks such as temperature detection and defect identification, providing reliable assurance for the accurate positioning and temperature detection of zero-value insulators, and improving the stability and effectiveness of the entire power operation and maintenance detection system. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the insulator dual-light image cross-modal registration method in one embodiment of the present invention; Figure 2 This is a schematic flowchart of the registration process in one embodiment of the present invention; Figure 3 This is a structural block diagram of an insulator dual-light image cross-modal registration system according to one embodiment of the present invention; Figure 4 This is a structural block diagram of the registration module in one embodiment of the present invention; Figure label: Among them, 11 is the acquisition module; 12 is the registration module; 21 is the extraction unit; 22 is the cross-modal fusion unit; 23 is the multi-scale fusion unit; and 24 is the generation unit. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this invention, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] One embodiment of the present invention provides a method for cross-modal registration of two-light images of insulators. For details, please refer to [link to relevant documentation]. Figure 1 , Figure 1 The diagram shown is a flowchart illustrating a method for cross-modal registration of two-light images of insulators according to one embodiment of the present invention, which includes steps S1-S2: S1: Acquire a dual-light synchronous image of the target insulator, wherein the dual-light synchronous image includes a visible light image and an infrared image.

[0022] In the field of power system operation and maintenance, accurate detection of zero-value insulators is a core link in ensuring the safe and stable operation of transmission lines. Non-contact detection technology has become the mainstream solution due to its advantages of safe operation and high efficiency. Its core logic is to obtain the temperature distribution information of the insulator through infrared thermal imaging and combine it with the geometric contour and texture details of visible light images to achieve defect location. The registration accuracy of dual-light images directly determines the defect detection effect. The premise is to obtain dual-light synchronous images with good spatiotemporal consistency, that is, visible light images and infrared images are acquired synchronously for the same insulator target under the same shooting scene. If there are problems such as spatiotemporal deviation, inconsistent viewing angle, or insufficient target integrity during the acquisition process, it will directly lead to an increase in subsequent registration errors and fail to meet the pixel-level alignment requirements. Therefore, this step focuses on the standardized acquisition of dual-light synchronous images to ensure the reliability of the data foundation.

[0023] In this embodiment, a multi-sensor inspection drone is used as the image acquisition carrier. This drone needs to be equipped with a high-definition visible light camera and a high-resolution infrared camera. The visible light camera is preferably an industrial-grade camera with at least 12 million pixels and a frame rate of at least 30fps, used to capture detailed features such as the insulator's skirt texture and the edges of metal attachments. The infrared camera is preferably an uncooled infrared focal plane camera with a temperature measurement accuracy of ±0.1℃ and a resolution of at least 640×512, used to accurately present the temperature field distribution of the insulator. It should be noted that this invention does not limit the specific model of the drone or the brand of the camera, as long as it meets the above-mentioned imaging accuracy and synchronization control requirements. For example, a DJI M300RTK drone and a FLIRVue ProR infrared camera can be used.

[0024] To ensure consistent viewing angles in dual-light images, the dual-camera components mounted on the UAV must be jointly calibrated before acquisition: a checkerboard standard calibration board (preferably 300mm×300mm in size, with 12×9 grids) is used. The calibration board is fixed on a horizontal plane and placed 5 meters directly in front of the UAV. Multiple sets of calibration images are captured simultaneously. The intrinsic parameters (focal length, principal point coordinates, distortion coefficients) and extrinsic parameters (relative position and attitude) of the two cameras are solved using the Zhang Zhengming calibration algorithm. The camera mounting bracket is adjusted to ensure that the parallelism deviation of the optical axes of the two cameras is less than 0.5°. At the same time, the focal length parameters of the two cameras are optimized to ensure that, at the target shooting distance, the insulator target occupies at least 60% of the image area in the visible light image and the temperature field distribution of the umbrella skirt structure is fully presented in the infrared image, avoiding the loss of target information due to mismatched fields of view.

[0025] During the data acquisition process, operators control the UAV via the ground station control system to hover in a stable airspace 5-8 meters directly in front of the target insulator. This distance range ensures that the image resolution meets the requirements for subsequent feature extraction while avoiding incomplete target capture due to excessive proximity or blurred details due to excessive distance. The UAV's built-in synchronization trigger module controls two cameras to simultaneously start shooting, with a synchronization accuracy error of no more than 1ms, ensuring that the timestamps of the visible light and infrared images are completely consistent, eliminating spatiotemporal deviations caused by motion blur. Simultaneously, the UAV maintains its hovering position and angle through its built-in attitude stabilization system (such as a three-axis stabilization gimbal), keeping the body shake within ±0.1° to avoid image blurring or target displacement caused by vibration.

[0026] It should be noted that during the data acquisition process, the drone's visual obstacle avoidance module must be used in real time to avoid obstacles such as conductors, tower crossarms, and vegetation, ensuring that the obstruction area of ​​the insulator target does not exceed 15%. If unavoidable partial obstruction exists, the drone's hovering angle must be adjusted and the image retaken until a complete target image is obtained. In this embodiment, the preferred storage format for the dual-light synchronous images is RAW or PNG. RAW format retains more image details, facilitating subsequent data enhancement processing, while PNG format has lossless compression characteristics, balancing storage efficiency and image quality. This invention does not strictly limit the storage format and can flexibly choose according to actual engineering needs.

[0027] Preferably, during the acquisition process, environmental parameters (such as light intensity, ambient temperature, and wind speed) and basic insulator information (such as voltage level, model, and installation location) can be recorded simultaneously. This information can serve as an auxiliary basis for subsequent data augmentation and model optimization. For example, different data augmentation strategies can be adopted in subsequent steps for images of sunny, strong light and cloudy, weak light scenarios. It is important to emphasize that the core of this step is to ensure the "synchronization" and "consistency" of the dual-light images, that is, time synchronization, consistent viewing angle, and target integrity. The specific parameters in the above implementation process (such as shooting distance, optical axis parallelism deviation, occlusion threshold, etc.) can be adaptively adjusted according to the actual inspection scenario and equipment performance. As long as the effective acquisition of dual-light synchronous images can be achieved, it falls within the protection scope of this invention.

[0028] This step, through standardized equipment configuration, joint calibration, and shooting procedures, acquired dual-light synchronized images that combine spatiotemporal consistency and target integrity. This lays a reliable data foundation for subsequent processing steps such as feature extraction based on insulator geometric priors and cross-modal attention interaction, effectively avoiding the problem of decreased registration accuracy caused by defects in the acquisition process, and ensuring the engineering feasibility and stability of the entire registration method.

[0029] S2: Input the dual-light synchronized image into a pre-trained cross-modal registration model for registration processing to obtain the cross-modal registration result of the dual-light synchronized image.

[0030] In the cross-modal registration scenario of power system insulators, due to the feature heterogeneity caused by the difference in imaging principles between visible light images and infrared images, traditional registration methods based on single-modal features (such as SIFT and ORB) struggle to establish effective correspondences. Registration accuracy is limited to the regional level, failing to meet the pixel-level alignment requirements for zero-value insulator temperature measurement. The core objective of this step is to standardize the registration of the dual-light synchronous images acquired in step S1 using a pre-trained cross-modal registration model. Leveraging the model's adaptability to modal differences and its ability to fuse multi-scale features, accurate alignment of the dual-light images is achieved, providing reliable data support for subsequent zero-value defect detection. It should be noted that the "pre-trained cross-modal registration model" mentioned in this step is trained and optimized based on a cross-modal registration dataset covering multiple voltage levels, insulator types, and environmental scenarios. It possesses strong robustness and scene adaptability, effectively handling complex inspection conditions such as changes in shooting angle, equipment noise, and partial obstruction.

[0031] In this embodiment, the structural design of the cross-modal registration neural network model revolves around the core logic of "modal adaptation - feature interaction - multi-scale fusion - parameter prediction", which specifically addresses the modal heterogeneity problem of two-light images.

[0032] Preferably, the model adopts an end-to-end architecture, which mainly includes four parts: a modal adaptive dual-branch feature extraction module, a cross-modal attention interaction module, an adaptive weight pyramid fusion module, and a registration parameter prediction module.

[0033] The modality-adaptive dual-branch feature extraction module adopts a symmetrical but parameter-independent dual-branch design. Each branch contains four feature extraction stages, outputting feature maps at scales of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 respectively. The visible light branch addresses the need for texture detail extraction. The first stage uses a 3×3 dense convolutional layer (64 kernels) and the ReLU activation function. The second to fourth stages use a combination of "residual blocks + depthwise separable convolutions," with the number of groups in the depthwise separable convolutions set to 8, which reduces computational cost while enhancing texture feature extraction. The infrared branch addresses the need for temperature gradient capture. The first stage uses a 3×3 dilated convolutional layer (dilation rate 2) and the LeakyReLU activation function (negative slope 0.1). The second to fourth stages use a combination of "attention residual blocks + grouped convolutions," with the number of groups in the grouped convolutions set to 4, enhancing the feature response of temperature anomaly regions through a channel attention mechanism. It should be noted that this invention does not limit the specific values ​​of parameters such as the number of convolutional kernels and the number of groups. These values ​​can be adaptively adjusted according to the hardware computing power and registration accuracy requirements, as long as modality-specific feature extraction can be achieved.

[0034] A cross-modal attention interaction module is embedded after each stage of the dual-branch feature extraction to address the feature heterogeneity problem caused by modal differences. Specifically, this module first enhances the visible light modal features and infrared modal features of the same scale through a self-attention mechanism: the self-attention mechanism uses a scaled dot product attention formula to calculate the correlation weight matrix between pixels, highlighting key structures such as insulator outlines and skirt edges in the visible light features, and high-temperature anomaly regions and temperature gradient abrupt regions in the infrared features, while suppressing interference from background-irrelevant information; then, a cross-attention mechanism is used to perform pixel-level correlation analysis on the enhanced features of the two modalities, calculating the correlation weight matrix between the modalities based on matrix multiplication. This matrix has the same dimension as the feature map size, quantifying the correspondence between visible light features and infrared features; finally, based on the correlation weight matrix, the features of the same scale are weighted and fused to generate a cross-modal fused feature map that combines the core information of both modalities, ensuring semantic consistency and alignment accuracy of the features.

[0035] The adaptive weighted pyramid fusion module employs a symmetrical pyramid structure and an adaptive weight allocation mechanism to achieve efficient aggregation of multi-scale features. First, the cross-modal fusion feature maps from the four stages are labeled as follows: (1 / 2 scale, fine-scale edge features) (1 / 4 scale, mid-layer texture features) (1 / 8 scale, high-level semantic features) (1 / 16 scale, global structural features); the weight prediction subnetwork consists of two fully connected layers (256 and 4 neurons respectively) and a Softmax activation function. It takes the pixel variance of the feature maps at each scale (reflecting feature saliency) as input and outputs weight coefficients { , , , },satisfy Initial setup for training =0.4、 =0.25、 =0.2、 =0.15, ensuring the priority of fine-scale edge features; the bidirectional fusion path includes an upsampling branch and a downsampling branch: the upsampling branch pairs... Perform 2x bilinear interpolation upsampling, and then add weighted samples. ( × Residual connections are performed, and after optimization by 3×3 convolutional layers (128 kernels), the data is iteratively upsampled to... Scale, to obtain The downsampling branch performs double max pooling downsampling on F1, and then... ( × Residual connections are performed, and after optimization by a 3×3 convolutional layer, iterative downsampling is performed to the F4 scale to obtain... Finally, Downsampling to 1 / 4 scale Upsampled to a 1 / 4 scale, the channels are stitched together and then dimensionality reduced by a 1×1 convolutional layer (256 kernels) to generate a global fusion feature map, which reduces parameter redundancy while retaining key information at multiple scales.

[0036] The registration parameter prediction module uses a regression head consisting of three fully connected layers. The input is a global fused feature map (256×64×64 dimensions after flattening), and the output is the spatial transformation parameters of the infrared image relative to the visible light image, including the translation amount ( , ), rotation angle (Unit: radians) and scaling factor s, this parameter describes the spatial positional deviation between the two images. The model training process employs a multi-task loss function constraint, with the total loss being a weighted sum of four parts: ① Feature alignment loss (weight 0.2), calculated based on the cosine similarity of cross-modal fused feature maps, constraining the consistency of features between the two modalities; ② Pixel alignment loss (weight 0.5), using mean squared error (MSE) to calculate the pixel grayscale difference between the transformed infrared and visible light images; ③ Smoothing loss (weight 0.1), using L1 regularization to constrain the continuity of spatial transformation parameters, avoiding excessive deformation; ④ Geometric constraint loss (weight 0.2), calculating the periodic deviation of the insulator skirts after registration (the difference between the actual spacing and the prior spacing) and the axisymmetric deviation (the offset between the actual axis of symmetry and the prior axis of symmetry), using L2 loss constraint.

[0037] The model training employs the Adam optimizer with an initial learning rate of 1e-4, which adaptively decays by 0.5 every 20 epochs. The batch size is set to 8, and the number of iterations is 100 epochs. To prevent overfitting, an early stopping strategy is used: training is terminated and the current optimal model parameters are saved when the registration error on the validation set shows no improvement for 10 consecutive epochs. A gradient pruning strategy is also introduced, with a pruning threshold of 1.0, to avoid gradient explosion during training. It should be noted that this invention does not strictly limit the optimizer type, learning rate decay method, batch size, and other training parameters; these can be adjusted based on actual training results, as long as model convergence and accuracy improvement are achieved. After training, the optimal model parameters are saved as a .pb file for quick loading and retrieval during project deployment, ensuring real-time registration requirements in UAV inspection scenarios.

[0038] The registration process is as follows: Figure 2 As shown, Figure 2 The diagram shown illustrates the registration process in one embodiment of the present invention, which includes steps S21-S24: S21: Based on the prior geometric information of the insulator, modal adaptation feature extraction is performed on the visible light image and the infrared image respectively to obtain multi-scale visible light modal features and infrared modal features; Preferably, the prior geometric information of the insulator in this embodiment specifically includes the periodic distribution characteristics and axisymmetric structural characteristics of the target insulator. The periodic distribution characteristics refer to the uniform arrangement of the insulator skirts along the axis, specifically manifested as a fixed ratio between the spacing of adjacent skirts and the skirt diameter to the total length of the insulator. The axisymmetric structural characteristics refer to the symmetrical distribution of the insulator about the central axis, specifically manifested as the symmetrical shape, size, and distribution position of the left and right skirts. Based on the above geometric characteristics, an insulator geometric prior template library is first constructed. The template library pre-stores the periodic distribution parameters (skirt spacing range 50-80mm, skirt diameter to total insulator length ratio 1:3-1:5) and axisymmetric structural constraint equations (using the horizontal or vertical centerline of the image as the axis of symmetry, satisfying...) for different types of insulators (such as XP series porcelain insulators and XWP series anti-pollution flashover insulators). or ,in, (The coordinates are the center coordinates of the axis of symmetry). It should be noted that this invention does not limit the specific parameter values ​​in the template library. They can be expanded and adjusted according to the model and specifications of the insulators in actual application scenarios, as long as they can accurately reflect the periodicity and axisymmetric characteristics of the insulators.

[0039] The specific process of generating the feature guidance map based on the aforementioned template library is as follows: First, the insulator type recognition submodule built into the model (implemented using a lightweight CNN network) is used to identify the insulator type in the input visible light and infrared images. Based on the recognition results, the corresponding geometric prior parameters are retrieved from the template library, and the parameter thresholds are adaptively adjusted in conjunction with the image size to generate a binarized feature guidance map with the same size as the input image. In this map, the pixel value of the target area where the insulator skirt is located is set to 1, and the pixel value of the background areas such as conductors, sky, and towers is set to 0. The "binarized feature guidance map" mentioned here refers to an image containing only two pixel grayscale values ​​(0 and 1). Its core function is to guide the subsequent feature extraction network to focus on the key areas of the insulator through pixel-level region marking, weakening the interference of background irrelevant information. It is the core carrier of geometric prior information injection in this step.

[0040] When fusing the feature-guided image with the original image, an element-wise addition fusion method is used. The specific operation is as follows: Let the original visible light image be... The corresponding feature guidance map is The first visible light image obtained after fusion is ,but Similarly, the original infrared image With feature guidance map The first infrared image is obtained by fusion. ,in, These are the pixel coordinates of the image. It should be noted that before fusion, the pixel values ​​of the feature guidance map need to be normalized to the same grayscale range as the original image (e.g., [0, 255]) to ensure that the fusion process does not destroy the feature information of the original image, while allowing the geometric prior information to effectively guide the network to focus on the target region. The advantage of this fusion method is that it can achieve an organic combination of geometric prior and original image features without complex parameter calculations, balancing fusion efficiency and effectiveness.

[0041] After feature fusion, a parameter-independent dual-branch feature extraction structure is used to extract features at multiple scales from the first visible light image and the first infrared image respectively. The dual-branch structure is symmetrical, but the convolution kernel weights, biases, and other parameters are initialized and iteratively updated completely independently to ensure that each branch adapts to the feature extraction requirements of the corresponding modality. In this embodiment, each branch contains four progressive feature extraction stages, which sequentially output feature maps at scales of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image. The specific design of each stage is as follows: The first stage (outputting 1 / 2 scale feature maps): The visible light branch uses a 3×3 dense convolutional layer (64 kernels, stride 2) and the ReLU activation function. Dense convolution enhances the feature interaction between adjacent pixels, focusing on extracting low-level fine-scale edge features such as the edge of the insulator skirt and the outline of metal accessories. The infrared branch uses a 3×3 dilated convolutional layer (dilution rate 2, 64 kernels, stride 2) and the LeakyReLU activation function (negative slope 0.1). Dilated convolution expands the receptive field without increasing parameters or computational cost, making it more suitable for capturing the global distribution features of temperature gradients in infrared images, and also outputs fine-scale edge features. It should be noted that this invention does not strictly limit parameters such as the number of kernels and the dilation rate, and can adaptively adjust them according to hardware computing power and registration accuracy requirements, as long as effective extraction of low-level features can be achieved.

[0042] The second to fourth stages (outputting feature maps at 1 / 4, 1 / 8, and 1 / 16 scales respectively): The visible light branch adopts a combination structure of "residual block + depthwise separable convolution". The residual block solves the gradient vanishing problem in deep networks through cross-layer connections. The depthwise separable convolution splits the standard convolution into depthwise convolution and pointwise convolution, reducing computational cost while focusing on extracting mid-layer local structural features (such as umbrella skirt texture and accessory connection structure) and high-layer global contour features (overall insulator morphology and umbrella skirt arrangement). The infrared branch adopts a combination structure of "attention residual block + grouped convolution". The attention residual block enhances the feature response of temperature anomaly regions (potential regions of zero-value insulators) through channel attention mechanism. The grouped convolution groups the feature maps by channel and then convolves them separately to improve the extraction accuracy of temperature gradient features, corresponding to the output of mid-layer local structural features (temperature gradient abrupt change regions) and high-layer global contour features (overall temperature field distribution).

[0043] S22: Perform cross-modal attention interaction processing on the visible light modal features and the infrared modal features at the same scale to generate multi-scale cross-modal fusion features; In the cross-modal registration process of insulator dual-light images, the feature heterogeneity caused by modal differences is a core technical challenge. Visible light modal features are mainly high-frequency textures and geometric contours, while infrared modal features are mainly low-frequency temperature gradients and thermal radiation distributions. The two lack direct semantic correlation, and traditional fusion methods, which rely solely on simple stitching or weighted summation, struggle to establish accurate inter-modal correspondences, thus limiting subsequent registration accuracy. This step employs same-scale cross-modal attention interaction processing. Its core objective is to strengthen the key features of each of the two modes, uncover pixel-level correlations between modes, and generate semantically consistent and accurately aligned cross-modal fusion features, providing high-quality feature support for multi-scale fusion and registration parameter prediction. It should be noted that this interaction processing is performed separately on the four same-scale feature maps (1 / 2, 1 / 4, 1 / 8, and 1 / 16) output from step S21, ensuring cross-modal alignment of features at each scale and avoiding registration deviations caused by single-scale interaction.

[0044] Preferably, in this embodiment, the feature enhancement processing employs a self-attention mechanism. By focusing on key features within a mode and suppressing irrelevant background interference, the discriminability of the features is improved. Specifically, for visible light mode feature maps of the same scale, the self-attention mechanism calculates the association weight between each pixel and all other pixels in the feature map, highlighting key structural features such as the edge of the insulator skirt, the outline of the metal attachment, and the skirt texture, while weakening the feature response of background areas such as conductors and the sky. For infrared mode feature maps of the same scale, the self-attention mechanism specifically enhances the feature weights of high-temperature anomaly regions (potential locations of zero-value insulators) and regions with abrupt temperature gradient changes, reducing false feature interference caused by equipment thermal noise and stray light from the environment. It should be noted that the present invention does not limit the specific implementation method of the self-attention mechanism; scaling dot product attention, multi-head attention, axial attention, etc., can be used, as long as they can achieve the enhancement of key features within the mode.

[0045] After feature enhancement, the first visible light modal feature and the first infrared modal feature are obtained. Then, pixel-level correlation analysis is performed based on a cross-attention mechanism to uncover the potential correspondence between the two modal features. The core logic of the cross-attention mechanism is to use the feature of one modality as the query and the feature of the other modality as the key and value, quantifying the degree of correlation between the two through matrix operations. In this embodiment, the cross-attention processing is divided into two parallel branches: the first branch uses the feature generated from the first visible light modal feature... For querying, generated using the first infrared mode feature For key, The value is used to calculate the correlation weight of the infrared mode relative to the visible light mode; the second branch is generated using the features of the first infrared mode. For querying, generated using the first visible light mode feature For key, The correlation weights of the visible light mode relative to the infrared mode are calculated using the given values. The correlation weights of the two branches are then averaged to generate an inter-modal correlation weight matrix with dimensions consistent with the feature map size. Each element in the matrix The value range is [0,1]. The larger the value, the higher the correlation between the (x,y) position in the visible light feature map and the corresponding position in the infrared feature map. It should be noted that in this embodiment, the generation method of the Q, K, and V matrices is consistent with the self-attention mechanism, which is implemented through 1×1 convolutional layer mapping. Moreover, the convolution kernel parameters are independent of the self-attention mechanism to ensure the specificity of cross-attention.

[0046] The fusion process based on the correlation weight matrix employs a weighted summation method to achieve deep fusion of dual-modal features at the same scale. The specific calculation formula is as follows: ,in, For the generated cross-modal fusion feature map, This is the feature map of the first visible light mode. Let W be the first infrared modal feature map, and W be the inter-modal correlation weight matrix, where "×" indicates element-wise multiplication. This formula prioritizes retaining the common features of both modalities at pixel locations with high correlation, while pixel locations with low correlation adaptively balance the specific features of the two modalities according to the weights. This ensures that the fused feature map includes both visible light texture structure information and infrared temperature distribution information. In this embodiment, the fused feature map needs to be passed through a 3×3 convolutional layer (the number of convolutional kernels is equal to the number of input features). Figure 1 The BatchNorm layer is optimized to eliminate feature noise that may be generated during the fusion process, thereby improving the smoothness and consistency of features.

[0047] S23: Perform multi-scale fusion processing on the cross-modal fusion features to generate global fusion features; In cross-modal registration of insulator dual-light images, effective fusion of multi-scale features is crucial for improving registration accuracy. Existing technologies, such as traditional pyramid fusion methods, often employ fixed paths and uniform weight allocation strategies. This fails to dynamically adjust fusion weights based on the saliency of features at different scales, leading to an imbalance in the importance of fine-scale edge features (the core of pixel-level alignment) and coarse-scale global features. Furthermore, it suffers from parameter redundancy and low fusion efficiency, making it difficult to fully exploit the complementary value of multi-scale features. This step, through a design combining feature saliency analysis, a bidirectional fusion path, and adaptive weights, aims to accurately quantify the contribution of features at each scale, efficiently aggregate multi-scale cross-modal fusion features, and generate global fusion features that combine detailed and global information, providing high-quality input for subsequent registration parameter prediction.

[0048] Preferably, in this embodiment, the multi-scale cross-modal fusion features output in step S22 are first subjected to feature saliency analysis. The core of this analysis is to quantify the contribution of each scale feature to the registration result, providing a basis for weight allocation. The multi-scale cross-modal fusion features include four scales: (1 / 2 scale, mainly focusing on fine-scale edge and contour features) (1 / 4 scale, mainly featuring mid-layer textures and local structural features) (1 / 8 scale, mainly focusing on high-level semantic features) (1 / 16 scale, primarily focusing on global structure and temperature distribution features). Feature saliency analysis is achieved by calculating the pixel variance of the feature maps at each scale. A larger variance indicates more significant gray-level differences at that scale, contains more key information (such as edges and temperature anomaly regions), and contributes more to the registration results. It should be noted that this invention does not limit the specific implementation method of feature saliency analysis; in addition to pixel variance, information entropy, gradient magnitude, and other indicators can also be used, as long as they can accurately quantify feature saliency.

[0049] Based on the feature saliency analysis results, the attention weight prediction subnetwork generates the weight coefficients corresponding to each scale. , , , The subnetwork consists of two fully connected layers and a Softmax activation function: the first fully connected layer maps the input pixel variance (dimension 1) to a 256-dimensional feature vector, the second fully connected layer further maps it to a 4-dimensional vector, and finally the Softmax activation function is used for normalization to ensure that the sum of the weight coefficients satisfies the condition. In the initial training phase, to prioritize the fine-scale edge feature weights required for pixel-level alignment, the initial weight coefficients are set to [value missing]. =0.4、 =0.25、 =0.2、 =0.15; During model training, the parameters of the weight prediction subnetwork are adaptively updated through backpropagation, enabling the network to automatically focus on the scale features that contribute the most to the current registration scene and dynamically optimize the weight allocation strategy.

[0050] Subsequently, based on the bidirectional fusion path and the aforementioned weight coefficients, the multi-scale cross-modal fusion features are weighted and aggregated. The bidirectional fusion path includes an upsampling fusion branch and a downsampling fusion branch, which are executed in parallel to achieve deep complementarity of multi-scale features.

[0051] Upsampling fusion branch: Iteratively fuses global contour features to fine-scale edge features. First, it... (1 / 16 scale) Perform 2x bilinear interpolation upsampling to make its scale consistent with... (1 / 8 scale) consistent, then with the weighted ( × Residual connections are performed. These connections avoid gradient vanishing through a skip-layer structure, ensuring effective transfer of deep features. Next, a 3×3 convolutional layer (with 128 kernels) optimizes the fused features, eliminating the blurring effect caused by upsampling. This process is repeated, upsampling the optimized feature map by a factor of 2 again, and then combining it with the weighted feature map. ( × Residual connections and convolution optimization are performed, and finally iterated to the point where... (1 / 2 scale) alignment yields upsampled path output features. .

[0052] Downsampling fusion branch: Iteratively fuses features from fine-scale edge features to global contour features. First, it... Perform double max pooling downsampling at (1 / 2 scale) to make its scale equal to... (1 / 4 scale) consistent, then with the weighted ( × Residual connections are performed; after optimization with a 3×3 convolutional layer (kernel count set to 128), it is further downsampled by 2 times and then combined with the weighted... ( × The residual connection convolution is finally downsampled to the same level as the input. Alignment at a 1 / 16 scale yields the output features of the downsampled path. It should be noted that this invention does not impose strict limitations on interpolation methods (such as bilinear interpolation, which can be replaced by bicubic interpolation), pooling methods (such as max pooling, which can be replaced by average pooling), and the number of convolutional kernels, which can be adjusted according to hardware performance and registration accuracy requirements.

[0053] Finally, the output features of the two fusion paths are concatenated to generate global fusion features. The specific steps are as follows: The output features of the upsampled path are... Perform double max pooling downsampling at (1 / 2 scale) to adjust to 1 / 4 scale; output features from the downsampling path. (1 / 16 scale) Perform 2x bilinear interpolation upsampling, also adjusted to 1 / 4 scale. The 1 / 4 scale was chosen as the stitching reference because this scale combines mid-level texture and local structural information, balancing the integrity of fine-scale details and coarse-scale global information, while avoiding computational redundancy caused by excessively large scales. The adjusted... and Channel concatenation is performed (the number of channels in the concatenated feature map is the sum of the number of channels in the two paths), followed by dimensionality reduction through a 1×1 convolutional layer (with 256 kernels) to eliminate parameter redundancy caused by channel concatenation, and finally a global fusion feature map containing key information at multiple scales (fine-scale edges, mid-level textures, high-level semantics, and global structure) is generated.

[0054] S24: Based on the global fusion features, predict the spatial transformation parameters, and perform geometric transformation on the infrared image according to the spatial transformation parameters to obtain the registration result of the dual-light synchronous image.

[0055] In this embodiment, the global fused feature map is input into a regression head consisting of three fully connected layers to predict spatial transformation parameters. The first fully connected layer of the regression head flattens the global fused feature map (1 / 4 scale, 256 channels) into a one-dimensional vector and maps it to a 1024-dimensional feature space; the second fully connected layer further maps it to 256 dimensions; the third fully connected layer outputs the final spatial transformation parameters, including translation, rotation angle, value range, and scaling ratio. This parameter set is used to accurately describe the spatial positional deviation of the infrared image relative to the visible light image, providing a quantitative basis for subsequent geometric transformations. It should be noted that this invention does not limit the network structure of the regression head; the number of fully connected layers can be increased or decreased according to the registration accuracy requirements, as long as the spatial transformation parameters can be accurately predicted.

[0056] An affine transformation matrix is ​​constructed based on the predicted spatial transformation parameters to perform geometric transformations on the infrared image. The expression for the affine transformation matrix is: in, These are the original pixel coordinates of the infrared image. The coordinates are the transformed pixel coordinates. Bilinear interpolation is used to calculate the grayscale value of the transformed pixel. This method ensures a smooth, jagged image after transformation by weighted averaging of the grayscale values ​​of four adjacent pixels, avoiding pixel distortion that could affect registration accuracy. In this embodiment, the pixel accuracy error of the geometric transformation is controlled within 0.1 pixels. This invention does not limit the interpolation method; in addition to bilinear interpolation, bicubic interpolation can also be used, as long as it achieves a smooth image transformation.

[0057] After the geometric transformation is completed, the cross-modal registration result of the dual-light synchronous image is output, which is the infrared image pixel-level aligned with the visible light image. The registration result must meet two core indicators: first, the registration error is less than 1.2 pixels; second, the alignment deviation of key parts such as the insulator skirt area and metal accessories does not exceed 1 pixel, ensuring the accuracy of subsequent temperature field and appearance feature fusion analysis. This step improves the model's generalization ability through data augmentation and the construction of a high-quality dataset; ensures the accuracy of parameter prediction through a multi-task loss function and a scientific training strategy; and guarantees the integrity and consistency of the registration result through a smooth geometric transformation algorithm. Ultimately, high-precision registration of dual-light images in complex scenarios is achieved, providing reliable data support for the accurate detection of zero-value insulators.

[0058] Another embodiment of the present invention provides a cross-modal registration system for dual-light images of insulators. For details, please refer to [link to relevant documentation]. Figure 3 , Figure 3 The diagram shown is a structural block diagram of an insulator dual-light image cross-modal registration system according to one embodiment of the present invention, which includes: The acquisition module 11 is used to acquire a dual-light synchronous image of the target insulator, wherein the dual-light synchronous image includes a visible light image and an infrared image; The registration module 12 is used to input the dual-light synchronized image into a pre-trained cross-modal registration model for registration processing, so as to obtain the cross-modal registration result of the dual-light synchronized image.

[0059] The registration process is as follows: Figure 4 As shown, Figure 4 The diagram shown is a structural block diagram of a registration module according to one embodiment of the present invention, which includes: Extraction unit 21 is used to perform mode-adaptive feature extraction on the visible light image and the infrared image based on the insulator geometric prior information, respectively, to obtain multi-scale visible light mode features and infrared mode features; The cross-modal fusion unit 22 is used to perform cross-modal attention interaction processing on the visible light modal features and the infrared modal features at the same scale to generate multi-scale cross-modal fusion features; The multi-scale fusion unit 23 is used to perform multi-scale fusion processing on the cross-modal fusion features to generate global fusion features; The generation unit 24 is used to predict spatial transformation parameters based on the global fusion features, perform geometric transformation on the infrared image according to the spatial transformation parameters, and generate the registration result of the dual-light synchronous image.

[0060] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for cross-modal registration of two-light images of insulators, characterized in that, include: Acquire a dual-light synchronized image of the target insulator, wherein the dual-light synchronized image includes a visible light image and an infrared image; The dual-light synchronized image is input into a pre-trained cross-modal registration model for registration processing to obtain the cross-modal registration result of the dual-light synchronized image; The registration process includes: Based on the insulator's geometric prior information, modal adaptation feature extraction is performed on the visible light image and the infrared image respectively to obtain multi-scale visible light modal features and infrared modal features; The visible light modal features and the infrared modal features are subjected to cross-modal attention interaction processing at the same scale to generate multi-scale cross-modal fusion features; The cross-modal fusion features are subjected to multi-scale fusion processing to generate global fusion features; Based on the global fusion features, spatial transformation parameters are predicted, and the infrared image is geometrically transformed according to the spatial transformation parameters to obtain the registration result of the dual-light synchronized image.

2. The insulator two-light image cross-modal registration method as described in claim 1, characterized in that, The prior geometric information of the insulator includes the periodic distribution characteristics and axisymmetric structural characteristics of the target insulator; The modal adaptation feature extraction is performed on the visible light image and the infrared image based on the insulator geometric prior information to obtain multi-scale visible light modal features and infrared modal features, including: Based on the periodic distribution characteristics and the axisymmetric structural characteristics, a feature guidance map of the target insulator is generated; The feature guidance map is fused with the visible light image and the infrared image respectively to obtain a first visible light image and a first infrared image; Based on a parameter-independent dual-branch feature extraction structure, multi-scale feature extraction is performed on the first visible light image and the first infrared image to obtain visible light modal features and infrared modal features, wherein the visible light modal features and the infrared modal features respectively include at least fine-scale edge features, local structural features and global contour features.

3. The insulator two-light image cross-modal registration method as described in claim 1, characterized in that, The step of performing cross-modal attention interaction processing on the visible light modal features and the infrared modal features at the same scale to generate multi-scale cross-modal fusion features includes: The visible light mode feature and the infrared mode feature are respectively subjected to feature enhancement processing to obtain the first visible light mode feature and the first infrared mode feature; Pixel-level correlation analysis is performed on the first visible light modal features and the first infrared modal features based on the cross-attention mechanism, and a correlation weight matrix between modalities is generated based on the analysis results; Based on the correlation weight matrix, the first visible light modal features and the first infrared modal features of the same scale are fused to generate multi-scale cross-modal fusion features.

4. The insulator two-light image cross-modal registration method as described in claim 1, characterized in that, The step of performing multi-scale fusion processing on the cross-modal fusion features to generate global fusion features includes: The cross-modal fusion features at each scale are subjected to feature saliency analysis, and weight coefficients corresponding to the cross-modal fusion features at the corresponding scale are generated based on the analysis results; wherein, the feature saliency analysis is used to quantify the contribution of features at each scale to the registration results; Based on the bidirectional fusion path and the weight coefficients, the multi-scale cross-modal fusion features are weighted and aggregated to obtain the output features corresponding to the bidirectional fusion path respectively. The bidirectional fusion path includes an upsampling fusion branch that iteratively fuses from global contour features to fine-scale edge features, and a downsampling fusion branch that iteratively fuses from the fine-scale edge features to the global contour features. The first output feature of the upsampling fusion branch and the second output feature of the downsampling fusion branch are concatenated to generate the global fusion feature.

5. The method for cross-modal registration of two-light images of insulators as described in claim 1, characterized in that, Before inputting the dual-light synchronized image into a pre-trained cross-modal registration model for registration processing, the method further includes: Data enhancement processing is performed on the dual-light synchronized image to obtain a first dual-light synchronized image; wherein, the data enhancement processing is designed to enhance the texture details of the visible light image and optimize the contrast of the infrared image; A cross-modal registration dataset is constructed based on the first dual-light synchronized image, wherein each sample in the dataset contains a pair of visible light images and infrared images of insulators; The initial cross-modal registration model is trained and optimized based on the cross-modal registration dataset to obtain the pre-trained cross-modal registration model.

6. A cross-modal registration system for dual-light images of insulators, characterized in that, include: The acquisition module is used to acquire a dual-light synchronous image of the target insulator, wherein the dual-light synchronous image includes a visible light image and an infrared image; The registration module is used to input the dual-light synchronized image into a pre-trained cross-modal registration model for registration processing, so as to obtain the cross-modal registration result of the dual-light synchronized image; The registration process includes: The extraction unit is used to perform mode-adaptive feature extraction on the visible light image and the infrared image based on the insulator geometric prior information, respectively, to obtain multi-scale visible light mode features and infrared mode features; A cross-modal fusion unit is used to perform cross-modal attention interaction processing on the visible light modal features and the infrared modal features at the same scale to generate multi-scale cross-modal fusion features; A multi-scale fusion unit is used to perform multi-scale fusion processing on the cross-modal fusion features to generate global fusion features; The generation unit is used to predict spatial transformation parameters based on the global fusion features, perform geometric transformation on the infrared image according to the spatial transformation parameters, and generate the registration result of the dual-light synchronized image.

7. The insulator dual-light image cross-modal registration system as described in claim 6, characterized in that, The prior geometric information of the insulator includes the periodic distribution characteristics and axisymmetric structural characteristics of the target insulator; The extraction unit is specifically used for: Based on the periodic distribution characteristics and the axisymmetric structural characteristics, a feature guidance map of the target insulator is generated; The feature guidance map is fused with the visible light image and the infrared image respectively to obtain a first visible light image and a first infrared image; Based on a parameter-independent dual-branch feature extraction structure, multi-scale feature extraction is performed on the first visible light image and the first infrared image to obtain visible light modal features and infrared modal features, wherein the visible light modal features and the infrared modal features respectively include at least fine-scale edge features, local structural features and global contour features.

8. The insulator dual-light image cross-modal registration system as described in claim 6, characterized in that, The cross-modal fusion unit is specifically used for: The visible light mode feature and the infrared mode feature are respectively subjected to feature enhancement processing to obtain the first visible light mode feature and the first infrared mode feature; Pixel-level correlation analysis is performed on the first visible light modal features and the first infrared modal features based on the cross-attention mechanism, and a correlation weight matrix between modalities is generated based on the analysis results; Based on the correlation weight matrix, the first visible light modal features and the first infrared modal features of the same scale are fused to generate multi-scale cross-modal fusion features.

9. The insulator dual-light image cross-modal registration system as described in claim 6, characterized in that, The multi-scale fusion unit is specifically used for: The cross-modal fusion features at each scale are subjected to feature saliency analysis, and weight coefficients corresponding to the cross-modal fusion features at the corresponding scale are generated based on the analysis results; wherein, the feature saliency analysis is used to quantify the contribution of features at each scale to the registration results; Based on the bidirectional fusion path and the weight coefficients, the multi-scale cross-modal fusion features are weighted and aggregated to obtain the output features corresponding to the bidirectional fusion path respectively. The bidirectional fusion path includes an upsampling fusion branch that iteratively fuses from global contour features to fine-scale edge features, and a downsampling fusion branch that iteratively fuses from the fine-scale edge features to the global contour features. The first output feature of the upsampling fusion branch and the second output feature of the downsampling fusion branch are concatenated to generate the global fusion feature.

10. The insulator dual-light image cross-modal registration system as described in claim 6, characterized in that, The registration module is also used for: Data enhancement processing is performed on the dual-light synchronized image to obtain a first dual-light synchronized image; wherein, the data enhancement processing is designed to enhance the texture details of the visible light image and optimize the contrast of the infrared image; A cross-modal registration dataset is constructed based on the first dual-light synchronized image, wherein each sample in the dataset contains a pair of visible light images and infrared images of insulators; The initial cross-modal registration model is trained and optimized based on the cross-modal registration dataset to obtain the pre-trained cross-modal registration model.

Images