Ultraviolet-visible-infrared three-spectrum image adaptive fusion method

Abstract

The invention relates to an ultraviolet-visible-infrared tri-spectral image adaptive fusion method, which comprises the following steps: acquiring a tri-spectral image of high-voltage equipment as an original image, and preprocessing to obtain a preprocessed image; constructing a feature extraction calculation network to obtain feature vectors of the three-spectrum image; calculating the contribution weight of each image in the fusion result; and carrying out multi-scale weighted fusion on the three-spectrum image, and carrying out post-processing on the final fused image. According to the method, the information integrity and accuracy are remarkably improved, the characteristics of a target under different spectrums can be obtained, the problem that information of a single spectrum is one-sided is solved, and tiny flaws which are difficult to distinguish by naked eyes and are fuzzy in thermal imaging can be accurately positioned in industrial defect detection; the imaging of the visible light spectrum is limited in a severe environment, and the ultraviolet and infrared spectrums can still work in low-light, smog, haze and other scenes; after fusion, the system can stably acquire images in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an adaptive fusion method for ultraviolet-visible-infrared three-spectral images. Background Technology

[0002] Ultraviolet (UV) images can capture the radiation characteristics of materials in the UV band and are often used to detect corona discharge in power equipment and initial flames in fires. Visible light images are consistent with human visual perception, have rich color and texture information, and can provide clear details of the appearance of objects. Infrared images can reflect the temperature distribution on the surface of objects and have unique advantages in night monitoring, human body temperature detection, and mechanical fault diagnosis.

[0003] By fusing ultraviolet, visible, and infrared images, the advantages of the three spectral types can be combined to obtain an image containing more information, thereby improving the accuracy and reliability of target detection, recognition, and analysis. Therefore, multispectral image fusion technology has important application value in many fields.

[0004] Currently, multispectral image adaptive fusion methods mainly include four types: multi-scale transformation, deep learning, model-driven, and attention mechanisms. Among them, multi-scale transformation, such as NSCT, can capture edge textures but high-frequency information is easily lost and computation is complex; deep learning, such as CNN and Transformer architectures, requires a large amount of labeled data and has obvious black-box characteristics and weak generalization; model-driven, such as HMPNet, is parameter sensitive and requires manual design of priors; attention mechanisms, such as MFTNet, have strong dynamic weight adjustment capabilities but consume large amounts of computational resources and are sensitive to noise. Summary of the Invention

[0005] To address the information limitations of single-spectral imaging, the present invention aims to provide an adaptive fusion method for ultraviolet-visible-infrared three-spectral images that can significantly improve information integrity and accuracy, enhance environmental adaptability and robustness, and expand application areas and functions.

[0006] To achieve the above objectives, the present invention employs the following technical solution: an adaptive fusion method for ultraviolet-visible-infrared three-spectral images, the method comprising the following sequential steps:

[0007] (1) Acquire the three-spectral image of the high-voltage equipment as the original image, wherein the three-spectral image includes an ultraviolet image, an infrared image and a visible light image;

[0008] (2) Preprocess the ultraviolet image, infrared image and visible light image respectively to obtain the preprocessed image;

[0009] (3) Construct a feature extraction computing network, extract the feature information of the preprocessed image through the feature extraction computing network, and perform image feature calculation to obtain the feature vector of the three-spectral image;

[0010] (4) Calculate the contribution weight of each image in the fusion result based on the feature vectors of the three-spectral images and the feature information of environmental parameters;

[0011] (5) Perform multi-scale weighted fusion of the three-spectral images according to the contribution weights to obtain the final fused image;

[0012] (6) Post-process the final fused image, including contrast enhancement, spectral consistency correction and image sharpening.

[0013] Step (1) specifically refers to: using an ultraviolet imager, an infrared thermal imager, and a high-definition visible light camera to acquire images of the high-voltage equipment, acquiring three-spectral images in different scenarios, with an image resolution of 1920×1080 pixels. The different scenarios include indoor, outdoor, nighttime, and daytime. The ultraviolet imager is used to capture the ultraviolet light signal generated by corona discharge on the surface of the equipment. The infrared thermal imager obtains the temperature distribution image of the equipment by detecting the thermal radiation on the surface of the equipment. The high-definition visible light camera is used to record the appearance details of the high-voltage equipment.

[0014] In step (2), the preprocessing includes image registration and noise suppression. The image registration refers to geometric registration using an ORB feature point matching method. Matching point pairs are selected by calculating the Hamming distance between feature points, and a homography matrix is ​​constructed to realize the geometric transformation of the image. The noise suppression refers to using median filtering for ultraviolet images, nonlocal mean denoising algorithm for visible light images, and bilateral filtering algorithm for infrared images.

[0015] In step (3), the construction of the feature extraction computing network and the extraction of feature information of the preprocessed image through the feature extraction computing network specifically refers to: designing a feature extraction computing network based on the characteristics of the three-spectral images to extract feature information of the visible light image, infrared image, and ultraviolet image respectively; the feature extraction computing network is based on the ResNet network as the backbone, combined with the Canny algorithm and the LBP algorithm, and for the ultraviolet and infrared spectra, a spectral feature enhancement module is added to the input end of the ResNet network;

[0016] For visible light images, extract semantic features, edge and texture detail features; calculate the mean feature of the RGB color space;

[0017] For ultraviolet images, morphological operations are used to enhance the contour of the ultraviolet radiation region and extract geometric features such as contour perimeter, area, and eccentricity; a distribution histogram of ultraviolet radiation intensity is calculated to obtain the statistical characteristics of ultraviolet radiation intensity.

[0018] For infrared images, based on the temperature data of the images, global statistical features such as temperature mean, variance, and extreme values ​​are extracted; the difference of Gaussian operator is used to detect hot edges and obtain the location and intensity information of hot edges; high-temperature regions are extracted through threshold segmentation algorithm and the area and shape features of high-temperature regions are analyzed.

[0019] To address the unique characteristics of ultraviolet and infrared spectra, a spectral enhancement module was designed to enhance the feature representation capabilities of specific spectral regions through learnable spectral response adjustments.

[0020] In step (3), the image feature calculation specifically includes the following sequential steps:

[0021] (3a) Entropy calculation:

[0022] ;

[0023] Where H represents the entropy value, Let i be the probability density of pixel value i within the local window. Grayscale =256, for each pixel in each spectral image, calculate the neighborhood entropy value of each pixel to generate an entropy map. , It is a spectral image;

[0024] (3b) Energy calculation:

[0025] ;

[0026] Where E represents energy, N is the number of pixels in the window, Ω is the local window, and f b (x', y') is the spectral image The feature value at pixel coordinates (x', y'); generating an energy map. This characterizes the local energy distribution of each spectral image;

[0027] (3c) Feature vector calculation: For each pixel coordinate The feature vector of its local region is defined as:

[0028] ;

[0029] in, In the spectral image In pixel coordinates The feature vector at that location, In spectral images In pixel coordinates Entropy diagram at the location, In spectral images In pixel coordinates The energy map at the location, where T is the ambient temperature, U is the ultraviolet radiation intensity, and L is the light intensity, realizes the fusion of environmental parameters and image features.

[0030] Step (4) specifically includes the following steps in sequence:

[0031] (4a) Calculate the characteristic Euclidean distance:

[0032] ;

[0033] In the formula, Representing spectral images With spectral images In pixel coordinates The characteristic Euclidean distance at the location. Representing spectral images In pixel coordinates The feature vector at that location, Representing spectral images In pixel coordinates The eigenvector at that location;

[0034] Calculate the characteristic cosine distance:

[0035] ;

[0036] in, Representing spectral images With spectral images In pixel coordinates The characteristic cosine distance at point [-1, 1] takes values ​​in the range [-1, 1].

[0037] (4b) Perform normalization: normalize the coordinates of each pixel. Calculate the pairwise similarity between all spectral images:

[0038] ;

[0039] in, Representing spectral images With spectral images In pixels The result after normalizing the characteristic cosine distance at the location; It is the maximum value among all possible Euclidean distances within the entire image region;

[0040] (4c) The formula for calculating the normalized weights based on similarity is as follows:

[0041] ;

[0042] in, Representing spectral images In pixel coordinates Position-adaptive weights B represents the spectral image that is not currently being weighted; B is the total number of spectral images.

[0043] (4d) When When 'b' is set to VIS, IR, or UV, the resulting image (visible light, infrared, and ultraviolet) in pixel coordinates before correction can be obtained. Contribution weight at each location , , Introducing environmental parameters as weight correction terms, the contribution weight of each spectral image at each pixel coordinate is calculated after correction:

[0044] ;

[0045] ;

[0046] ;

[0047] in, , , These represent the corrected visible light, infrared, and ultraviolet images in pixel coordinates, respectively. The contribution weight at each location; L is the light intensity, T is the ambient temperature, and U is the ultraviolet radiation intensity; , , The adjustment coefficients are optimized using a genetic algorithm. , , These represent the average, maximum, and minimum values ​​of light intensity, respectively. , , These represent the average, maximum, and minimum ambient temperatures, respectively. , , These represent the average, maximum, and minimum values ​​of ultraviolet radiation intensity, respectively.

[0048] Step (5) specifically includes the following steps in sequence:

[0049] (5a) Non-subsampled contour wave transform (NSCT) is used to decompose the preprocessed trispectral image into a low-frequency sub-band and multiple high-frequency sub-bands. The low-frequency sub-band contains the main structure and background information of the image, while the high-frequency sub-band contains the edge and texture information of the image.

[0050] (5b) Determine the fusion strategy for low-frequency subbands based on contribution weights: In nighttime or low-light environments, use the low-frequency subbands of infrared images as the main component, combined with the weighted information of the low-frequency subbands of visible and ultraviolet images; In daytime environments with sufficient light, use the low-frequency subbands of visible images as the main component, combined with the effective information of the low-frequency subbands of infrared and ultraviolet images; When an ultraviolet radiation anomaly is detected, increase the weight ratio of the low-frequency subbands of the ultraviolet image.

[0051] (5c) For the high-frequency subband, weighted fusion is performed according to the adaptive weight of each spectral image. At the same time, guided filtering is introduced to optimize the fused high-frequency subband. The high-frequency subband of the visible light image is used as the guided image to enhance the detail clarity and edge sharpness of the fused image.

[0052] (5d) Using NSCT inverse transform, the fused low-frequency subband and high-frequency subband are reconstructed into the final fused image.

[0053] Step (6) specifically includes the following steps in sequence:

[0054] (6a) Contrast enhancement: The multi-scale MSR algorithm based on Retinex theory is used to enhance the local and global contrast of the preliminary fused image, thereby improving the visual effect and detail recognition of the image;

[0055] (6b) Spectral consistency correction: The final fused image is converted from the RGB color space to the Lab color space. By adjusting the distribution of the brightness L, green red, blue yellow channels, the spectral consistency of the image is corrected to reduce the color shift problem generated during the fusion process. Finally, the image is converted back to the RGB color space.

[0056] (6c) Image sharpening: The image is sharpened using an unsharpened mask algorithm to enhance the image edges and make the fused image clearer.

[0057] The processing of the spectral feature enhancement module includes the following steps:

[0058] (8a) Spectral response adjustment: For the input feature map X∈R C×H1×W Perform channel-level adjustments to generate a spectral weighting map. :

[0059] ;

[0060] Where σ is the Sigmoid function, W1 and b3 are the learnable convolution kernel and bias, respectively; C represents the number of channels, H1 is the height of the input image, and W is the width of the input image;

[0061] (8b) Spectral feature enhancement: Enhance the input feature map X using the spectral weight map:

[0062] ;

[0063] in, This is the feature map after spectral enhancement. For enhancement coefficient;

[0064] (8c) Add a channel attention mechanism: Recalibrate the importance of channels using global information to generate channel attention weights. :

[0065] ;

[0066] in, Indicates global average pooling. and All are convolution kernels. It is a bias; It is an activation function;

[0067] (8d) Apply channel attention weights to the spectrally enhanced features:

[0068] ;

[0069] Where Y represents the final enhanced image features.

[0070] As can be seen from the above technical solution, the beneficial effects of the present invention are as follows: First, the present invention significantly improves the integrity and accuracy of information: the three spectral characteristics are complementary; ultraviolet spectrum can detect ultraviolet radiation, visible light spectrum can present rich colors and details, and infrared spectrum can sense thermal radiation; through fusion, the characteristics of the target under different spectra can be obtained, solving the problem of one-sided information from a single spectrum. In industrial defect detection, it can accurately locate minute defects that are difficult to distinguish with the naked eye and are blurred by thermal imaging. Second, it enhances environmental adaptability and robustness: visible light spectrum is limited in imaging under harsh environments, while ultraviolet and infrared spectra can still work in low light, smoke, haze and other scenarios; after fusion, the system can be used in complex environments. In complex environments, such as dense smoke at a fire scene, fused images can stably acquire images, preserving the fire source location information from infrared thermal imaging while combining ultraviolet spectroscopy to identify harmful gases, providing a reliable basis for rescue efforts. Third, it expands application areas and functions: fused images integrate multi-source information, promoting the development of fields such as intelligent monitoring of power systems, intelligent security, environmental monitoring, and biomedicine. In power systems, fused images can comprehensively acquire information on equipment appearance defects, discharge conditions, and abnormal temperatures. In security monitoring, fused images can not only identify faces and objects under visible light but also detect abnormal heat sources and chemical substances through infrared and ultraviolet features, achieving multi-dimensional early warning. Attached Figure Description

[0071] Figure 1This is a flowchart of the method of the present invention;

[0072] Figure 2 This is a schematic diagram of multi-scale fusion in this invention. Detailed Implementation

[0073] like Figure 1 As shown, an adaptive fusion method for ultraviolet-visible-infrared three-spectral images includes the following sequential steps:

[0074] (1) Acquire the three-spectral image of the high-voltage equipment as the original image, wherein the three-spectral image includes an ultraviolet image, an infrared image and a visible light image;

[0075] (2) Preprocess the ultraviolet image, infrared image and visible light image respectively to obtain the preprocessed image;

[0076] (3) Construct a feature extraction computing network, extract the feature information of the preprocessed image through the feature extraction computing network, and perform image feature calculation to obtain the feature vector of the three-spectral image;

[0077] (4) Calculate the contribution weight of each image in the fusion result based on the feature vectors of the three-spectral images and the feature information of environmental parameters;

[0078] (5) Perform multi-scale weighted fusion of the three-spectral images according to the contribution weights to obtain the final fused image;

[0079] (6) Post-process the final fused image, including contrast enhancement, spectral consistency correction and image sharpening.

[0080] Step (1) specifically refers to: using an ultraviolet imager, an infrared thermal imager, and a high-definition visible light camera to acquire images of the high-voltage equipment, acquiring three-spectral images in different scenarios, with an image resolution of 1920×1080 pixels. The different scenarios include indoor, outdoor, nighttime, and daytime. The ultraviolet imager is used to capture the ultraviolet light signal generated by corona discharge on the surface of the equipment. The infrared thermal imager obtains the temperature distribution image of the equipment by detecting the thermal radiation on the surface of the equipment. The high-definition visible light camera is used to record the appearance details of the high-voltage equipment.

[0081] In step (2), the preprocessing includes image registration and noise suppression. The image registration refers to geometric registration using an ORB feature point matching method. Matching point pairs are selected by calculating the Hamming distance between feature points, and a homography matrix is ​​constructed to realize the geometric transformation of the image. The noise suppression refers to using median filtering for ultraviolet images, nonlocal mean denoising algorithm for visible light images, and bilateral filtering algorithm for infrared images.

[0082] In step (3), the construction of the feature extraction computing network and the extraction of feature information of the preprocessed image through the feature extraction computing network specifically refers to: designing a feature extraction computing network based on the characteristics of the three-spectral images to extract feature information of the visible light image, infrared image, and ultraviolet image respectively; the feature extraction computing network is based on the ResNet network as the backbone, combined with the Canny algorithm and the LBP algorithm, and for the ultraviolet and infrared spectra, a spectral feature enhancement module is added to the input end of the ResNet network;

[0083] For visible light images, extract semantic features, edge and texture detail features; calculate the mean feature of the RGB color space;

[0084] For ultraviolet images, morphological operations are used to enhance the contour of the ultraviolet radiation region and extract geometric features such as contour perimeter, area, and eccentricity; a distribution histogram of ultraviolet radiation intensity is calculated to obtain the statistical characteristics of ultraviolet radiation intensity.

[0085] For infrared images, based on the temperature data of the images, global statistical features such as temperature mean, variance, and extreme values ​​are extracted; the difference of Gaussian operator is used to detect hot edges and obtain the location and intensity information of hot edges; high-temperature regions are extracted through threshold segmentation algorithm and the area and shape features of high-temperature regions are analyzed.

[0086] To address the unique characteristics of ultraviolet and infrared spectra, a spectral enhancement module was designed to enhance the feature representation capabilities of specific spectral regions through learnable spectral response adjustments.

[0087] In step (3), the image feature calculation specifically includes the following sequential steps:

[0088] (3a) Entropy calculation:

[0089] ;

[0090] Where H represents the entropy value, Let i be the probability density of pixel value i within the local window. Grayscale =256, for each pixel in each spectral image, calculate the neighborhood entropy value of each pixel to generate an entropy map. , It is a spectral image;

[0091] (3b) Energy calculation:

[0092] ;

[0093] Where E represents energy, N is the number of pixels in the window, Ω is the local window, and f b (x', y') is the spectral image The feature value at pixel coordinates (x', y'); generating an energy map. This characterizes the local energy distribution of each spectral image;

[0094] (3c) Feature vector calculation: For each pixel coordinate The feature vector of its local region is defined as:

[0095] ;

[0096] in, In the spectral image In pixel coordinates The feature vector at that location, In spectral images In pixel coordinates Entropy diagram at the location, In spectral images In pixel coordinates The energy map at the location, where T is the ambient temperature, U is the ultraviolet radiation intensity, and L is the light intensity, realizes the fusion of environmental parameters and image features.

[0097] Step (4) specifically includes the following steps in sequence:

[0098] (4a) Calculate the characteristic Euclidean distance:

[0099] ;

[0100] In the formula, Representing spectral images With spectral images In pixel coordinates The characteristic Euclidean distance at the location. Representing spectral images In pixel coordinates The feature vector at that location, Representing spectral images In pixel coordinates The eigenvector at that location;

[0101] Calculate the characteristic cosine distance:

[0102] ;

[0103] in, Representing spectral images With spectral images In pixel coordinates The characteristic cosine distance at point [-1, 1] takes values ​​in the range [-1, 1].

[0104] (4b) Perform normalization: normalize the coordinates of each pixel. Calculate the pairwise similarity between all spectral images:

[0105] ;

[0106] in, Representing spectral images With spectral images In pixels The result after normalizing the characteristic cosine distance at the location; It is the maximum value among all possible Euclidean distances within the entire image region;

[0107] (4c) The formula for calculating the normalized weights based on similarity is as follows:

[0108] ;

[0109] in, Representing spectral images In pixel coordinates Position-adaptive weights B represents the spectral image that is not currently being weighted; B is the total number of spectral images.

[0110] (4d) When When 'b' is set to VIS, IR, or UV, the resulting image (visible light, infrared, and ultraviolet) in pixel coordinates before correction can be obtained. Contribution weight at each location , , Introducing environmental parameters as weight correction terms, the contribution weight of each spectral image at each pixel coordinate is calculated after correction:

[0111] ;

[0112] ;

[0113] ;

[0114] in, , , These represent the corrected visible light, infrared, and ultraviolet images in pixel coordinates, respectively. The contribution weight at each location; L is the light intensity, T is the ambient temperature, and U is the ultraviolet radiation intensity; , , The adjustment coefficients are optimized using a genetic algorithm. , , These represent the average, maximum, and minimum values ​​of light intensity, respectively. , , These represent the average, maximum, and minimum ambient temperatures, respectively. , , These represent the average, maximum, and minimum values ​​of ultraviolet radiation intensity, respectively.

[0115] like Figure 2 As shown, step (5) specifically includes the following steps in sequence:

[0116] (5a) Non-subsampled contour wave transform (NSCT) is used to decompose the preprocessed trispectral image into a low-frequency sub-band and multiple high-frequency sub-bands. The low-frequency sub-band contains the main structure and background information of the image, while the high-frequency sub-band contains the edge and texture information of the image.

[0117] (5b) Determine the fusion strategy for low-frequency subbands based on contribution weights: In nighttime or low-light environments, use the low-frequency subbands of infrared images as the main component, combined with the weighted information of the low-frequency subbands of visible and ultraviolet images; In daytime environments with sufficient light, use the low-frequency subbands of visible images as the main component, combined with the effective information of the low-frequency subbands of infrared and ultraviolet images; When an ultraviolet radiation anomaly is detected, increase the weight ratio of the low-frequency subbands of the ultraviolet image.

[0118] (5c) For the high-frequency subband, weighted fusion is performed according to the adaptive weight of each spectral image. At the same time, guided filtering is introduced to optimize the fused high-frequency subband. The high-frequency subband of the visible light image is used as the guided image to enhance the detail clarity and edge sharpness of the fused image.

[0119] (5d) Using NSCT inverse transform, the fused low-frequency subband and high-frequency subband are reconstructed into the final fused image.

[0120] Step (6) specifically includes the following steps in sequence:

[0121] (6a) Contrast enhancement: The multi-scale MSR algorithm based on Retinex theory is used to enhance the local and global contrast of the preliminary fused image, thereby improving the visual effect and detail recognition of the image;

[0122] (6b) Spectral consistency correction: The final fused image is converted from the RGB color space to the Lab color space. By adjusting the distribution of the brightness L, green red, blue yellow channels, the spectral consistency of the image is corrected to reduce the color shift problem generated during the fusion process. Finally, the image is converted back to the RGB color space.

[0123] (6c) Image sharpening: The image is sharpened using an unsharpened mask algorithm to enhance the image edges and make the fused image clearer.

[0124] The processing of the spectral feature enhancement module includes the following steps:

[0125] (8a) Spectral response adjustment: For the input feature map X∈R C×H1×W Perform channel-level adjustments to generate a spectral weighting map. :

[0126] ;

[0127] Where σ is the Sigmoid function, W1 and b3 are the learnable convolution kernel and bias, respectively; C represents the number of channels, H1 is the height of the input image, and W is the width of the input image;

[0128] (8b) Spectral feature enhancement: Enhance the input feature map X using the spectral weight map:

[0129] ;

[0130] in, This is the feature map after spectral enhancement. For enhancement coefficient;

[0131] (8c) Add a channel attention mechanism: Recalibrate the importance of channels using global information to generate channel attention weights. :

[0132] ;

[0133] in, Indicates global average pooling. and All are convolution kernels. It is a bias; It is an activation function;

[0134] (8d) Apply channel attention weights to the spectrally enhanced features:

[0135] ;

[0136] Where Y represents the final enhanced image features.

[0137] Comparative experiments were conducted on the present invention and traditional methods, namely the weighted average method and the wavelet transform-based fusion method, in different scenarios. The results are shown in Table 1 below:

[0138] Table 1

[0139]

[0140] The experimental data show that the present invention outperforms traditional methods in all objective evaluation indicators, indicating that the fused images have higher information richness, structural similarity and image quality, thus verifying the effectiveness and superiority of the present invention.

[0141] In summary, this invention significantly improves information integrity and accuracy: the three spectral characteristics are complementary; ultraviolet spectroscopy can detect ultraviolet radiation, visible light spectroscopy can present rich colors and details, and infrared spectroscopy can sense thermal radiation; through fusion, the characteristics of the target under different spectra can be obtained, solving the problem of one-sided information from a single spectrum. In industrial defect detection, it can accurately locate minute defects that are difficult to see with the naked eye or are blurred by thermal imaging. This invention enhances environmental adaptability and robustness: visible light spectroscopy is limited in imaging under harsh environments, while ultraviolet and infrared spectra can still work in low light, smoke, haze, and other scenarios; after fusion, the system can stably acquire images in complex environments. For example, in the dense smoke of a fire scene, fused images retain the location information of the fire source from infrared thermal imaging and combine it with ultraviolet spectroscopy to identify harmful gases, providing a reliable basis for rescue. This invention expands the application fields and functions: fused images integrate multi-source information, promoting the development of fields such as intelligent monitoring of power systems, intelligent security, environmental monitoring, and biomedicine. In power systems, fused images can comprehensively acquire information on equipment appearance defects, discharge conditions, and abnormal temperatures. In security monitoring, fused images can not only identify faces and objects under visible light, but also detect abnormal heat sources and chemical substances through infrared and ultraviolet features, achieving multi-dimensional early warning.

[0142] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. An adaptive fusion method for ultraviolet-visible-infrared three-spectral images, characterized in that: The method includes the following steps in sequence: (1) Acquire the three-spectral image of the high-voltage equipment as the original image, wherein the three-spectral image includes an ultraviolet image, an infrared image and a visible light image; (2) Preprocess the ultraviolet image, infrared image and visible light image respectively to obtain the preprocessed image; (3) Construct a feature extraction computing network, extract the feature information of the preprocessed image through the feature extraction computing network, and perform image feature calculation to obtain the feature vector of the three-spectral image; (4) Calculate the contribution weight of each image in the fusion result based on the feature vectors of the three-spectral images and the feature information of environmental parameters; (5) Perform multi-scale weighted fusion of the three-spectral images according to the contribution weights to obtain the final fused image; (6) Post-process the final fused image, including contrast enhancement, spectral consistency correction and image sharpening.

2. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 1, characterized in that: Step (1) specifically refers to: using an ultraviolet imager, an infrared thermal imager, and a high-definition visible light camera to acquire images of the high-voltage equipment, acquiring three-spectral images in different scenarios, with an image resolution of 1920×1080 pixels. The different scenarios include indoor, outdoor, nighttime, and daytime. The ultraviolet imager is used to capture the ultraviolet light signal generated by corona discharge on the surface of the equipment. The infrared thermal imager obtains the temperature distribution image of the equipment by detecting the thermal radiation on the surface of the equipment. The high-definition visible light camera is used to record the appearance details of the high-voltage equipment.

3. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 1, characterized in that: In step (2), the preprocessing includes image registration and noise suppression. The image registration refers to geometric registration using an ORB feature point matching method. Matching point pairs are selected by calculating the Hamming distance between feature points, and a homography matrix is ​​constructed to realize the geometric transformation of the image. The noise suppression refers to using median filtering for ultraviolet images, nonlocal mean denoising algorithm for visible light images, and bilateral filtering algorithm for infrared images.

4. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 1, characterized in that: In step (3), the construction of the feature extraction computing network and the extraction of feature information of the preprocessed image through the feature extraction computing network specifically refers to: designing a feature extraction computing network based on the characteristics of the three-spectral images to extract feature information of the visible light image, infrared image, and ultraviolet image respectively; the feature extraction computing network is based on the ResNet network as the backbone, combined with the Canny algorithm and the LBP algorithm, and for the ultraviolet and infrared spectra, a spectral feature enhancement module is added to the input end of the ResNet network; For visible light images, extract semantic features, edge and texture detail features; Calculate the mean characteristics of the RGB color space; For ultraviolet images, morphological operations are used to enhance the contour of the ultraviolet radiation region and extract geometric features such as contour perimeter, area, and eccentricity. Calculate the distribution histogram of ultraviolet radiation intensity to obtain the statistical characteristics of ultraviolet radiation intensity; For infrared images, global statistical features such as temperature mean, variance, and extreme values ​​are extracted based on the temperature data of the images. The difference of Gaussian operator is used to detect hot edges and obtain their location and intensity information; a threshold segmentation algorithm is used to extract high-temperature regions and analyze their area and shape features. To address the unique characteristics of ultraviolet and infrared spectra, a spectral enhancement module was designed to enhance the feature representation capabilities of specific spectral regions through learnable spectral response adjustments.

5. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 1, characterized in that: In step (3), the image feature calculation specifically includes the following sequential steps: (3a) Entropy calculation: ； Where H represents the entropy value, Let i be the probability density of pixel value i within the local window. Grayscale =256, for each pixel in each spectral image, calculate the neighborhood entropy value of each pixel to generate an entropy map. , It is a spectral image; (3b) Energy calculation: ； Where E represents energy, N is the number of pixels in the window, Ω is the local window, and f b (x', y') is the spectral image The feature value at pixel coordinates (x', y'); generating an energy map. This characterizes the local energy distribution of each spectral image; (3c) Feature vector calculation: For each pixel coordinate The local feature vector is defined as follows: ； in, In the spectral image In pixel coordinates The feature vector at that location, In spectral images In pixel coordinates Entropy diagram at the location, In spectral images In pixel coordinates The energy map at the location, where T is the ambient temperature, U is the ultraviolet radiation intensity, and L is the light intensity, realizes the fusion of environmental parameters and image features.

6. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 1, characterized in that: Step (4) specifically includes the following steps in sequence: (4a) Calculate the characteristic Euclidean distance: ； In the formula, Representing spectral images With spectral images In pixel coordinates The characteristic Euclidean distance at the location. Representing spectral images In pixel coordinates The feature vector at that location, Representing spectral images In pixel coordinates The eigenvector at that location; Calculate the characteristic cosine distance: ； in, Representing spectral images With spectral images In pixel coordinates The characteristic cosine distance at point [-1, 1] takes values ​​in the range [-1, 1]. (4b) Perform normalization: normalize the coordinates of each pixel. Calculate the pairwise similarity between all spectral images: ； in, Representing spectral images With spectral images In pixels The result after normalizing the characteristic cosine distance at the location; It is the maximum value among all possible Euclidean distances within the entire image region; (4c) The formula for calculating the normalized weights based on similarity is as follows: ； in, Representing spectral images In pixel coordinates Position-adaptive weights B represents the spectral image that is not currently being weighted; B is the total number of spectral images. (4d) When When 'b' is set to VIS, IR, or UV, the resulting image (visible light, infrared, and ultraviolet) in pixel coordinates before correction can be obtained. Contribution weight at each location , , Introducing environmental parameters as weight correction terms, the contribution weight of each spectral image at each pixel coordinate is calculated after correction: ； ； ； in, , , These represent the corrected visible light, infrared, and ultraviolet images in pixel coordinates, respectively. The contribution weight at each location; L is the light intensity, T is the ambient temperature, and U is the ultraviolet radiation intensity; , , The adjustment coefficients are optimized using a genetic algorithm. , , These represent the average, maximum, and minimum values ​​of light intensity, respectively. , , These represent the average, maximum, and minimum ambient temperatures, respectively. , , These represent the average, maximum, and minimum values ​​of ultraviolet radiation intensity, respectively.

7. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 1, characterized in that: Step (5) specifically includes the following steps in sequence: (5a) Non-subsampled contour wave transform (NSCT) is used to decompose the preprocessed trispectral image into a low-frequency sub-band and multiple high-frequency sub-bands. The low-frequency sub-band contains the main structure and background information of the image, while the high-frequency sub-band contains the edge and texture information of the image. (5b) Determine the fusion strategy for low-frequency subbands based on contribution weights: In nighttime or low-light environments, the low-frequency subbands of infrared images are used as the main component, combined with the weighted information of the low-frequency subbands of visible and ultraviolet images. In daylight conditions, the low-frequency sub-band of the visible light image is the primary focus, while effective information from the low-frequency sub-bands of the infrared and ultraviolet images is fused together; when an ultraviolet radiation anomaly is detected, the weight ratio of the low-frequency sub-band of the ultraviolet image is increased. (5c) For the high-frequency subband, weighted fusion is performed according to the adaptive weight of each spectral image. At the same time, guided filtering is introduced to optimize the fused high-frequency subband. The high-frequency subband of the visible light image is used as the guided image to enhance the detail clarity and edge sharpness of the fused image. (5d) Using NSCT inverse transform, the fused low-frequency subband and high-frequency subband are reconstructed into the final fused image.

8. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 1, characterized in that: Step (6) specifically includes the following steps in sequence: (6a) Contrast enhancement: The multi-scale MSR algorithm based on Retinex theory is used to enhance the local and global contrast of the preliminary fused image, thereby improving the visual effect and detail recognition of the image; (6b) Spectral consistency correction: The final fused image is converted from the RGB color space to the Lab color space. By adjusting the distribution of the brightness L, green red, blue yellow channels, the spectral consistency of the image is corrected to reduce the color shift problem generated during the fusion process. Finally, the image is converted back to the RGB color space. (6c) Image sharpening: The image is sharpened using an unsharpened mask algorithm to enhance the image edges and make the fused image clearer.

9. The adaptive fusion method for ultraviolet-visible-infrared three-spectral images according to claim 4, characterized in that: The processing of the spectral feature enhancement module includes the following steps: (8a) Spectral response adjustment: For the input feature map X∈R C×H1×W Perform channel-level adjustments to generate a spectral weighting map. : ； Where σ is the Sigmoid function, W1 and b3 are the learnable convolution kernel and bias, respectively; C represents the number of channels, H1 is the height of the input image, and W is the width of the input image; (8b) Spectral feature enhancement: Enhance the input feature map X using the spectral weight map: ； in, This is the feature map after spectral enhancement. For enhancement coefficient; (8c) Add a channel attention mechanism: Recalibrate the importance of channels using global information to generate channel attention weights. : ； in, Indicates global average pooling. and All are convolution kernels. It is a bias; It is an activation function; (8d) Apply channel attention weights to the spectrally enhanced features: ； Where Y represents the final enhanced image features.

Images