High-altitude cross-waveband image intelligent registration and quality migration enhancement method and system

Abstract

The application provides a high-altitude cross-waveband image intelligent registration and quality migration enhancement method and system, relates to the technical field of image processing, and comprises the following steps: performing atmospheric attenuation compensation and multi-scale feature decomposition on visible light and infrared images, obtaining modal decoupling feature representation through a cross-modal feature alignment network, and calculating a registration transformation matrix to realize image registration; and based on quality perception coding vectors and modal decoupling features, performing adaptive enhancement on the registered infrared image by taking visible light image quality features as a guide. The method realizes accurate registration and quality migration of cross-waveband images, and improves the quality and information quantity of infrared images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for intelligent registration and quality transfer enhancement of high-altitude cross-band images. Background Technology

[0002] High-altitude cross-band image registration and enhancement technology has significant application value in fields such as remote sensing monitoring and disaster assessment. This technology aims to fuse image information from different imaging sensors (such as visible light and infrared sensors) to obtain a more comprehensive and reliable understanding of the scene. Current conventional practices typically follow a series of independent processing steps to accomplish this task.

[0003] Conventional technical workflows typically begin by preprocessing the raw visible and infrared images, such as using histogram equalization or radiometric correction based on simplified atmospheric models to adjust image contrast and brightness. Subsequently, registration methods based on feature points (e.g., SIFT, SURF) or mutual information are used to directly calculate spatial transformation relationships on the two original images or their preprocessed versions to achieve image alignment. After obtaining the registered images, enhancement processing is often performed independently. Common methods include enhancing the infrared image using histogram specification or guided filtering, attempting to bring it closer to the quality of the visible image in terms of visual detail or contrast.

[0004] Under high-altitude imaging conditions, visible light and infrared images not only exhibit significant modal differences (such as differences in texture details and thermal radiation information), but are also jointly affected by attenuation from complex atmospheric environments (such as aerosols and water vapor), with the degree of attenuation closely related to imaging altitude and wavelength. Existing preprocessing methods typically do not adequately consider compensation for these differences in atmospheric transmission attenuation related to altitude and wavelength, resulting in a weak foundation for radiometric consistency between images and directly affecting subsequent registration accuracy. Conventional workflows treat registration and enhancement as two sequential and separate stages. In the registration stage, the structural correlation and semantic consistency of deep features between cross-modal images are not fully considered, relying only on surface grayscale or feature point information, leading to unstable registration results when modal differences are large and noise interference is strong. In the subsequent enhancement stage, there is a lack of systematic analysis and transfer guidance on the rich quality attributes of the source image (such as visible light images) (such as sharpness, contrast, and texture richness), resulting in an enhancement process that is often blind and lacks adaptability, making it difficult to generate a fused result that maintains both infrared modal characteristics and incorporates the high-quality visual attributes of the visible light image. Summary of the Invention

[0005] This invention provides a method and system for intelligent registration and quality transfer enhancement of high-altitude cross-band images, which can solve the problems in the prior art.

[0006] A first aspect of the present invention provides a method for intelligent registration and quality transfer enhancement of high-altitude cross-band images, comprising:

[0007] Atmospheric transmission attenuation compensation related to altitude was performed on the visible light band image and the infrared band image in the high-altitude cross-band image to obtain the compensated visible light image and the compensated infrared image. Multi-scale feature decomposition is performed on the compensated visible light image and the compensated infrared image respectively to obtain a set of visible light multi-scale features and a set of infrared multi-scale features. By constructing a cross-modal feature alignment network based on the mutual information maximization criterion and structure preservation constraint, the visible light multi-scale feature set and the infrared multi-scale feature set are projected onto a mode-independent unified representation space to obtain a modal decoupling feature representation; based on the modal decoupling feature representation, the geometric transformation relationship between the visible light band image and the infrared band image is calculated to obtain the registration transformation matrix; The infrared band image is subjected to spatial geometric transformation according to the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image. Quality attribute analysis is performed on the visible light band image and the registered infrared image to obtain a quality-aware coding vector. Based on the quality-aware coding vector and the modal decoupling feature representation, the registered infrared image is adaptively enhanced by using the target quality features of the visible light band image as a guiding condition, thereby generating a quality transfer enhanced image.

[0008] Altitude-dependent atmospheric transmission attenuation compensation is performed on the visible light and infrared images in the high-altitude cross-band images to obtain compensated visible light and infrared images, respectively: Obtain the atmospheric pressure and atmospheric density values ​​at the current altitude layer, and calculate the atmospheric attenuation coefficient in the visible light band and the atmospheric attenuation coefficient in the infrared band based on the atmospheric pressure and atmospheric density values, respectively. Spatial domain compensation processing is performed on the visible light band image. The original radiation intensity value of each pixel is extracted by traversing each pixel position. The original radiation intensity value is multiplied by the reciprocal of the atmospheric attenuation coefficient of the visible light band. The product result is exponentially compensated according to the atmospheric path length to obtain the compensated radiation intensity value and write it into the corresponding pixel position. After all pixels are processed, the compensated visible light image is obtained. The infrared band image is subjected to frequency domain compensation processing. The infrared band image is converted into a frequency domain representation through frequency domain transformation and decomposed into multiple frequency components. For each frequency component, a frequency-related compensation weight coefficient is calculated based on the atmospheric attenuation coefficient of the infrared band. The frequency component is multiplied by the compensation weight coefficient to obtain the compensated frequency component. All compensated frequency components are recombined into a compensated frequency domain representation and inversely transformed to the spatial domain to obtain the compensated infrared image.

[0009] Multi-scale feature decomposition is performed on the compensated visible light image and the compensated infrared image respectively to obtain the visible light multi-scale feature set and the infrared multi-scale feature set, including: The compensated visible light image is subjected to Laplacian pyramid decomposition to construct a multi-layer Gaussian filter downsampling sequence to obtain a low-frequency approximation layer; Differential operations are performed on low-frequency approximation layers at adjacent scales to obtain multiple detail layers with passbands. Direction-selective filtering is performed on each detail layer to separate multiple directional components. Local phase information and amplitude response information of each directional component are extracted. The directional components, local phase information and amplitude response information of all detail layers are organized into the visible light multi-scale feature set. Wavelet packet decomposition is performed on the compensated infrared image to obtain a multi-layer wavelet coefficient tree structure; Energy statistics are performed on the wavelet coefficients of each node in the wavelet coefficient tree structure to obtain the sub-band energy distribution characteristics. Correlation analysis is performed on the wavelet coefficients of adjacent scales to obtain cross-scale dependence characteristics. The wavelet coefficients, sub-band energy distribution characteristics and cross-scale dependence characteristics of all decomposed levels are organized into the infrared multi-scale feature set.

[0010] By constructing a cross-modal feature alignment network based on the mutual information maximization criterion and structure preservation constraints, the visible light multi-scale feature set and the infrared multi-scale feature set are projected onto a mode-independent unified representation space, resulting in a mode-decoupled feature representation including: In the cross-modal feature alignment network, the visible light feature coding branch maps the visible light multi-scale feature set to a visible light intermediate representation through multi-layer nonlinear transformation, and the infrared feature coding branch maps the infrared multi-scale feature set to an infrared intermediate representation through multi-layer nonlinear transformation; The mutual information quantization value is calculated based on the joint distribution entropy and edge distribution entropy between the visible light intermediate representation and the infrared intermediate representation, and the maximization of the mutual information quantization value is set as the mutual information optimization objective. Extract the spatial neighborhood relationships in the visible light multi-scale feature set and the infrared multi-scale feature set, calculate the degree of preservation of the spatial neighborhood relationships between the visible light intermediate representation and the infrared intermediate representation after projection, and set maximizing the degree of preservation of the spatial neighborhood relationships as the structure preservation optimization objective; The mutual information optimization objective is combined with the structure preservation optimization objective to obtain a joint optimization objective. By iteratively optimizing the joint optimization objective, the transformation parameters of the visible light feature coding branch and the infrared feature coding branch are updated, and the visible light multi-scale feature set and the infrared multi-scale feature set are projected and transformed respectively to obtain visible light projection features and infrared projection features located in a unified representation space. The visible light projection features and the infrared projection features are organized into the modal decoupled feature representation.

[0011] The geometric transformation relationship between the visible light band image and the infrared band image is calculated to obtain the registration transformation matrix, which includes: The visible light projection features are subjected to corner detection based on response intensity to obtain a set of visible light anchor points, and the infrared projection features are subjected to feature point detection based on scale space extrema to obtain a set of infrared anchor points. For each visible light anchor point in the visible light anchor point set, local neighborhood window features are extracted from the visible light projection features and a multi-directional gradient statistical histogram is constructed as a visible light anchor point descriptor. For each infrared anchor point in the infrared anchor point set, local scale normalization features are extracted from the infrared projection features and a direction encoding vector is constructed as an infrared anchor point descriptor. Calculate the feature distance between the visible light anchor point descriptor and the infrared anchor point descriptor, and select the infrared anchor point with the smallest feature distance for each visible light anchor point to form an initial set of matching point pairs; Perform consistency verification on the initial set of matching point pairs, calculate the offset vector distribution characteristics of the spatial location offset vector, and retain the set of reliable matching point pairs that conform to the offset vector distribution characteristics; Based on the spatial coordinates of the set of reliable matching points, a linear constraint relationship is constructed. Based on the linear constraint relationship, the rotation parameters, translation parameters, and scaling parameters are solved and organized into the registration transformation matrix.

[0012] The infrared band image is subjected to a spatial geometric transformation based on the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image. Quality attribute analysis is then performed on the visible light band image and the registered infrared image to obtain a quality-aware coding vector, including: The rotation, translation, and scaling parameters in the registration transformation matrix are decomposed into an affine transformation operation sequence. The pixel coordinates of the infrared band image are mapped point by point according to the affine transformation operation sequence. Bilinear interpolation is performed on the non-integer coordinate positions generated after mapping to calculate the pixel intensity value, thereby generating a registered infrared image that is spatially aligned with the visible light band image. The visible light band image is decomposed into a set of visible light frequency domain components at multiple scales. Local change energy features and edge sharpness indices are extracted from the visible light frequency domain component set as visible light quality metrics. The registered infrared image is decomposed into a set of infrared frequency domain components at multiple scales. Contrast dynamic range features and texture richness indices are extracted from the infrared frequency domain component set as infrared quality metrics. The visible light quality metric and the infrared quality metric are concatenated according to their spatial correspondence to obtain a joint quality metric vector. The joint quality metric vector is then subjected to dimensionality reduction encoding to obtain the quality-aware encoding vector.

[0013] Based on the quality-aware coding vector and the modality decoupled feature representation, the registered infrared image is adaptively enhanced by using the target quality features of the visible light band image as a guiding condition, generating a quality transfer enhanced image, including: The visible light quality metric and the infrared quality metric are parsed from the quality-aware encoding vector and the difference quantification index between them is calculated. A quality difference conversion rule is constructed based on the difference quantification index. The local change energy features and edge sharpness index of the visible light band image are converted into quality transfer target parameters through the quality difference conversion rule. The infrared projection features are extracted from the modal decoupling feature representation. The spatial weight distribution of the infrared projection features is dynamically adjusted according to the quality transfer target parameters through a spatial attention mechanism. The channel response intensity of the infrared projection features is dynamically adjusted according to the quality transfer target parameters through a channel attention mechanism to obtain the quality-guided enhancement features. Perform an inverse projection transformation on the quality-guided enhancement features to obtain an enhanced intermediate representation, and perform an upsampling operation on the enhanced intermediate representation to restore the spatial resolution to the registered infrared image to obtain an enhanced feature map; The registered infrared image and the enhanced feature map are fused pixel by pixel, and the quality improvement component in the enhanced feature map is superimposed onto the corresponding pixel position of the registered infrared image through a residual connection method to generate the quality transfer enhanced image.

[0014] A second aspect of the present invention provides an intelligent registration and quality transfer enhancement system for high-altitude cross-band images, comprising: The compensation unit is used to perform altitude-related atmospheric transmission attenuation compensation on the visible light band image and the infrared band image in the high-altitude cross-band image, respectively, to obtain the compensated visible light image and the compensated infrared image. The decomposition unit is used to perform multi-scale feature decomposition on the compensated visible light image and the compensated infrared image respectively to obtain a set of visible light multi-scale features and a set of infrared multi-scale features. Alignment unit is used to project the visible light multi-scale feature set and the infrared multi-scale feature set onto a mode-independent unified representation space by constructing a cross-modal feature alignment network based on mutual information maximization criteria and structure preservation constraints, thereby obtaining a mode-decoupled feature representation; The registration unit is used to calculate the geometric transformation relationship between the visible light band image and the infrared band image based on the modal decoupling feature representation, and obtain the registration transformation matrix. The analysis unit is used to perform spatial geometric transformation on the infrared band image according to the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image, and to perform quality attribute analysis on the visible light band image and the registered infrared image to obtain a quality-aware coding vector. An enhancement unit is used to adaptively enhance the registered infrared image based on the quality-aware coding vector and the modal decoupling feature representation, by using the target quality features of the visible light band image as a guiding condition, to generate a quality transfer enhanced image.

[0015] A third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0016] Fourth aspect of the present invention, A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0017] The beneficial effects of this application are as follows: This method effectively solves a key challenge in high-altitude cross-band image registration and fusion. By performing altitude-dependent atmospheric transmission attenuation compensation on visible and infrared images, it significantly reduces the non-uniform degradation caused by atmospheric scattering and absorption on signals of different bands, providing basic image data with higher radiometric consistency for subsequent processing and improving the reliability of image information. Multi-scale feature decomposition of the compensated image simultaneously captures structural, textural, and semantic information at multiple scales, constructing a more comprehensive and robust feature representation and laying a refined data foundation for cross-modal alignment.

[0018] The core innovation of this method is the construction of a cross-modal feature alignment network based on mutual information maximization and structure preservation constraints. This network projects features from different bands onto a modally independent unified representation space, achieving modal decoupling of features. The mutual information maximization criterion drives the network to learn the essential information shared between bands, while the structure preservation constraint ensures that the important structural features unique to each band are not destroyed during the decoupling process. The resulting modal decoupling feature representation maximizes the removal of sensor modal differences while preserving the geometric structural information used for registration, making subsequent calculations of geometric transformation relationships based on these features more accurate and stable, and resulting in a higher precision registration transformation matrix.

[0019] Spatial correction of infrared images using a high-precision registration transformation matrix achieves pixel-level alignment with visible light images, resolving geometric misalignment issues caused by differences in imaging viewpoint and platform attitude. Based on this, quality attribute analysis quantifies the quality differences among images in terms of sharpness, contrast, and information entropy. The generated quality-aware coding vector provides clear guidance for subsequent quality transfer. Adaptive enhancement of the registered infrared image is performed using the target quality features of the visible light image as a condition. This process is not a simple information superposition, but rather intelligently transfers high-quality attributes of the visible light image (such as high texture detail and high edge sharpness) to the infrared image based on the quality coding vector and modal decoupling features. This generates an enhanced image that contains both infrared thermal radiation information and the visual quality of the visible light image, significantly improving the overall perceptual quality and information utilization efficiency of the fusion result. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the intelligent registration and quality transfer enhancement method for high-altitude cross-band images according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the cross-modal feature alignment network process. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0023] Figure 1 This is a flowchart illustrating the intelligent registration and quality transfer enhancement method for high-altitude cross-band images according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: Atmospheric transmission attenuation compensation related to altitude was performed on the visible light band image and the infrared band image in the high-altitude cross-band image to obtain the compensated visible light image and the compensated infrared image. Multi-scale feature decomposition is performed on the compensated visible light image and the compensated infrared image respectively to obtain a set of visible light multi-scale features and a set of infrared multi-scale features. By constructing a cross-modal feature alignment network based on the mutual information maximization criterion and structure preservation constraint, the visible light multi-scale feature set and the infrared multi-scale feature set are projected onto a mode-independent unified representation space to obtain a mode-decoupled feature representation. Based on the modal decoupling feature representation, the geometric transformation relationship between the visible light band image and the infrared band image is calculated to obtain the registration transformation matrix; The infrared band image is subjected to spatial geometric transformation according to the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image. Quality attribute analysis is performed on the visible light band image and the registered infrared image to obtain a quality-aware coding vector. Based on the quality-aware coding vector and the modal decoupling feature representation, the registered infrared image is adaptively enhanced by using the target quality features of the visible light band image as a guiding condition, thereby generating a quality transfer enhanced image.

[0024] In one optional implementation, atmospheric transmission attenuation compensation related to altitude is applied to the visible light band image and the infrared band image in the high-altitude cross-band image, respectively, to obtain compensated visible light images and compensated infrared images, including: Obtain the atmospheric pressure and atmospheric density values ​​at the current altitude layer, and calculate the atmospheric attenuation coefficient in the visible light band and the atmospheric attenuation coefficient in the infrared band based on the atmospheric pressure and atmospheric density values, respectively. Spatial domain compensation processing is performed on the visible light band image. The original radiation intensity value of each pixel is extracted by traversing each pixel position. The original radiation intensity value is multiplied by the reciprocal of the atmospheric attenuation coefficient of the visible light band. The product result is exponentially compensated according to the atmospheric path length to obtain the compensated radiation intensity value and write it into the corresponding pixel position. After all pixels are processed, the compensated visible light image is obtained. The infrared band image is subjected to frequency domain compensation processing. The infrared band image is converted into a frequency domain representation through frequency domain transformation and decomposed into multiple frequency components. For each frequency component, a frequency-related compensation weight coefficient is calculated based on the atmospheric attenuation coefficient of the infrared band. The frequency component is multiplied by the compensation weight coefficient to obtain the compensated frequency component. All compensated frequency components are recombined into a compensated frequency domain representation and inversely transformed to the spatial domain to obtain the compensated infrared image.

[0025] The flight platform acquires real-time altitude data from altitude sensors and queries the corresponding atmospheric pressure (P) and density (ρ) values ​​for that altitude level using a standard atmospheric model. For the visible light band, based on Rayleigh scattering theory, the atmospheric attenuation coefficient β for the visible light band is... vis The calculation considers the combined effects of aerosol scattering and molecular absorption, and its value is proportional to the negative fourth power of the wavelength. An atmospheric density correction factor ρ / ρ0 is also introduced, where ρ0 is the standard atmospheric density at sea level. For the infrared band, the atmospheric attenuation coefficient β... ir The calculation needs to take into account the influence of the water vapor absorption band and the carbon dioxide absorption band. The average attenuation coefficient of the band is calculated by using a simplified form of the MODTRAN atmospheric radiation transfer model and combining it with the current atmospheric pressure value.

[0026] A pixel-by-pixel traversal strategy is used to perform spatial domain compensation processing on visible light band images. For a pixel with coordinates (x, y) in the image, its original radiant intensity value I is read. origin (x, y) represents the radiant energy reaching the sensor after atmospheric attenuation. The core of the compensation process lies in the reverse recovery of atmospheric transmission loss. The calculation process is as follows: the original radiant intensity value is multiplied by the reciprocal of the atmospheric attenuation coefficient in the visible light band, 1 / β. vis Multiplying yields a preliminary compensation result; exponential compensation correction is then performed based on the atmospheric path length L, which is calculated using the difference between the current flight altitude and the target ground object's altitude. The exponential compensation factor is exp(β). vis • L) form, compensated radiation intensity value I comp (x, y) is written back to the corresponding pixel position. After this process is performed on all pixels, a complete compensated visible light image is formed, and its grayscale dynamic range is effectively restored.

[0027] Frequency domain compensation processing is performed on the infrared band image. A Fast Fourier Transform (FFT) is used to transform the infrared band image from the spatial domain to the frequency domain, obtaining a complex spectral distribution. Frequency domain decomposition employs an eight-directional frequency decomposition strategy, dividing the spectrum into three levels: low-frequency, mid-frequency, and high-frequency components. For each frequency component, a frequency-related compensation weight coefficient is calculated based on the infrared band atmospheric attenuation coefficient. The low-frequency component corresponds to the large-scale structural information of the image and is less affected by atmospheric attenuation; therefore, its compensation weight coefficient is set to 1 + 0.2β. ir The mid-frequency components correspond to edge and texture information, and are moderately affected by attenuation; the compensation weighting coefficient is set to 1 + 0.5β. ir The high-frequency components correspond to detailed information and are most significantly affected by attenuation; the compensation weighting coefficient is set to 1 + 0.8β. ir Each frequency component is multiplied by its corresponding compensation weight coefficient using complex multiplication to obtain the compensated frequency components. All compensated frequency components are then recombined according to their original frequency domain structure to form a complete compensated frequency domain representation. This representation is then converted back to the spatial domain using an inverse Fourier transform to generate the compensated infrared image. Frequency domain compensation processing can perform differentiated enhancement for different frequency components, effectively restoring high-frequency detail information in infrared images and avoiding the noise amplification problem introduced by spatial domain compensation.

[0028] In one optional implementation, multi-scale feature decomposition is performed on the compensated visible light image and the compensated infrared image respectively to obtain a visible light multi-scale feature set and an infrared multi-scale feature set, including: The compensated visible light image is subjected to Laplacian pyramid decomposition to construct a multi-layer Gaussian filter downsampling sequence to obtain a low-frequency approximation layer; Differential operations are performed on low-frequency approximation layers at adjacent scales to obtain multiple detail layers with passbands. Direction-selective filtering is performed on each detail layer to separate multiple directional components. Local phase information and amplitude response information of each directional component are extracted. The directional components, local phase information and amplitude response information of all detail layers are organized into the visible light multi-scale feature set. Wavelet packet decomposition is performed on the compensated infrared image to obtain a multi-layer wavelet coefficient tree structure; Energy statistics are performed on the wavelet coefficients of each node in the wavelet coefficient tree structure to obtain the sub-band energy distribution characteristics. Correlation analysis is performed on the wavelet coefficients of adjacent scales to obtain cross-scale dependence characteristics. The wavelet coefficients, sub-band energy distribution characteristics and cross-scale dependence characteristics of all decomposed levels are organized into the infrared multi-scale feature set.

[0029] A five-layer Laplacian pyramid structure was constructed for the compensated visible light image. In the first layer, a 5×5 Gaussian kernel was used to perform convolution filtering on the compensated visible light image, with a standard deviation set to 1.2. After filtering, the image was downsampled every other pixel in the row and column directions to obtain a size reduced to the original. Figure 2 The first low-frequency approximation layer is one-tenth of the total. The Gaussian filtering and downsampling operations are repeated on the first low-frequency approximation layer to generate the second through fifth low-frequency approximation layers, with the resolution decreasing sequentially for each layer. The k-th low-frequency approximation layer is upsampled to the (k-1)-th layer size using bilinear interpolation, and then pixel-by-pixel differencing is performed with the (k-1)-th low-frequency approximation layer to obtain the k-th detail layer with a passband. A total of four detail layers are obtained.

[0030] A Gabor filter bank with four directions is applied to each detail layer, with direction angles set to 0 degrees, 45 degrees, 90 degrees, and 135 degrees, a frequency parameter of 0.15, and a bandwidth parameter of 1.5, thereby separating four directional components: horizontal, diagonal, vertical, and anti-diagonal. A Hilbert transform is applied to each directional component to extract local phase information, and the phase value range is normalized to the interval between negative π and positive π. Amplitude response information is obtained by calculating the gradient magnitude of the directional components, and the magnitude is compressed to the range of 0 to 1 using a nonlinear mapping function. The 16 directional components of the four detail layers, the corresponding phase information matrices, and the amplitude response matrices are concatenated in hierarchical order to form a visible light multi-scale feature set containing 48 feature channels.

[0031] The compensated infrared image is subjected to a four-level db4 wavelet packet decomposition. The first level of decomposition produces four nodes: low-frequency approximation coefficients, horizontal detail coefficients, vertical detail coefficients, and diagonal detail coefficients. Further decomposition of these four nodes produces 16 child nodes in the second level, 64 child nodes in the third level, and 256 child nodes in the fourth level, forming a complete wavelet coefficient tree structure.

[0032] The sum of squares of the wavelet coefficient matrix of each node in the tree structure is calculated and divided by the total number of coefficients to obtain the normalized energy value of that subband. The energy values ​​of all nodes within the same level are statistically analyzed to form the subband energy distribution vector for that level. Four energy distribution vectors are formed across the four levels as subband energy distribution features. The Pearson correlation coefficient is calculated between the wavelet coefficient of a node in the k-th level and the wavelet coefficient of its parent node at the corresponding position in the (k-1)-th level. Positions with an absolute correlation coefficient greater than 0.6 are marked as strong dependency regions. The proportion of strong dependency regions for each node is statistically analyzed, and a cross-scale dependency feature matrix is ​​constructed, with the matrix dimension being the number of nodes multiplied by the dependency strength. The wavelet coefficients of the four levels (340 nodes in total), the four subband energy distribution vectors, and the cross-scale dependency feature matrix are organized into an infrared multi-scale feature set. This set contains the frequency characteristics, energy distribution patterns, and inter-scale correlation information in the wavelet domain.

[0033] In one optional implementation, by constructing a cross-modal feature alignment network based on the mutual information maximization criterion and structure preservation constraints, the visible light multi-scale feature set and the infrared multi-scale feature set are projected onto a modally independent unified representation space to obtain modally decoupled feature representations, including: In the cross-modal feature alignment network, the visible light feature coding branch maps the visible light multi-scale feature set to a visible light intermediate representation through multi-layer nonlinear transformation, and the infrared feature coding branch maps the infrared multi-scale feature set to an infrared intermediate representation through multi-layer nonlinear transformation; The mutual information quantization value is calculated based on the joint distribution entropy and edge distribution entropy between the visible light intermediate representation and the infrared intermediate representation, and the maximization of the mutual information quantization value is set as the mutual information optimization objective. Extract the spatial neighborhood relationships in the visible light multi-scale feature set and the infrared multi-scale feature set, calculate the degree of preservation of the spatial neighborhood relationships between the visible light intermediate representation and the infrared intermediate representation after projection, and set maximizing the degree of preservation of the spatial neighborhood relationships as the structure preservation optimization objective; The mutual information optimization objective is combined with the structure preservation optimization objective to obtain a joint optimization objective. By iteratively optimizing the joint optimization objective, the transformation parameters of the visible light feature coding branch and the infrared feature coding branch are updated, and the visible light multi-scale feature set and the infrared multi-scale feature set are projected and transformed respectively to obtain visible light projection features and infrared projection features located in a unified representation space. The visible light projection features and the infrared projection features are organized into the modal decoupled feature representation.

[0034] like Figure 2 As shown, the method includes: In the cross-modal feature alignment network, the visible light feature encoding branch adopts a three-layer convolutional neural network structure. Each layer has a 3×3 kernel size and 64, 128, and 256 channels respectively, with ReLU as the activation function. The visible light multi-scale feature set is input into this encoding branch. After layer-by-layer convolution and non-linear activation, the feature map is compressed into a 512-dimensional visible light intermediate representation vector through global average pooling in the last layer. The infrared feature encoding branch uses the same network architecture to encode the infrared multi-scale feature set, also outputting a 512-dimensional infrared intermediate representation vector.

[0035] Based on the two obtained intermediate representation vectors, the mutual information quantization value is calculated. Specifically, a joint probability distribution P(x_v, x_i) and marginal probability distributions P(x_v) and P(x_i) are constructed, where x_v represents the visible light intermediate representation and x_i represents the infrared intermediate representation. The probability distributions are estimated using a kernel density estimation method with a Gaussian kernel function and a bandwidth parameter set to 0.5. The mutual information quantization value is calculated by the difference between the joint entropy and the marginal entropy. Maximizing this quantization value is set as the objective function for mutual information optimization, with a weighting coefficient of 1.0.

[0036] When extracting spatial neighborhood relationships, for each feature point in the visible light multi-scale feature set, an 8-neighborhood connectivity graph is constructed, and the Euclidean distance matrix between the feature point and its neighboring feature points is calculated. The same operation is performed on the infrared multi-scale feature set to obtain the neighborhood relationship matrix for the infrared features. After projecting the visible light intermediate representation and the infrared intermediate representation, their respective neighborhood distance relationships are recalculated. By comparing the Frobenius norm difference between the distance relationship matrices before and after projection, the degree of preservation of spatial neighborhood relationships is quantified. The closer this preservation value is to 1, the more complete the structure preservation. Maximizing this value is used as the objective function for structure preservation optimization, with a weighting coefficient set to 0.8.

[0037] The joint optimization objective function consists of a weighted sum of the mutual information optimization objective and the structure-preserving optimization objective. The Adam optimizer is used to iteratively update the network parameters, with an initial learning rate of 0.001, which decays to 0.9 times the original value every 50 iterations. During training, 32 pairs of feature samples are randomly selected as batch inputs from the visible light multi-scale feature set and the infrared multi-scale feature set in each iteration. The gradient of the loss function with respect to the convolutional weights and bias parameters of each layer is calculated using the backpropagation algorithm, updating the transformation parameters of the visible light feature encoding branch and the infrared feature encoding branch. After 500 iterations, the network converges to a stable state.

[0038] The optimized encoding branches perform forward projection transformations on the complete visible light multi-scale feature sets and infrared multi-scale feature sets respectively, outputting visible light projection feature matrices and infrared projection feature matrices, both with dimensions H×W×256, where H is the height dimension and W is the width dimension of the projection feature matrix. In the unified representation space, these two projection feature matrices have the same feature distribution characteristics and semantic expressive power, no longer affected by the differences in the original modes. The visible light projection features and infrared projection features are concatenated along the channel dimension to form an H×W×512 mode decoupled feature representation. This representation simultaneously preserves the complementary information of the two modes and eliminates the representational differences between modes, providing a reliable feature foundation for the accurate calculation of the subsequent registration transformation matrix.

[0039] In one optional implementation, calculating the geometric transformation relationship between the visible light band image and the infrared band image to obtain the registration transformation matrix includes: The visible light projection features are subjected to corner detection based on response intensity to obtain a set of visible light anchor points, and the infrared projection features are subjected to feature point detection based on scale space extrema to obtain a set of infrared anchor points. For each visible light anchor point in the visible light anchor point set, local neighborhood window features are extracted from the visible light projection features and a multi-directional gradient statistical histogram is constructed as a visible light anchor point descriptor. For each infrared anchor point in the infrared anchor point set, local scale normalization features are extracted from the infrared projection features and a direction encoding vector is constructed as an infrared anchor point descriptor. Calculate the feature distance between the visible light anchor point descriptor and the infrared anchor point descriptor, and select the infrared anchor point with the smallest feature distance for each visible light anchor point to form an initial set of matching point pairs; Perform consistency verification on the initial set of matching point pairs, calculate the offset vector distribution characteristics of the spatial location offset vector, and retain the set of reliable matching point pairs that conform to the offset vector distribution characteristics; Based on the spatial coordinates of the set of reliable matching points, a linear constraint relationship is constructed. Based on the linear constraint relationship, the rotation parameters, translation parameters, and scaling parameters are solved and organized into the registration transformation matrix.

[0040] After obtaining the modal decoupling feature representation, a precise spatial correspondence is established between visible light projection features and infrared projection features. For visible light projection features, the Harris corner detection algorithm is used to extract corner locations with significant response intensity. Specifically, the local gray-level change matrix of each pixel location is calculated, and the corner response function value is determined by calculating the eigenvalues ​​of this matrix. Locations with response function values ​​exceeding an adaptive threshold and being local maxima are selected as visible light anchor points, forming a set of visible light anchor points. For infrared projection features, the SIFT feature point detection method is used to construct a scale-space pyramid. Extreme points of the difference-of-Gaussian image are found at different scale levels as infrared anchor points, forming a set of infrared anchor points.

[0041] To establish matching relationships between anchor points, discriminative descriptors need to be constructed. For each anchor point in the visible light anchor point set, a 16×16 pixel local neighborhood window is extracted centered on that anchor point. The gradient magnitude and gradient direction of each pixel within the window are calculated. The 360-degree directional range is divided into 8 directional intervals, and the local window is further divided into 4×4 sub-regions. The cumulative gradient magnitude values ​​of the 8 directions are counted within each sub-region, forming a 128-dimensional multi-directional gradient statistical histogram as the visible light anchor point descriptor. For each anchor point in the infrared anchor point set, the sampling radius is determined based on the anchor point's characteristic scale. Local neighborhood features at the corresponding scale are extracted, and the principal direction is calculated for rotational normalization. Local feature response values ​​are sampled in the normalized coordinate system and Gaussian weighted to construct a 64-dimensional direction encoding vector as the infrared anchor point descriptor.

[0042] The Euclidean distance between the visible light anchor descriptor and the infrared anchor descriptor is calculated as the feature distance metric. For each anchor in the visible light anchor set, the feature distance between it and all infrared anchors is calculated by traversing the infrared anchor set. The infrared anchor with the smallest feature distance is selected as the matching candidate point. At the same time, the ratio of the smallest distance to the second smallest distance is required to be less than 0.7 to ensure the uniqueness of the match. This forms the initial set of matching point pairs.

[0043] Consistency verification is performed on the initial set of matched point pairs to eliminate erroneous pairs. The spatial offset vector of each matched point pair is calculated, representing the positional difference between the visible light anchor point and the corresponding infrared anchor point. The direction and amplitude distribution of all offset vectors are statistically analyzed, and the distribution characteristics are fitted using kernel density estimation to determine the dominant direction and dominant amplitude range. Matched point pairs with a direction deviation of less than 15 degrees and an amplitude deviation of less than 20% of the average amplitude are retained to form a reliable set of matched point pairs.

[0044] Based on the spatial coordinates (x_v, y_v) and (x_i, y_i) of each pair of points in the reliable matching point pair set, a system of linear constraint equations for affine transformation is established. For each matching point pair, two constraint equations are provided for the rotation angle θ, translation vector (t_x, t_y), and scaling factor s. The overdetermined system of equations is solved using the least squares method to obtain the optimal transformation parameters. The solved rotation parameters θ, translation parameters (t_x, t_y), and scaling parameters (s_x, s_y) are organized into a 3×3 homogeneous transformation matrix, which is the registration transformation matrix.

[0045] In one optional implementation, the infrared band image is subjected to a spatial geometric transformation based on the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image. Quality attribute analysis is then performed on the visible light band image and the registered infrared image to obtain a quality-aware coding vector, including: The rotation, translation, and scaling parameters in the registration transformation matrix are decomposed into an affine transformation operation sequence. The pixel coordinates of the infrared band image are mapped point by point according to the affine transformation operation sequence. Bilinear interpolation is performed on the non-integer coordinate positions generated after mapping to calculate the pixel intensity value, thereby generating a registered infrared image that is spatially aligned with the visible light band image. The visible light band image is decomposed into a set of visible light frequency domain components by multi-scale decomposition. Local variation energy features and edge sharpness indicators are extracted from the set of visible light frequency domain components as visible light quality metrics. The registered infrared image is decomposed into a multi-scale set of infrared frequency domain components, and contrast dynamic range features and texture richness index are extracted from the infrared frequency domain component set as infrared quality metrics. The visible light quality metric and the infrared quality metric are concatenated according to their spatial correspondence to obtain a joint quality metric vector. The joint quality metric vector is then subjected to dimensionality reduction encoding to obtain the quality-aware encoding vector.

[0046] After completing cross-modal feature alignment and obtaining the registration transformation matrix, precise spatial geometric transformation operations need to be performed on the infrared band image. The registration transformation matrix is ​​decomposed into rotation parameters θ, translation parameters (t_x, t_y), and scaling parameters (s_x, s_y), constructing an affine transformation operation sequence. For each pixel coordinate (x, y) in the infrared band image, point-by-point mapping is performed according to the operation sequence: first, a scaling transformation is applied to obtain (s_x · x, s_y · y), then a rotation transformation is applied, and finally, the translation vector is superimposed. Since the target coordinates generated during the mapping process are often non-integer values, for example, coordinates mapped to (125.37, 89.62), bilinear interpolation is used to calculate the pixel intensity value at that location. Specifically, the pixel values ​​at four integer coordinate positions (125, 89), (126, 89), (125, 90), and (126, 90) surrounding the mapped coordinates are taken, and a weighted sum is calculated based on the distance weights between the target position and the four adjacent points to obtain the interpolated pixel intensity value. After transforming all pixels of the infrared band image, a registered infrared image that is precisely aligned in spatial position with the visible light band image is generated.

[0047] To achieve guided enhancement of quality perception, quality attribute analysis needs to be performed on the two registered images. A three-layer Laplacian pyramid decomposition is performed on the visible light image to obtain three frequency domain components: high-frequency, mid-frequency, and low-frequency, forming a visible light frequency domain component set. Local variation energy features are extracted from the high-frequency component, calculated by summing the squares of pixel gradient magnitudes within a sliding window. Edge sharpness indicators are extracted from the mid-frequency component, quantifying edge sharpness by calculating the standard deviation of the gradient direction. The two types of feature values ​​are normalized to the 0-1 range and averaged to obtain the visible light quality metric. The same three-layer pyramid decomposition is performed on the registered infrared image to obtain the infrared frequency domain component set. Contrast dynamic range features are extracted from the low-frequency component, calculating the difference between the 95th and 5th quantiles of pixel intensity values. Texture richness indicators are extracted from the mid-frequency component, characterizing texture complexity by calculating the entropy of the gray-level co-occurrence matrix. The two types of feature values ​​are normalized and weighted averaged to obtain the infrared quality metric.

[0048] Visible light quality metrics and infrared quality metrics are concatenated according to the spatial correspondence of image patches. The image is divided into an 8×8 grid, with each grid patch corresponding to a pair of quality metrics, forming a 128-dimensional joint quality metric vector. To reduce the computational complexity of subsequent processing, principal component analysis is used to perform dimensionality reduction encoding on the joint quality metric vector, retaining the top 32 principal components with a cumulative contribution rate of 95%, resulting in a quality-aware encoded vector. This vector compactly represents the quality distribution differences between the visible light and infrared images in different spatial regions, providing accurate guiding information for subsequent quality transfer enhancement.

[0049] In one optional implementation, based on the quality-aware coding vector and the modality decoupled feature representation, adaptive enhancement of the registered infrared image is performed by using the target quality features of the visible light band image as a guiding condition to generate a quality transfer enhanced image, including: The visible light quality metric and the infrared quality metric are parsed from the quality-aware encoding vector and the difference quantification index between them is calculated. A quality difference conversion rule is constructed based on the difference quantification index. The local change energy features and edge sharpness index of the visible light band image are converted into quality transfer target parameters through the quality difference conversion rule. The infrared projection features are extracted from the modal decoupling feature representation. The spatial weight distribution of the infrared projection features is dynamically adjusted according to the quality transfer target parameters through a spatial attention mechanism. The channel response intensity of the infrared projection features is dynamically adjusted according to the quality transfer target parameters through a channel attention mechanism to obtain the quality-guided enhancement features. Perform an inverse projection transformation on the quality-guided enhancement features to obtain an enhanced intermediate representation, and perform an upsampling operation on the enhanced intermediate representation to restore the spatial resolution to the registered infrared image to obtain an enhanced feature map; The registered infrared image and the enhanced feature map are fused pixel by pixel, and the quality improvement component in the enhanced feature map is superimposed onto the corresponding pixel position of the registered infrared image through a residual connection method to generate the quality transfer enhanced image.

[0050] After image registration, quality transfer enhancement processing is performed on the registered infrared image to address the quality differences between visible and infrared images. The visible light quality metric Q_v and the infrared quality metric Q_i are obtained by parsing the quality-aware encoded vector, and the difference quantification index ΔQ = |Q_v - Q_i| is calculated. When ΔQ exceeds a preset threshold of 0.15, a quality difference conversion rule is constructed. This rule analyzes the local change energy characteristics of the visible light image, uses a gradient operator to calculate the gray-level change amplitude within the pixel neighborhood, and obtains the energy distribution matrix. Simultaneously, an edge sharpness index is extracted, and the edge sharpness is evaluated using the Laplacian operator response intensity. The energy distribution matrix and sharpness index are mapped to quality transfer target parameters, which include an enhancement intensity coefficient in the spatial domain and a detail preservation weight in the frequency domain.

[0051] Infrared projection features corresponding to the infrared image are separated from the modal decoupling feature representation. These features have dimensions of C × H × W, where C represents the number of channels. When applying the spatial attention mechanism, attention weights are calculated for each spatial location of the infrared projection features. Specifically, global average pooling and global max pooling operations are performed to obtain spatial context information. The results of these two pooling operations are then input into a convolutional layer to generate a spatial attention map. The attention map is scaled and adjusted according to the enhancement intensity coefficient in the quality transfer target parameters, giving higher spatial weights to regions with significant quality differences. The channel attention mechanism operates on the feature channel dimension. After compressing the spatial dimension through global pooling, it passes through two fully connected layers to learn the dependencies between channels. The channel response intensity is dynamically adjusted based on the detail preservation weights in the target parameters, strengthening feature channels containing high-frequency information. The sequential application of these two attention mechanisms yields quality-guided enhancement features.

[0052] An inverse projection transformation is performed on the quality-guided enhancement features. This transformation is reversed using a linear transformation matrix (the opposite of that used when generating modal decoupling features) to map the features back to the original feature space, resulting in an enhanced intermediate representation. This intermediate representation retains the feature attributes adjusted by the quality-guided process. Bilinear interpolation is then used to upsample the enhanced intermediate representation, gradually restoring the spatial resolution until it matches the size of the registered infrared image. A step size of 2 is set during upsampling, and multiple upsampling stages ensure a smooth feature transition, ultimately generating an enhanced feature map that perfectly corresponds to the spatial dimensions of the registered infrared image.

[0053] When performing pixel-by-pixel fusion, the registered infrared image is denoted as I. align The enhanced feature map is denoted as F. enh Using a residual connection method, the output value at each pixel position (x, y) is calculated and then processed via I... out (x, y) = I align (x, y) + α · F enh (x, y) achieve fusion, where I out (x, y) represents the pixel intensity output value of the quality transfer enhanced image at spatial coordinates (x, y), I align (x, y) represents the original pixel value of the registered infrared image at that location, F enh (x, y) represents the enhancement component value of the enhanced feature map at that location, and α is the fusion coefficient, ranging from 0.3 to 0.7. Residual connections ensure that the basic structure of the original registered infrared image is preserved, and the quality improvement components in the enhanced feature map are additively superimposed to the corresponding pixel locations. Pixel values ​​are truncated in the fusion result to limit the value range within the valid interval, ultimately generating a quality-transfer enhanced image. This image, while maintaining the thermal radiation characteristics of the infrared image, achieves sharpness and detail close to that of a visible light image.

[0054] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0055] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0056] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent registration and quality transfer enhancement of high-altitude cross-band images, characterized in that, include: Atmospheric transmission attenuation compensation related to altitude was performed on the visible light band image and the infrared band image in the high-altitude cross-band image to obtain the compensated visible light image and the compensated infrared image. Multi-scale feature decomposition is performed on the compensated visible light image and the compensated infrared image respectively to obtain a set of visible light multi-scale features and a set of infrared multi-scale features. By constructing a cross-modal feature alignment network based on the mutual information maximization criterion and structure preservation constraints, the visible light multi-scale feature set and the infrared multi-scale feature set are projected onto a modally independent unified representation space to obtain modally decoupled feature representations, including: In the cross-modal feature alignment network, the visible light feature coding branch maps the visible light multi-scale feature set to a visible light intermediate representation through multi-layer nonlinear transformation, and the infrared feature coding branch maps the infrared multi-scale feature set to an infrared intermediate representation through multi-layer nonlinear transformation; The mutual information quantization value is calculated based on the joint distribution entropy and edge distribution entropy between the visible light intermediate representation and the infrared intermediate representation, and the maximization of the mutual information quantization value is set as the mutual information optimization objective. Extract the spatial neighborhood relationships in the visible light multi-scale feature set and the infrared multi-scale feature set, calculate the degree of preservation of the spatial neighborhood relationships between the visible light intermediate representation and the infrared intermediate representation after projection, and set maximizing the degree of preservation of the spatial neighborhood relationships as the structure preservation optimization objective; The mutual information optimization objective is combined with the structure preservation optimization objective to obtain a joint optimization objective. By iteratively optimizing the joint optimization objective, the transformation parameters of the visible light feature coding branch and the infrared feature coding branch are updated, and the visible light multi-scale feature set and the infrared multi-scale feature set are projected and transformed respectively to obtain visible light projection features and infrared projection features located in a unified representation space. The visible light projection features and infrared projection features are organized into the modal decoupling feature representation. Based on the modal decoupling feature representation, the geometric transformation relationship between the visible light band image and the infrared band image is calculated to obtain the registration transformation matrix; The infrared band image is subjected to spatial geometric transformation according to the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image. Quality attribute analysis is performed on the visible light band image and the registered infrared image to obtain a quality-aware coding vector. Based on the quality-aware coding vector and the modal decoupling feature representation, the registered infrared image is adaptively enhanced by using the target quality features of the visible light band image as a guiding condition, thereby generating a quality transfer enhanced image.

2. The method according to claim 1, characterized in that, Altitude-dependent atmospheric transmission attenuation compensation is performed on the visible light and infrared images in the high-altitude cross-band images to obtain compensated visible light and infrared images, respectively: Obtain the atmospheric pressure and atmospheric density values ​​at the current altitude layer, and calculate the atmospheric attenuation coefficient in the visible light band and the atmospheric attenuation coefficient in the infrared band based on the atmospheric pressure and atmospheric density values, respectively. Spatial domain compensation processing is performed on the visible light band image. The original radiation intensity value of each pixel is extracted by traversing each pixel position. The original radiation intensity value is multiplied by the reciprocal of the atmospheric attenuation coefficient of the visible light band. The product result is exponentially compensated according to the atmospheric path length to obtain the compensated radiation intensity value and write it into the corresponding pixel position. After all pixels are processed, the compensated visible light image is obtained. The infrared band image is subjected to frequency domain compensation processing. The infrared band image is converted into a frequency domain representation through frequency domain transformation and decomposed into multiple frequency components. For each frequency component, a frequency-related compensation weight coefficient is calculated based on the atmospheric attenuation coefficient of the infrared band. The frequency component is multiplied by the compensation weight coefficient to obtain the compensated frequency component. All compensated frequency components are recombined into a compensated frequency domain representation and inversely transformed to the spatial domain to obtain the compensated infrared image.

3. The method according to claim 1, characterized in that, Multi-scale feature decomposition is performed on the compensated visible light image and the compensated infrared image respectively to obtain the visible light multi-scale feature set and the infrared multi-scale feature set, including: The compensated visible light image is subjected to Laplacian pyramid decomposition to construct a multi-layer Gaussian filter downsampling sequence to obtain a low-frequency approximation layer; Differential operations are performed on low-frequency approximation layers at adjacent scales to obtain multiple detail layers with passbands. Direction-selective filtering is performed on each detail layer to separate multiple directional components. Local phase information and amplitude response information of each directional component are extracted. The directional components, local phase information and amplitude response information of all detail layers are organized into the visible light multi-scale feature set. Wavelet packet decomposition is performed on the compensated infrared image to obtain a multi-layer wavelet coefficient tree structure; Energy statistics are performed on the wavelet coefficients of each node in the wavelet coefficient tree structure to obtain the sub-band energy distribution characteristics. Correlation analysis is performed on the wavelet coefficients of adjacent scales to obtain cross-scale dependence characteristics. The wavelet coefficients, sub-band energy distribution characteristics and cross-scale dependence characteristics of all decomposed levels are organized into the infrared multi-scale feature set.

4. The method according to claim 1, characterized in that, The geometric transformation relationship between the visible light band image and the infrared band image is calculated to obtain the registration transformation matrix, which includes: The visible light projection features are subjected to corner detection based on response intensity to obtain a set of visible light anchor points, and the infrared projection features are subjected to feature point detection based on scale space extrema to obtain a set of infrared anchor points. For each visible light anchor point in the visible light anchor point set, local neighborhood window features are extracted from the visible light projection features and a multi-directional gradient statistical histogram is constructed as a visible light anchor point descriptor. For each infrared anchor point in the infrared anchor point set, local scale normalization features are extracted from the infrared projection features and a direction encoding vector is constructed as an infrared anchor point descriptor. Calculate the feature distance between the visible light anchor point descriptor and the infrared anchor point descriptor, and select the infrared anchor point with the smallest feature distance for each visible light anchor point to form an initial set of matching point pairs; Perform consistency verification on the initial set of matching point pairs, calculate the offset vector distribution characteristics of the spatial location offset vector, and retain the set of reliable matching point pairs that conform to the offset vector distribution characteristics; Based on the spatial coordinates of the set of reliable matching points, a linear constraint relationship is constructed. Based on the linear constraint relationship, the rotation parameters, translation parameters, and scaling parameters are solved and organized into the registration transformation matrix.

5. The method according to claim 1, characterized in that, The infrared band image is subjected to a spatial geometric transformation based on the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image. Quality attribute analysis is then performed on the visible light band image and the registered infrared image to obtain a quality-aware coding vector, including: The rotation, translation, and scaling parameters in the registration transformation matrix are decomposed into an affine transformation operation sequence. The pixel coordinates of the infrared band image are mapped point by point according to the affine transformation operation sequence. Bilinear interpolation is performed on the non-integer coordinate positions generated after mapping to calculate the pixel intensity value, thereby generating a registered infrared image that is spatially aligned with the visible light band image. The visible light band image is decomposed into a set of visible light frequency domain components by multi-scale decomposition. Local variation energy features and edge sharpness indicators are extracted from the set of visible light frequency domain components as visible light quality metrics. The registered infrared image is decomposed into a multi-scale set of infrared frequency domain components, and contrast dynamic range features and texture richness index are extracted from the infrared frequency domain component set as infrared quality metrics. The visible light quality metric and the infrared quality metric are concatenated according to their spatial correspondence to obtain a joint quality metric vector. The joint quality metric vector is then subjected to dimensionality reduction encoding to obtain the quality-aware encoding vector.

6. The method according to claim 5, characterized in that, Based on the quality-aware coding vector and the modality decoupled feature representation, the registered infrared image is adaptively enhanced by using the target quality features of the visible light band image as a guiding condition, generating a quality transfer enhanced image, including: The visible light quality metric and the infrared quality metric are parsed from the quality-aware encoding vector. The difference quantification index between the visible light quality metric and the infrared quality metric is calculated. A quality difference conversion rule is constructed based on the difference quantification index. The local change energy features and edge sharpness index of the visible light band image are converted into quality transfer target parameters through the quality difference conversion rule. Infrared projection features are extracted from the modal decoupling feature representation. The spatial weight distribution of the infrared projection features is dynamically adjusted according to the quality transfer target parameters through a spatial attention mechanism. The channel response intensity of the infrared projection features is dynamically adjusted according to the quality transfer target parameters through a channel attention mechanism to obtain quality-guided enhancement features. Perform an inverse projection transformation on the quality-guided enhancement features to obtain an enhanced intermediate representation, and perform an upsampling operation on the enhanced intermediate representation to restore the spatial resolution to the registered infrared image to obtain an enhanced feature map; The registered infrared image and the enhanced feature map are fused pixel by pixel, and the quality improvement component in the enhanced feature map is superimposed onto the corresponding pixel position of the registered infrared image through a residual connection method to generate the quality transfer enhanced image.

7. A high-altitude cross-band image intelligent registration and quality transfer enhancement system, used to implement the method as described in any one of claims 1-6, characterized in that, include: The compensation unit is used to perform altitude-related atmospheric transmission attenuation compensation on the visible light band image and the infrared band image in the high-altitude cross-band image, respectively, to obtain the compensated visible light image and the compensated infrared image. The decomposition unit is used to perform multi-scale feature decomposition on the compensated visible light image and the compensated infrared image respectively to obtain a set of visible light multi-scale features and a set of infrared multi-scale features. Alignment unit is used to project the visible light multi-scale feature set and the infrared multi-scale feature set onto a mode-independent unified representation space by constructing a cross-modal feature alignment network based on mutual information maximization criteria and structure preservation constraints, thereby obtaining a mode-decoupled feature representation; The registration unit is used to calculate the geometric transformation relationship between the visible light band image and the infrared band image based on the modal decoupling feature representation, and obtain the registration transformation matrix. The analysis unit is used to perform spatial geometric transformation on the infrared band image according to the registration transformation matrix to obtain a registered infrared image aligned with the visible light band image, and to perform quality attribute analysis on the visible light band image and the registered infrared image to obtain a quality-aware coding vector. An enhancement unit is used to adaptively enhance the registered infrared image based on the quality-aware coding vector and the modal decoupling feature representation, by using the target quality features of the visible light band image as a guiding condition, to generate a quality transfer enhanced image.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images