Infrared image enhancement method and system based on local phase correlation

Abstract

The application relates to the technical field of image processing, and discloses an infrared image enhancement method and system based on local phase correlation, wherein the infrared image enhancement method based on local phase correlation is used for acquiring continuous multiple infrared images, performing intelligent block division on a reference frame, calculating variance features, selecting an information-rich region of interest, independently performing phase correlation operation in each region to extract a local translation vector, obtaining global displacement estimation through weighted fusion, improving robustness by using an outlier detection algorithm, and finally realizing sub-pixel level image alignment and intelligent weighted fusion. The application is suitable for real-time enhancement processing of spaceborne infrared remote sensing images, can guarantee processing quality while meeting the resource constraint requirements of an embedded platform, and provides an efficient and reliable technical scheme for space remote sensing image processing.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an infrared image enhancement method and system based on local phase correlation. Background Technology

[0002] Infrared imaging technology has significant applications in military reconnaissance, forest fire prevention, aerospace remote sensing, and geological exploration. Compared to visible light images, infrared images can be imaged under low-light or no-light conditions and have stronger anti-interference capabilities. However, because infrared images reflect the thermal radiation information of target objects, their inherent characteristics include low contrast, blurred details, and a narrow grayscale dynamic range. These characteristics severely limit the effectiveness of infrared images in complex environments and the accuracy of subsequent automatic identification and processing.

[0003] Traditional infrared image enhancement methods mainly include single-frame processing techniques such as histogram equalization, logarithmic transformation, gamma correction, and the Retinex algorithm. While these methods are effective in improving overall image brightness and contrast, they generally suffer from problems such as over-enhancement, loss of detail, noise amplification, and edge blurring. Especially in aerospace infrared remote sensing imaging applications, due to sensor resolution limitations, varying imaging conditions, and insufficient spatial resolution, relying solely on single-frame enhancement techniques is insufficient to meet the demands of high-quality image processing. Most existing enhancement methods are designed for general visual scenarios and lack optimization for the characteristics of infrared remote sensing images, making it difficult to fully adapt to the high dynamic changes and sparse thermal feature distribution in space imaging.

[0004] While deep learning-based augmentation methods developed in recent years have achieved good results in certain scenarios, their high computational complexity, heavy reliance on computing resources, and requirement for large amounts of training data make them difficult to deploy on resource-constrained embedded platforms with high real-time requirements. Furthermore, some methods, due to their high computational complexity, lack the capability to run in real-time on satellite embedded platforms and cannot guarantee a reasonable balance between image quality and downlink efficiency under limited data transmission link bandwidth. Therefore, there is an urgent need to propose an infrared image augmentation method that is simple in structure, computationally efficient, and provides stable enhancement effects, improving image visual clarity and target identifiability while balancing computational resource consumption and deployment convenience, thus meeting the application requirements of infrared images in complex environments. Summary of the Invention

[0005] This invention provides an infrared image enhancement method and system based on local phase correlation, which solves the technical problems of low image visual clarity and target identifiability in related technologies.

[0006] This invention provides an infrared image enhancement method based on local phase correlation, comprising the following steps:

[0007] Acquire multiple consecutive frames of infrared images, divide the infrared images into sub-regions, and select the first seven high-variance regions as regions of interest through variance analysis.

[0008] Linear stretching is performed on the region of interest, phase correlation operation is performed in each region, cross power spectrum is calculated and normalized by Fourier transform, phase correlation map is obtained by inverse Fourier transform and displacement vector is extracted.

[0009] The displacement vectors obtained from multiple regions of interest are weighted and averaged according to their variance values ​​to obtain a global displacement estimate. An outlier detection algorithm based on the absolute deviation of the median is used to remove unreliable displacement estimates, and the final global displacement vector is obtained.

[0010] Based on the calculated global displacement vector, bilinear interpolation is used to achieve sub-pixel level image alignment, resulting in a registered and aligned image sequence.

[0011] The fusion weights are dynamically adjusted based on the registration confidence and signal-to-noise ratio characteristics of each pixel location, and intelligent weighted superposition fusion is performed to obtain a high-quality intermediate image after fusion.

[0012] Linear grayscale stretching is performed on the fused high-quality intermediate image, and the dynamic range stretching coefficient is calculated based on the actual minimum and maximum values ​​of the image to output an infrared enhanced image with rich details and good noise suppression.

[0013] In a preferred embodiment, in acquiring continuous multi-frame infrared images, the satellite attitude control system ensures that the imaging payload continuously observes the same area, acquires image sequences with an interval of 0.1 to 1 second, uses a Gaussian filter to remove noise from the original images, sets the filtering parameter sigma to 1.2, and performs grayscale normalization to map pixel values ​​to the range of zero to one.

[0014] In a preferred embodiment, dividing the infrared image into sub-regions and selecting the top seven regions with high variance as regions of interest through variance analysis includes: dividing the reference frame into blocks of 256 pixels, calculating the expected value of all pixel grayscale values ​​in each block as the region mean, calculating the expected value of the squares of all pixel grayscale values ​​in the region as the mean square, the variance value being equal to the mean square minus the square of the region mean, and selecting the top seven regions with the largest variance values ​​as regions of interest.

[0015] In a preferred embodiment, the linear stretching and phase correlation operation includes: statistically analyzing the minimum and maximum gray values ​​within the region of interest; performing a linear transformation according to the formula that the stretched pixel value equals the original pixel value minus the minimum value divided by the difference between the maximum and minimum values; then performing a two-dimensional Fourier transform on the two region images respectively; calculating the cross-power spectrum and normalizing it according to the normalized phase correlation formula; obtaining a phase correlation map through an inverse Fourier transform; and determining the displacement vector by finding the peak position in the phase correlation map.

[0016] In a preferred embodiment, the weighted average and outlier detection include: collecting displacement vectors of all regions of interest, normalizing the variance values ​​of each region as weight coefficients, performing a weighted average according to the formula that the weighted average displacement is equal to the sum of the products of each displacement vector and its normalized weight divided by the total weight, calculating the median and absolute deviation of all displacements, and identifying and removing displacement estimates that deviate from the median by more than three times the absolute deviation as outliers.

[0017] In a preferred embodiment, the bilinear interpolation for subpixel-level image alignment includes: determining the corresponding coordinates of the target position in the original image; finding four integer pixel positions around the coordinates; calculating four weight coefficients based on the decimal part of the target position, where the weight of the top left corner is equal to one minus the horizontal decimal part multiplied by one minus the vertical decimal part, the weight of the top right corner is equal to the horizontal decimal part multiplied by one minus the vertical decimal part, the weight of the bottom left corner is equal to one minus the horizontal decimal part multiplied by the vertical decimal part, and the weight of the bottom right corner is equal to the horizontal decimal part multiplied by the vertical decimal part; and then weighting and summing the four pixels according to their weights to obtain the target pixel value.

[0018] In a preferred embodiment, the intelligent weighted overlay fusion includes: evaluating the registration confidence of each pixel location based on phase-correlated peak intensity and local texture intensity; estimating the signal-to-noise ratio characteristics by analyzing the noise level of local regions; standardizing the phase-correlated peak intensity, local texture intensity, and noise level index respectively; calculating the fusion weight according to the formula that the confidence score equals 0.4 times the standardized peak intensity plus 0.3 times the standardized texture intensity plus 0.3 times the standardized noise index; and performing pixel-level weighted averaging on each frame image.

[0019] In a preferred embodiment, the output infrared enhanced image with rich detail and good noise suppression includes: statistically analyzing the minimum and maximum gray values ​​of the fused image, calculating the dynamic range stretching coefficient based on the actual minimum and maximum values ​​of the image, and performing linear gray-scale stretching according to the formula that the enhanced pixel value equals the fused pixel value minus the minimum value multiplied by the stretching coefficient, thereby mapping subtle gray-scale changes to a larger output range and outputting an infrared enhanced image with rich detail and good noise suppression.

[0020] In a preferred embodiment, an infrared image enhancement method based on local phase correlation further includes a motion consistency detection mechanism, which evaluates the registration quality by calculating the normalized cross-correlation coefficient of the images before and after alignment. When the correlation coefficient is lower than a preset threshold, the phase correlation parameters are automatically adjusted or the displacement estimation calculation is recalculated to ensure the accuracy and stability of image registration.

[0021] In a preferred embodiment, a local phase correlation-based infrared image enhancement system is used to perform the above-described local phase correlation-based infrared image enhancement method, including:

[0022] The image acquisition and intelligent segmentation module is used to acquire multiple consecutive frames of infrared images, divide the infrared images into sub-regions, and select the first seven high-variance regions as regions of interest through variance analysis.

[0023] The preprocessing and phase correlation calculation module is used to perform linear stretching on the region of interest, perform phase correlation calculations in each region, calculate and normalize the cross power spectrum through Fourier transform, obtain the phase correlation map and extract the displacement vector through inverse Fourier transform;

[0024] The weighted fusion and outlier detection module is used to calculate the global displacement estimate by weighting the displacement vectors obtained from multiple regions of interest based on their variance values. An outlier detection algorithm based on the absolute deviation of the median is used to remove unreliable displacement estimates and obtain the final global displacement vector.

[0025] The image alignment module is used to achieve sub-pixel-level image alignment based on the calculated global displacement vector using bilinear interpolation, resulting in a registered and aligned image sequence.

[0026] The intelligent weighted fusion module is used to dynamically adjust the fusion weights based on the registration confidence and signal-to-noise ratio characteristics of each pixel position, and perform intelligent weighted superposition fusion to obtain a high-quality intermediate image after fusion.

[0027] The enhanced output module is used to perform linear grayscale stretching on the fused high-quality intermediate image. It calculates the dynamic range stretching coefficient based on the actual minimum and maximum values ​​of the image, and outputs an infrared enhanced image with rich details and good noise suppression.

[0028] The beneficial effects of this invention are as follows:

[0029] By employing an innovative local block processing strategy, the computational complexity of phase correlation in traditional full-image Fast Fourier Transform (FFT) is reduced. Simultaneously, the phase correlation calculations for each sub-region are independent, facilitating high-speed processing on parallel computing architectures such as Programmable Gate Arrays (FPGAs) and Graphics Processing Units (GPUs), thus shortening processing time and meeting the real-time processing requirements of spaceborne platforms. This method does not rely on a training model and adopts a Fourier transform-based framework, ensuring controllable algorithm complexity.

[0030] By leveraging redundant information between multiple frames, frequency domain registration and intelligent weighted superposition effectively improve the signal-to-noise ratio and spatial resolution of images, making the edges of ground objects clearer and the background details richer. The phase correlation method is insensitive to changes in illumination and thermal noise disturbances, and can stably adapt to the image registration requirements under complex imaging conditions. Localization improvements enhance the registration accuracy and adaptability to complex scenes. The intelligent weighted fusion mechanism dynamically adjusts the fusion weights based on the registration confidence and signal-to-noise ratio characteristics of each pixel position, effectively preserving the edge sharpness of moving targets while improving the signal-to-noise ratio of the background area, thus achieving a comprehensive improvement in image quality. Attached Figure Description

[0031] Figure 1 This is a flowchart of an infrared image enhancement method based on local phase correlation according to the present invention;

[0032] Figure 2 This is a block diagram of an infrared image enhancement system based on local phase correlation according to the present invention;

[0033] Figure 3 This is a comparison chart of the processing efficiency of the algorithm of this invention;

[0034] Figure 4 This is a radar chart showing the image quality improvement effect of the present invention;

[0035] Figure 5 This is a pie chart showing the computational resource saving effect of the present invention. Detailed Implementation

[0036] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0037] At least one embodiment of the present invention discloses an infrared image enhancement method based on local phase correlation, such as Figure 1As shown, it includes the following steps:

[0038] S100: Acquire multiple consecutive frames of infrared images, divide the infrared images into sub-regions, and select the first 7 high-variance regions as regions of interest through variance analysis.

[0039] Acquiring multiple consecutive frames of images from an infrared remote sensing payload requires a certain time interval between frames to form a basis for redundant information. Specifically, the satellite attitude control system ensures that the imaging payload continuously observes the same area, acquiring a sequence of multiple images with time intervals of 0.1 to 1 second. Each image is typically 1280 x 1024 pixels and represented using 16-bit grayscale. During image acquisition, the impact of factors such as satellite platform vibration and atmospheric disturbance on image quality needs to be considered. The payload's internal stabilization mechanism and image preprocessing unit perform preliminary noise filtering and radiometric correction on the acquired raw images.

[0040] The image is divided into several sub-regions, and the region of interest most suitable for phase correlation is determined through variance analysis. First, the reference frame image is uniformly divided into blocks, typically 256 x 256 pixels. For each candidate region, the variance feature is calculated as follows: the expected value of all pixel grayscale values ​​within the region is calculated, denoted as the region mean; the expected value of the squares of all pixel grayscale values ​​within the region is calculated, denoted as the mean squared; the variance value is equal to the mean squared minus the square of the region mean.

[0041] ;

[0042] in, This represents the regional variance value. This represents the mathematical expectation of the pixel grayscale values ​​within the region. It represents the mathematical expectation of the square of the pixel grayscale values ​​within the region;

[0043] The variance value is used to assess the information richness of a region. The larger the variance value, the richer the texture of the region, and the more suitable it is for accurate phase correlation calculations. The seven regions with the largest variance values ​​are automatically selected as regions of interest. These regions typically contain features with obvious characteristics, such as building edges, river outlines, and terrain changes.

[0044] Optionally, in some embodiments, a multispectral infrared imaging payload is used to simultaneously acquire infrared images in different bands, thereby improving image quality through complementary information between bands; in addition, a region selection strategy based on gradient magnitude can be adopted, which identifies regions with rich edges as candidate regions of interest (ROIs) by calculating the magnitude distribution of image gradients.

[0045] S200 performs linear stretching on the region of interest, performs phase correlation operations in each region, calculates and normalizes the cross power spectrum through Fourier transform, obtains the phase correlation map through inverse Fourier transform, and extracts the displacement vector.

[0046] Linear stretching is applied to each selected region of interest to enhance local contrast. The purpose of linear stretching is to standardize the grayscale dynamic range of each region and improve the accuracy of phase correlation calculation. To avoid numerical instability caused by an excessively small dynamic range, the dynamic range needs to be preprocessed: when the difference between the maximum and minimum grayscale values ​​in a region is less than a preset threshold (usually 10), the dynamic range of that region is set to the preset minimum dynamic range value to ensure the numerical stability of the stretching coefficient.

[0047] The specific process is as follows: Calculate the minimum and maximum grayscale values ​​within the region; check and adjust the dynamic range to prevent division by zero or numerical overflow; calculate the stretching coefficient, which is equal to the maximum dynamic range divided by the current dynamic range; perform a linear transformation on each pixel within the region.

[0048] ;

[0049] in, This represents the stretched pixel grayscale value. Represents the original pixel grayscale value. and These represent the minimum and maximum grayscale values ​​within the region of interest, respectively. This indicates the target dynamic range, which is usually set to 255. Indicates the elongation coefficient;

[0050] Phase correlation operations are performed independently within each sub-region to accurately extract the local translation vector. Phase correlation is a frequency domain registration method based on Fourier transform, which has the advantages of high computational efficiency and strong noise resistance. For each region of interest, the corresponding regions of the reference frame and the target frame are extracted and processed as follows: a two-dimensional Fourier transform is performed on the two region images to obtain the corresponding frequency domain representation; the cross power spectrum of the two frequency domain images is calculated, specifically by multiplying the Fourier transform result of the first image by the conjugate of the Fourier transform result of the second image; the cross power spectrum is normalized by dividing it by the absolute value of the amplitude, retaining pure phase information.

[0051] ;

[0052] in, and These represent the two-dimensional Fourier transform results of the two regions of the image, respectively. express Complex conjugate, Represents the cross power spectrum. This represents the normalized cross-power spectrum. To prevent division by zero, the value is usually taken as a decimal. ;

[0053] The phase correlation map is obtained by performing an inverse Fourier transform on the normalized cross-power spectrum. The phase correlation map is a two-dimensional matrix, where the peak positions correspond to the displacement between the two images. By finding the position of the maximum value in the phase correlation map, the displacement vector of that region can be determined.

[0054] Optionally, in some embodiments, adaptive histogram equalization is used instead of linear stretching, or a phase correlation method based on logarithmic polar coordinate transformation is used to simultaneously detect rotation and translation.

[0055] S300: The displacement vectors obtained from multiple regions of interest are weighted and averaged according to their variance values ​​to obtain a global displacement estimate. An outlier detection algorithm based on the absolute deviation of the median is used to remove unreliable displacement estimates, and the final global displacement vector is obtained.

[0056] The displacement vectors obtained from multiple regions of interest are weighted and averaged according to their variance values ​​to obtain the final global displacement estimate. The purpose of weighted fusion is to comprehensively utilize the registration information of each region to improve the accuracy and reliability of the overall registration. The weight allocation is based on the amount of information in each region, and regions with richer information are given higher weights.

[0057] To ensure the reasonableness of weight allocation, the variance values ​​need to be normalized beforehand. Since the variance values ​​in different regions may differ by orders of magnitude, directly using the raw variance values ​​as weights may result in excessively high weights for some high-variance regions, affecting the stability of the fusion result. The normalization process uses the min-max standardization method.

[0058] ;

[0059] in This represents the normalized variance value of the i-th region. This represents the variance value of the i-th region. and These represent the minimum and maximum variance values ​​across all regions, respectively. The normalized variance values ​​range from zero to one, ensuring the numerical stability of the weight allocation.

[0060] The specific calculation process is as follows: collect the displacement estimation results of all regions of interest, including horizontal and vertical displacements; normalize the variance values ​​of each region; use the normalized variance values ​​as weighting coefficients; and calculate the weighted average displacement.

[0061] ;

[0062] in, The normalized variance of the i-th region of interest is used as a weighting coefficient. and These represent the horizontal and vertical displacement components of the i-th region, respectively. and These represent the final global horizontal and vertical displacements, respectively. This represents the total number of regions of interest.

[0063] An outlier detection algorithm based on median absolute deviation is employed to eliminate unreliable displacement estimates. In multi-region displacement estimation, some regions may generate erroneous displacement estimates due to image noise, local occlusion, or texture loss. The purpose of outlier detection is to identify and eliminate these unreliable estimates, avoiding their negative impact on the final result.

[0064] The median absolute deviation method is robust and not easily affected by outliers. The specific process is as follows: Calculate the median value of all horizontal displacements, denoted as the median of the horizontal displacement; calculate the absolute value of the difference between each horizontal displacement and the median, and obtain the median of these absolute values, denoted as the absolute deviation of the horizontal displacement; set an outlier detection threshold, typically 2 to 3 times the absolute deviation; remove displacement estimates that deviate from the median by more than the threshold.

[0065] ;

[0066] in, This represents the median value of all horizontal displacements. This represents the horizontal displacement value of each region of interest. This represents the absolute deviation of the median of the horizontal displacement. This represents the horizontal displacement of the i-th region. This indicates the total number of regions of interest.

[0067] For horizontal displacement components, an estimated displacement is identified as an outlier if it meets the following conditions:

[0068] ;

[0069] in, This represents the outlier detection threshold, which is typically set to 3.

[0070] The same outlier detection and processing is performed on the vertical displacement components. After outlier removal, a weighted fusion calculation is performed again to obtain a more reliable global displacement estimate.

[0071] Optionally, in some embodiments, robust estimation methods based on the Random Sampling Consensus (RANSAC) algorithm or outlier detection methods based on cluster analysis are employed. These methods can handle multimodal displacement distributions and are suitable for complex scenarios with multiple moving targets.

[0072] S400, based on the calculated global displacement vector, uses bilinear interpolation to achieve sub-pixel level image alignment, resulting in a registered and aligned image sequence;

[0073] Based on the calculated global displacement vector, bilinear interpolation is used to achieve subpixel-level image alignment. Image alignment is the process of registering multiple frames of images to the same spatial coordinate system. The alignment accuracy directly affects the subsequent fusion effect. Subpixel-level alignment can handle tiny displacements of less than one pixel, thus improving the registration accuracy.

[0074] Bilinear interpolation is a commonly used image resampling method. It calculates the pixel value of the target location by performing weighted interpolation on the four pixels surrounding the target location. The interpolation weights are determined based on the distance between the target location and the surrounding pixels, with a larger weight for closer pixels.

[0075] The specific process for calculating the pixel value at the target location is as follows: determine the corresponding coordinates of the target location in the original image; find the four integer pixel positions surrounding those coordinates; calculate the interpolation weights based on the distance relationship; and perform a weighted sum to obtain the target pixel value.

[0076] ;

[0077] in, This indicates the position of the aligned image. pixel values, Indicates the original image at position pixel values, These represent the weighting coefficients of bilinear interpolation. and These represent the horizontal and vertical coordinates of the target position, respectively. and The values ​​are 0 and 1 respectively, used to iterate through the four pixels surrounding the target location.

[0078] Weighting coefficient The specific calculation method is as follows: Let the coordinates of the target position be... ,in , , and If the decimal parts are respectively for the horizontal and vertical directions, then the four weighting coefficients are as follows: , , , The sum of these four weighting coefficients is always equal to 1, ensuring the numerical stability of the interpolation result.

[0079] During the image alignment process, a motion consistency detection mechanism is introduced to evaluate the registration quality by comparing the correlation between the images before and after alignment. If the correlation is low, it indicates that there may be a registration error, and parameter adjustment or recalculation is required.

[0080] Optionally, in some embodiments, a fine alignment method based on optical flow is employed. Building upon phase-correlation coarse registration, an optical flow algorithm is used to further optimize the registration accuracy of local regions. Optical flow can capture the motion vector of each pixel in an image, making it suitable for handling complex non-rigid deformations.

[0081] The S500 dynamically adjusts the fusion weights based on the registration confidence and signal-to-noise ratio characteristics of each pixel location, and performs intelligent weighted superposition fusion to obtain a high-quality intermediate image after fusion.

[0082] Based on the registration confidence and signal-to-noise ratio characteristics of each pixel position, the fusion weight is dynamically adjusted to effectively preserve the edge sharpness of moving targets while improving the signal-to-noise ratio of the background area. Image fusion is the process of combining multiple aligned images into a single high-quality image, and the fusion strategy directly affects the final image quality.

[0083] The registration confidence score is calculated based on the sharpness and consistency of the phase correlation peak. For each pixel location, its grayscale consistency in each frame image is calculated. The higher the consistency, the more accurate the registration, and the higher the confidence score is given. The signal-to-noise ratio feature is estimated by analyzing the texture complexity and noise level of the local region.

[0084] The specific process of weight calculation is as follows: evaluate the registration confidence of each pixel position in each frame of the image; analyze the signal-to-noise ratio characteristics of the local region; calculate the fusion weight by combining the confidence and signal-to-noise ratio information; and perform pixel-level weighted fusion.

[0085] Confidence assessment is determined by comprehensively analyzing multiple factors such as phase correlation peak intensity, local texture intensity, and noise level. Specifically, the sharper the phase correlation peak, the more accurate the registration; the richer the local texture, the greater the information content in the region; and the lower the noise level, the better the data quality.

[0086] The confidence assessment function is implemented as follows: calculate the phase correlation peak intensity index, and measure the uniqueness and accuracy of registration by the ratio of the peak value to the second peak value; then calculate the local texture intensity index, and use the standard deviation of the gradient magnitude to quantify the texture richness; estimate the noise level index, and evaluate the image noise by the energy distribution of high-frequency components.

[0087] Since the three indicators have different numerical ranges and dimensions, standardization preprocessing is required to ensure the effectiveness of the combination; the peak intensity indicator is normalized by dividing by the theoretical maximum ratio; the texture intensity indicator is mapped to the zero-to-one interval using the min-max normalization method; the noise level indicator is normalized after inverse transformation, so that low noise corresponds to a high score; the three standardized indicators are denoted as follows. , and The final confidence score is calculated as follows: The registration confidence score is obtained in the range of zero to one.

[0088] ;

[0089] in, Indicates the position of the fused image pixel values, Indicates the position of the image aligned to frame i. pixel values, Indicates the position of the i-th frame image. The fusion weight, and These represent the horizontal and vertical coordinates of the image, respectively. Indicates the sequence number of the image frame. This indicates the number of image frames involved in the fusion process.

[0090] Optionally, in some embodiments, a fusion strategy based on multi-scale decomposition is employed. This method first decomposes each frame of the image into frequency components at different scales, then fuses them at each scale separately, and finally reconstructs the final result. This method can better preserve the image's detailed information and edge features.

[0091] S600 performs linear grayscale stretching on the fused high-quality intermediate image, calculates the dynamic range stretching coefficient based on the actual minimum and maximum values ​​of the image, and outputs an infrared enhanced image with rich details and good noise suppression.

[0092] By optimizing multi-frame fusion processing, linear gray-level stretching is performed on the fused image. The dynamic range stretching coefficient is calculated based on the actual minimum and maximum values ​​of the image, mapping subtle gray-level changes to a larger output range. The final output is an infrared enhanced image with rich details and good noise suppression.

[0093] The purpose of linear grayscale stretching is to fully utilize the dynamic range of an image and enhance its visual effect. The fused image usually has a narrow grayscale distribution, and stretching can enhance the image's contrast and detail visibility.

[0094] The specific process of stretching is as follows: statistically analyze the minimum and maximum gray values ​​of the fused image; calculate the stretching coefficient to ensure that the stretched image makes full use of the entire gray range; perform linear transformation on each pixel; and perform necessary saturation processing to prevent overflow.

[0095] ;

[0096] in, Indicates the location of the enhanced image. pixel values, Indicates the location of the fused image pixel values, and These represent the minimum and maximum gray values ​​of the fused image, respectively. and These represent the horizontal and vertical coordinates of the image, respectively.

[0097] During the stretching process, the decimal part of the grayscale value is retained to maintain calculation accuracy, and finally, it is rounded to convert it into an integer grayscale value; the enhanced image has higher contrast, richer details, and better visual effects.

[0098] Optionally, in some embodiments, an adaptive contrast-limited histogram equalization method is used instead of simple linear stretching; this method can automatically adjust the enhancement level according to the image content, avoiding the loss of detail or noise amplification caused by over-enhancement.

[0099] like Figure 2 As shown, an infrared image enhancement system based on local phase correlation provided by the present invention includes:

[0100] The image acquisition and intelligent segmentation module is used to acquire multiple consecutive frames of infrared images, divide the infrared images into sub-regions, and select the first seven high-variance regions as regions of interest through variance analysis.

[0101] The preprocessing and phase correlation calculation module is used to perform linear stretching on the region of interest, perform phase correlation calculations in each region, calculate and normalize the cross power spectrum through Fourier transform, obtain the phase correlation map and extract the displacement vector through inverse Fourier transform;

[0102] The weighted fusion and outlier detection module is used to calculate the global displacement estimate by weighting the displacement vectors obtained from multiple regions of interest based on their variance values. An outlier detection algorithm based on the absolute deviation of the median is used to remove unreliable displacement estimates and obtain the final global displacement vector.

[0103] The image alignment module is used to achieve sub-pixel-level image alignment based on the calculated global displacement vector using bilinear interpolation, resulting in a registered and aligned image sequence.

[0104] The intelligent weighted fusion module is used to dynamically adjust the fusion weights based on the registration confidence and signal-to-noise ratio characteristics of each pixel position, and perform intelligent weighted superposition fusion to obtain a high-quality intermediate image after fusion.

[0105] The enhanced output module is used to perform linear grayscale stretching on the fused high-quality intermediate image. It calculates the dynamic range stretching coefficient based on the actual minimum and maximum values ​​of the image, and outputs an infrared enhanced image with rich details and good noise suppression.

[0106] This invention also provides an application example:

[0107] A remote sensing satellite manufacturing company has deployed the technical solution of this invention in the onboard image processing system of its new generation infrared imaging satellite. The satellite is mainly used for tasks such as forest fire monitoring, urban heat island effect analysis and agricultural growth status assessment. The infrared imaging payload carried by the satellite can acquire long-wave infrared images with a wavelength range of 8 to 12 micrometers, a spatial resolution of 10 meters, and an imaging swath width of 60 kilometers.

[0108] In practical applications, each satellite pass can acquire 3 to 5 consecutive images of the same area, with an inter-frame time interval of approximately 0.5 seconds. Due to minor vibrations of the satellite platform and atmospheric disturbances, sub-pixel-level spatial displacement exists between each image frame. Traditional single-frame image processing methods cannot fully utilize the redundancy of multi-frame information, resulting in limited image quality.

[0109] To verify the technical effectiveness of this invention, an infrared image test dataset containing various practical application scenarios was constructed. This dataset is based on the background of aerospace remote sensing applications and covers different imaging conditions and target types, as shown in Table 1.

[0110] Table 1: Composition of the Test Dataset

[0111] .

[0112] The dataset contains 68 sets of multi-frame image sequences, each set consisting of 4 consecutive images, totaling 272 images. The imaging conditions and target features of each scene are representative, enabling comprehensive verification of the algorithm's adaptability and robustness in different application environments. Image parameters under different imaging conditions are shown in Table 2.

[0113] Table 2: Image parameters under different imaging conditions

[0114] .

[0115] Table 2 details the key parameters under different imaging conditions, providing a quantitative benchmark for algorithm performance evaluation; the image quality is highest under ideal conditions and lowest under mixed conditions, forming a complete test gradient.

[0116] Technical effect verification: such as Figure 3 The graph shows a comparison of the processing efficiency of different algorithms. The horizontal axis represents different processing methods, and the vertical axis represents processing time. The graph clearly shows that the processing time of the present invention is lower than that of the traditional method, verifying the effectiveness of the local block strategy in improving computational efficiency. The traditional full-graph FFT method requires 2.35 seconds to complete the processing, while the method of the present invention only requires 0.82 seconds, which is nearly twice as fast and meets the requirements of real-time processing.

[0117] like Figure 4 As shown, the image quality improvement effect of the present invention is demonstrated from multiple dimensions. The chart includes five evaluation dimensions: contrast, sharpness, signal-to-noise ratio, edge preservation, and overall quality. The original image scores low in each dimension, forming small polygons. After processing by the present invention, the scores in each dimension are improved, forming larger polygons, indicating a comprehensive improvement in image quality. The improvement is particularly significant in contrast and sharpness.

[0118] like Figure 5 As shown, the invention demonstrates its effectiveness in saving various computing resources. The chart divides the total resource consumption into the saved portion and the remaining portion, intuitively displaying the degree of resource saving. FFT computation is reduced by 72.3%, processing time by 65.1%, memory usage by 58.8%, and power consumption by 61.7%. These data fully verify the advantages of this invention in resource optimization, providing strong support for the actual deployment of spaceborne platforms.

[0119] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. An infrared image enhancement method based on local phase correlation, characterized in that, Includes the following steps: Acquire multiple consecutive frames of infrared images, divide the infrared images into sub-regions, and select the first seven high-variance regions as regions of interest through variance analysis. Linear stretching is performed on the region of interest, phase correlation operation is performed in each region, cross power spectrum is calculated and normalized by Fourier transform, phase correlation map is obtained by inverse Fourier transform and displacement vector is extracted. The displacement vectors obtained from multiple regions of interest are weighted and averaged according to their variance values ​​to obtain a global displacement estimate. An outlier detection algorithm based on the absolute deviation of the median is used to remove unreliable displacement estimates, and the final global displacement vector is obtained. Based on the calculated global displacement vector, bilinear interpolation is used to achieve sub-pixel level image alignment, resulting in a registered and aligned image sequence. The fusion weights are dynamically adjusted based on the registration confidence and signal-to-noise ratio characteristics of each pixel location, and intelligent weighted superposition fusion is performed to obtain a high-quality intermediate image after fusion. Linear grayscale stretching is performed on the fused high-quality intermediate image, and the dynamic range stretching coefficient is calculated based on the actual minimum and maximum values ​​of the image to output an infrared enhanced image with rich details and good noise suppression.

2. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, In the process of acquiring continuous multi-frame infrared images, the satellite attitude control system ensures that the imaging payload continuously observes the same area, and acquires image sequences with an interval of 0.1 to 1 second. A Gaussian filter is used to filter out noise from the original image, with the filter parameter sigma set to 1.

2. At the same time, grayscale normalization is performed to map the pixel values ​​to the range of zero to one.

3. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, The step of dividing the infrared image into sub-regions and selecting the top seven regions with high variance as regions of interest through variance analysis includes: dividing the reference frame into blocks of 256 pixels, calculating the expected value of the grayscale values ​​of all pixels in each block as the region mean, calculating the expected value of the squares of the grayscale values ​​of all pixels in the region as the mean square, the variance value is equal to the mean square minus the square of the region mean, and selecting the top seven regions with the largest variance values ​​as regions of interest.

4. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, The linear stretching and phase correlation operations include: statistically analyzing the minimum and maximum gray values ​​within the region of interest; performing a linear transformation according to the formula that the stretched pixel value equals the original pixel value minus the minimum value and then divided by the difference between the maximum and minimum values; then performing a two-dimensional Fourier transform on the two region images respectively; calculating the cross-power spectrum and normalizing it according to the normalized phase correlation formula; obtaining the phase correlation map through inverse Fourier transform; and determining the displacement vector by finding the peak position in the phase correlation map.

5. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, The weighted average and outlier detection include: collecting displacement vectors of all regions of interest, normalizing the variance values ​​of each region as weight coefficients, performing a weighted average according to the formula that the weighted average displacement is equal to the sum of the products of each displacement vector and its normalized weight divided by the total weight, calculating the median and absolute deviation of all displacements, and identifying and removing displacement estimates that deviate from the median by more than three times the absolute deviation as outliers.

6. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, The bilinear interpolation method for subpixel-level image alignment includes: determining the corresponding coordinates of the target position in the original image; finding four integer pixel positions around these coordinates; calculating four weight coefficients based on the decimal part of the target position, where the weight of the top left corner is equal to one minus the horizontal decimal part multiplied by one minus the vertical decimal part, the weight of the top right corner is equal to the horizontal decimal part multiplied by one minus the vertical decimal part, the weight of the bottom left corner is equal to one minus the horizontal decimal part multiplied by the vertical decimal part, and the weight of the bottom right corner is equal to the horizontal decimal part multiplied by the vertical decimal part; and then weighting and summing the four pixels according to their weights to obtain the target pixel value.

7. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, The intelligent weighted overlay fusion includes: evaluating the registration confidence of each pixel position based on the phase-correlation peak intensity and local texture intensity; estimating the signal-to-noise ratio characteristics by analyzing the local noise level; standardizing the phase-correlation peak intensity, local texture intensity, and noise level index respectively; calculating the fusion weight according to the formula that the confidence score is equal to 0.4 times the standardized peak intensity plus 0.3 times the standardized texture intensity plus 0.3 times the standardized noise index; and performing pixel-level weighted averaging on each frame image.

8. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, The output infrared enhanced image with rich detail and good noise suppression includes: statistically analyzing the minimum and maximum gray values ​​of the fused image, calculating the dynamic range stretching coefficient based on the actual minimum and maximum values ​​of the image, and performing linear gray-level stretching according to the formula that the enhanced pixel value equals the fused pixel value minus the minimum value multiplied by the stretching coefficient, thus mapping subtle gray-level changes to a larger output range and outputting an infrared enhanced image with rich detail and good noise suppression.

9. The infrared image enhancement method based on local phase correlation according to claim 1, characterized in that, It also includes a motion consistency detection mechanism, which evaluates the registration quality by calculating the normalized cross-correlation coefficient of the images before and after alignment. When the correlation coefficient is lower than a preset threshold, the phase correlation parameter is automatically adjusted or the displacement estimation is recalculated to ensure the accuracy and stability of image registration.

10. An infrared image enhancement system based on local phase correlation, used to perform an infrared image enhancement method based on local phase correlation as described in any one of claims 1-9, characterized in that, include: The image acquisition and intelligent segmentation module is used to acquire multiple consecutive frames of infrared images, divide the infrared images into sub-regions, and select the first seven high-variance regions as regions of interest through variance analysis. The preprocessing and phase correlation calculation module is used to perform linear stretching on the region of interest, perform phase correlation calculations in each region, calculate and normalize the cross power spectrum through Fourier transform, obtain the phase correlation map and extract the displacement vector through inverse Fourier transform; The weighted fusion and outlier detection module is used to calculate the global displacement estimate by weighting the displacement vectors obtained from multiple regions of interest based on their variance values. An outlier detection algorithm based on the absolute deviation of the median is used to remove unreliable displacement estimates and obtain the final global displacement vector. The image alignment module is used to achieve sub-pixel-level image alignment based on the calculated global displacement vector using bilinear interpolation, resulting in a registered and aligned image sequence. The intelligent weighted fusion module is used to dynamically adjust the fusion weights based on the registration confidence and signal-to-noise ratio characteristics of each pixel position, and perform intelligent weighted superposition fusion to obtain a high-quality intermediate image after fusion. The enhanced output module is used to perform linear grayscale stretching on the fused high-quality intermediate image. It calculates the dynamic range stretching coefficient based on the actual minimum and maximum values ​​of the image, and outputs an infrared enhanced image with rich details and good noise suppression.

Images