Infrared image frequency division enhancement method based on diffusion model

Abstract

The invention provides an infrared image frequency division enhancement method based on a diffusion model, and the method comprises the steps: firstly carrying out the multi-scale frequency domain decomposition of an infrared image through discrete wavelet transform, and dividing the image into a low-frequency component and a high-frequency component; in the low-frequency component processing process, a diffusion equation is constructed in combination with a thermal radiation physical model, and discretization solution is performed by using a finite difference method. The high-frequency component is converted from a space domain to a frequency domain through Fourier transform, energy distribution of the high-frequency component is analyzed, and a diffusion probability function is adjusted. In the fusion stage, according to the signal-to-noise ratio of the local area of the image, the fusion weight of the low-frequency component and the high-frequency component is dynamically adjusted, the processing results of all the local areas are integrated in a weighted fusion mode, and finally the high-quality enhanced infrared image is generated. The method has excellent performance in improving image contrast, definition and detail performance, is suitable for the fields of security monitoring, military reconnaissance, industrial detection and the like, and can effectively improve monitoring, analysis and detection effects.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an infrared image frequency division enhancement method based on a diffusion model. Background Art

[0002] Infrared imaging technology plays an indispensable role in many key areas, such as security monitoring, where it is used to monitor targets at night or in low-visibility environments; in military reconnaissance, it helps detect hidden targets and identify thermal signatures; and in industrial inspection, it helps prevent malfunctions by detecting thermal anomalies in equipment. Its principle is to image the thermal radiation characteristics of an object, which is significantly different from visible light imaging. In actual application scenarios, infrared imaging faces a series of difficult challenges. Due to the physical characteristics of the infrared imaging system itself, as well as the complex and changing external environment interference, such as atmospheric attenuation and temperature fluctuations, infrared images generally exhibit low contrast, high noise, and blurred details. This seriously restricts its further efficient application in various fields, making the accurate identification and analysis of targets much more difficult.

[0003] There are many traditional infrared image enhancement methods. Histogram equalization, for example, can improve overall image contrast to a certain extent, but it can over-enhance noise and has a poor effect on image detail, easily causing image distortion. Gaussian filtering, on the other hand, focuses primarily on noise suppression, but inevitably blurs image edges and details, leading to loss of key image information. Conventional methods often employ simple filtering techniques, such as low-pass and high-pass filtering, for image frequency separation. These methods are extremely crude in separating frequency features and cannot accurately distinguish low-frequency background information from high-frequency detail information. In actual processing, detail information is often sacrificed when suppressing noise, or noise is amplified when enhancing detail, making it difficult to find a good balance between the two.

[0004] In recent years, with the rapid development of deep learning technology, several deep learning-based image enhancement methods have been applied to infrared imaging. However, most of these methods lack a deep understanding of and effective utilization of the unique physical properties of infrared images. Their models generally generalize poorly, and their adaptability to diverse scenarios and complex environments is limited. For example, when faced with infrared images against complex backgrounds, these methods struggle to accurately extract target features, preventing them from achieving effective image enhancement. Furthermore, some existing infrared image processing methods suffer from significant limitations in computational complexity and processing efficiency, making them inadequate for applications requiring high real-time performance, such as real-time security monitoring and rapid industrial inspection. Summary of the Invention

[0005] To overcome these technical bottlenecks, this paper innovatively proposes a frequency-domain infrared image enhancement method based on a diffusion model. This method aims to fundamentally address the challenges of improving infrared image quality and meet the urgent demand for high-quality infrared images in various fields. By applying a discrete wavelet transform to perform multi-scale frequency domain decomposition, the method meticulously decomposes the infrared image into low-frequency and high-frequency components, providing a rich and accurate frequency feature foundation for subsequent targeted processing. In processing the low-frequency components, a diffusion equation is constructed based on a thermal radiation physics model, closely linking the diffusion coefficient to the local temperature gradient in the image. The diffusion equation is discretized using the finite difference method to effectively suppress noise while maximally preserving thermal structural features. In processing the high-frequency components, a frequency-domain attention mechanism is introduced into the diffusion model. The transfer probability function is adjusted based on the frequency-domain energy distribution of the high-frequency components obtained by Fourier transform, focusing on enhancing details with high frequency-domain energy. Finally, based on the signal-to-noise ratio characteristics of the local image region, the fusion weights of the low-frequency and high-frequency components are dynamically determined. Through weighted fusion and splicing of the fusion results from each local region, a high-quality enhanced infrared image is generated. Through this series of innovative steps, the method of the present invention demonstrates significant advantages in improving infrared image quality. It can effectively enhance image contrast, suppress noise, and highlight details. With good adaptability and efficiency, it can be widely used in security monitoring, military reconnaissance, industrial inspection, and many other fields that rely on infrared image analysis. The method of the present invention can achieve efficient and precise enhancement of infrared images, significantly improving the quality of infrared images and overcoming the limitations of traditional methods in noise suppression, detail enhancement, and computational efficiency.

[0006] The present invention is achieved by adopting the following technical solutions:

[0007] A method for frequency division enhancement of infrared images based on a diffusion model, the method specifically comprising the following steps:

[0008] S1: Preprocess the input infrared image and then use the multi-scale frequency domain decomposition module to perform multi-scale frequency domain decomposition on the pre-processed image using discrete wavelet transform to obtain low-frequency and high-frequency components. The low-frequency component mainly carries the large-scale structure and background characteristics of the image and is used to analyze the overall layout and main target location. The high-frequency component contains key details such as object edges and surface texture, which are crucial for target recognition and feature analysis.

[0009] S2: Based on the low-frequency constrained diffusion subnetwork, the low-frequency component obtained in step S1 is subjected to low-frequency constrained diffusion to obtain a preliminarily processed low-frequency component; a diffusion equation is constructed based on a thermal radiation physics model, and the local temperature gradient is calculated using the preliminarily processed low-frequency component to determine the diffusion coefficient in the diffusion equation; based on the preliminarily processed low-frequency component, the diffusion equation is discretized and solved using a finite difference method to obtain a processed low-frequency component; by adaptively adjusting the diffusion speed, the noise in the low-temperature background area is effectively suppressed, and the characteristics of the thermal structure edge area are maintained, thereby improving the quality of the low-frequency component;

[0010] S3: Using a high-frequency generation diffusion subnetwork to perform high-frequency generation diffusion on the high-frequency component to obtain a preliminarily processed high-frequency component; introducing a frequency-domain attention mechanism into the diffusion model, obtaining the frequency-domain energy distribution of the high-frequency component obtained in step S1 through Fourier transform and adjusting the transfer probability function accordingly, and processing the preliminarily processed high-frequency component based on the diffusion model to obtain an enhanced high-frequency component; in this process, the high-frequency detail portion obtains a higher update probability, thereby effectively enhancing the image details and improving the clarity of the high-frequency component;

[0011] S4: Calculate the signal-to-noise ratio of the low-frequency component processed in step S2 and the high-frequency component enhanced in step S3 in each local area according to the signal-to-noise ratio characteristics of the local area of ​​the image, and dynamically determine the fusion weight of the processed low-frequency component and the enhanced high-frequency component in each local area based on the signal-to-noise ratio;

[0012] S5: Using the fusion weights determined in step S4, weighted fusion is performed on the processed low-frequency components of each local area and the enhanced high-frequency components, and the fusion results of all local areas are spliced ​​to generate an enhanced infrared image.

[0013] In the above technical solution, further, the enhancement method is implemented based on an infrared image frequency division enhancement model, and the training method of the infrared image frequency division enhancement model is: using training data to optimize the network parameters of the model, and the model executes steps S1 to S5 on the input infrared image based on the optimized network parameters to obtain an enhanced infrared image.

[0014] Specifically, in step S1, the multi-scale frequency domain decomposition module uses a convolutional neural network (CNN) to extract the low-frequency and high-frequency components of the original infrared image. The operation steps are as follows:

[0015] 1) First, for the original infrared image I raw (x, y), (x, y) are image coordinates, and the two-point calibration method is used to correct the detector response (i.e., preprocess the original infrared image). Assume that the detector (such as a thermal infrared camera) is at a high temperature reference source T high and low temperature reference source Tlow The responses are R high and R low , then the preprocessed infrared image I pre (x,y) is:

[0016]

[0017] 2) Use Symlets wavelet basis (such as sym8) to pre (x, y) performs J-layer discrete wavelet transform. The number of decomposition layers J depends on the number of pixels N in the horizontal direction of the image. x and the number of pixels in the vertical direction N y OK, the formula is:

[0018]

[0019] In each layer, the low-frequency component A is obtained by discrete wavelet transform j (x, y) and high-frequency components in the horizontal, vertical, and diagonal directions j represents the jth layer; the discrete wavelet transform process is implemented through convolution operations on wavelet basis functions. The discrete wavelet transform process is completed by a series of convolution layers that simulate wavelet transform filtering. Horizontal and vertical filtering are implemented by two corresponding sets of convolution kernels, and the convolution kernel weights are determined by the Symlets wavelet basis coefficients. This module has fixed parameters and does not require extensive training.

[0020] Furthermore, in steps S2 and S3: the low-frequency constrained diffusion sub-network and the high-frequency generative diffusion sub-network have the same network structure, both consisting of a series of convolutional modules;

[0021] The low-frequency constrained diffusion sub-network is used to perform low-frequency constrained diffusion on the low-frequency component to obtain a preliminarily processed low-frequency component. The specific method is as follows: after the low-frequency component enters the first convolution module, it first passes through the convolution layer to extract features of the low-frequency component, then enters the batch normalization layer to normalize the features extracted by the convolution layer, and then processes it through the ReLU activation function; then enters the downsampling module, uses the maximum pooling operation to reduce the feature map resolution; then passes through the second convolution module with the same structure as the first convolution module to further extract features; at the same time, introduces a residual connection to add the output of the first convolution module to the output of the second convolution module; then, the feature map resolution is restored through the upsampling module, and then passes through the convolution module with the same structure as the first convolution module to further extract features to obtain a preliminarily processed low-frequency component;

[0022] The method of using the high-frequency generating diffusion sub-network to perform high-frequency generating diffusion on the high-frequency component to obtain the preliminarily processed high-frequency component is the same as the method of using the low-frequency constrained diffusion sub-network to perform low-frequency constrained diffusion on the low-frequency component.

[0023] Specifically, in step S2, a diffusion equation is constructed and discretely solved based on the thermal radiation physics model to suppress noise in the low-frequency components, especially in smooth background areas. Simultaneously, in areas with large temperature gradients, such as thermal structure edges, the diffusion rate is slowed to preserve the thermal structure characteristics, ultimately improving the quality of the low-frequency components.

[0024] In step S2, a diffusion equation is constructed based on a thermal radiation physical model, and a local temperature gradient is calculated based on the preliminarily processed low-frequency component to determine the diffusion coefficient in the diffusion equation. Based on the preliminarily processed low-frequency component, the diffusion equation is discretized and solved using a finite difference method to obtain a processed low-frequency component. The specific steps are as follows:

[0025] The low-frequency constrained diffusion subnetwork consists of multiple convolutional modules. Taking the first convolutional module as an example, it first passes through a convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1 to extract features from the low-frequency components. It then enters a batch normalization layer to normalize the convolutional features, accelerating network convergence. The ReLU activation function is then used to enhance the network's nonlinear representation capabilities. This is followed by a downsampling module using max pooling with a kernel size of 2×2 and a stride of 2 to reduce the feature map resolution and minimize computational overhead. Further feature extraction is then performed through a convolutional module with the same structure. Furthermore, a residual connection is introduced to sum the outputs of earlier convolutional modules with those of later ones, alleviating the vanishing gradient problem and helping the network learn more complex features. The upsampling module, with an upsampling factor of 2, restores the feature map resolution. Further feature extraction is then performed through a convolutional module with the same structure, yielding a preliminary processed low-frequency component.

[0026] 1) For the low-frequency component A of the preliminary processing j (x,y) is normalized, the formula is:

[0027]

[0028] in, is the normalized low-frequency component; T env is the ambient temperature parameter, which is obtained by analyzing the characteristics of the ambient area in the infrared image; T max and T min are the maximum and minimum values ​​of the scene temperature, respectively, obtained through statistical analysis of the overall temperature distribution of the infrared image;

[0029] 2) Based on the thermal radiation physical model, a diffusion equation is constructed to suppress the noise of low-frequency components and retain the thermal structure characteristics. The diffusion equation is:

[0030]

[0031] Among them, A j (x, y, t) is the value of the low-frequency component of the jth layer at (x, y) at time t; the diffusion coefficient D(x, y, t) is related to the thermal radiation gradient field of the low-frequency component of the preliminary processing Correlation, defined as:

[0032]

[0033] Among them, α and β are trainable parameters, which are optimized on the training data through the back propagation algorithm.

[0034] 3) The discretization process of the finite difference method is implemented in the network with fixed operation steps. The discretization is iteratively calculated according to the following formula to obtain the processed low-frequency component:

[0035]

[0036] in, It represents the k+1th iteration value of the low-frequency component preliminarily processed by the jth layer at the discrete grid (i, j); Δt, Δx, and Δy are fixed parameters of the diffusion equation, corresponding to the time step, horizontal spatial step, and vertical spatial step, respectively.

[0037] Specifically, in step S3, a frequency domain attention mechanism is introduced into the diffusion model to focus on enhancing the details of the high-frequency components with high frequency domain energy and optimize the quality of the high-frequency components.

[0038] The frequency domain attention mechanism is introduced into the diffusion model, and the frequency domain energy distribution of the high-frequency component obtained in step S1 is obtained by Fourier transform and the transfer probability function is adjusted accordingly. The specific method is as follows:

[0039] The high-frequency components in the horizontal, vertical and diagonal directions obtained in step S1 are For any direction component The following processing steps are performed: (a) Frequency domain energy distribution calculation:

[0040] right Perform a two-dimensional fast Fourier transform (2D-FFT) to obtain a frequency domain representation And calculate the frequency domain energy distribution

[0041]

[0042] where F{·} represents the two-dimensional discrete Fourier transform operator, Reflects the directional component In the frequency domain coordinate (f x ,f y) at the energy intensity;

[0043] (b) Frequency domain attention weight generation:

[0044] Based on frequency domain energy distribution Calculate the normalized frequency domain attention weight Used to quantify the contribution of different frequency points to detail enhancement:

[0045]

[0046] Among them, the denominator is the sum of the full frequency domain energy, ensuring that the weight w γ Normalization, the value range is [0,1], the higher the energy, the greater the corresponding weight.

[0047] The high-frequency generative diffusion subnetwork performs high-frequency generative diffusion on the high-frequency components, producing pre-processed high-frequency components. The main network of the high-frequency generative diffusion subnetwork consists of a series of convolutional modules, simulating the improved diffusion process. Taking the first convolutional module as an example, the convolution kernel size is 3×3, the stride is 1, and the padding is 1. It then passes through a batch normalization layer and a Reluctant Unit (ReLU) activation function. It then passes through a downsampling module, employing a max pooling operation with a kernel size of 2×2 and a stride of 2. Further feature extraction is performed by a convolutional module with the same structure. The output of the earlier convolutional module is added to the output of the later convolutional module via a residual connection to enhance the network's learning capabilities. It then passes through an upsampling module with an upsampling factor of 2 to restore the feature map resolution. It then passes through a convolutional module with the same structure for further feature extraction, producing the pre-processed high-frequency components (i.e., the output of the high-frequency generative diffusion subnetwork). Parameters such as the convolution kernel weights and bias are optimized using training data to fine-tune the diffusion process and highlight high-frequency details. This design enables the high-frequency diffusion process to prioritize important detail areas based on the frequency domain energy distribution, improving the quality of the high-frequency components.

[0048] (c) Transfer probability function adjustment:

[0049] The frequency domain energy distribution of the high-frequency components processed initially is used to adjust the transfer probability function. Specifically, the frequency domain attention weight w γ Incorporate the transfer probability function of the diffusion model to adjust the pixel state update probability; let the basic transfer probability function be (The appropriate probability distribution function form can be selected according to the actual situation), which means that at the tth iteration, the pixel (i, j) changes from state v t Transfer to v t+1 The probability of the adjusted transition probability function is expressed as

[0050]

[0051] in is the corresponding coordinate of pixel (i, j) in the frequency domain.

[0052] Finally, the high-frequency components processed initially are diffused based on the diffusion model to obtain enhanced high-frequency components. Through the amplification effect of the frequency domain attention weights, regions with high frequency domain energy (i.e., regions with spatial details) have a higher probability of being updated during the diffusion process, thereby achieving directional enhancement of high-frequency details.

[0053] Specifically, step S4 dynamically determines the fusion weight of the low-frequency and high-frequency components by calculating the signal-to-noise ratio of the local area of ​​the image. The signal-to-noise ratio of the low-frequency component processed in step S2 and the high-frequency component enhanced in step S3 in each local area is calculated based on the signal-to-noise ratio characteristics of the local area of ​​the image. The specific method is:

[0054] The low-frequency component processed in step S2 and the high-frequency component enhanced in step S3 are divided into multiple non-overlapping local regions R i , where i = 1, 2, 3, ..., N.

[0055] For each local region R i , let the number of pixels in the area be M i , the noise variance is σ 2 , pre-estimate the noise variance σ by statistical analysis of the training data 2 , calculate the signal-to-noise ratio SNR of the processed low-frequency component respectively L,i And the signal-to-noise ratio (SNR) of the enhanced high-frequency component H,i , the formula is:

[0056]

[0057]

[0058] Wherein, L(x, y) is the low-frequency component processed in step S2, and H′(x, y) is the high-frequency component enhanced in step S3.

[0059] Specifically, in step S4, the fusion weights of the low-frequency component and the high-frequency component in each local area are dynamically determined based on the signal-to-noise ratio. The specific operation steps are as follows:

[0060] For each local region R i , according to the low-frequency signal-to-noise ratio SNR of the area L,i and high-frequency signal-to-noise ratio SNR H,i , calculate the fusion weight w i :

[0061]

[0062] Among them, ∈ is the regularization constant to prevent the denominator from being zero; the calculated weight w i Mapped to the interval [0,1] to ensure the rationality of fusion.

[0063] Specifically, step S5 performs weighted fusion and splicing of the processed low-frequency components and enhanced high-frequency components of each local area based on the fusion weight determined in step S4 to generate an enhanced infrared image. The specific operation steps are as follows:

[0064] For each local region R i , according to the fusion weight w i The low-frequency component L(x, y) processed in step S2 and the high-frequency component H′(x, y) enhanced in step S3 are weightedly fused to obtain the local fusion result I local,i (x,y):

[0065] I local,i (x,y)=w i ·L i (x,y)+(1-w i )·H′ i (x,y)

[0066] Among them, L i (x,y) and H′ i (x,y) are the regions R i The corresponding processed low-frequency components and enhanced high-frequency components;

[0067] Finally, the fusion results of all local regions I local,i (x, y) are stitched together to obtain the final enhanced infrared image I f (x, y); when adjacent regions overlap, a weighted average method is used to smooth the transition boundary to eliminate splicing artifacts. The enhanced image achieves the best overall visual effect and information integrity.

[0068] Specifically, the above-mentioned infrared image frequency division enhancement method based on a diffusion model is implemented based on an infrared image frequency division enhancement model. The infrared image frequency division enhancement model includes a low-frequency constrained diffusion subnetwork and a high-frequency generative diffusion subnetwork, and its training process is implemented based on a joint loss function. The joint loss function comprehensively considers the noise suppression and thermal structure preservation effects of the low-frequency components, the detail enhancement degree of the high-frequency components, and the overall quality of the fused image; the network is trained using a large amount of labeled infrared image data, and the network parameters of the model are continuously adjusted through the back-propagation algorithm to minimize the joint loss function; based on the optimized network parameters, steps S1 to S5 are sequentially executed on the newly input infrared image to output a high-quality enhanced infrared image.

[0069] Construct a joint loss function L, which consists of the low-frequency loss Llow , high frequency loss L high and fusion loss L merge Composition, namely:

[0070] L=λ low L low +λ high L high +λ merge L merge

[0071] Among them, λ low ,λ high and λ merge They are low frequency loss L low , high frequency loss L high and fusion loss L merge Corresponding weights, each loss weight satisfies λ low +λ high +λ merge =1, and are both greater than 0.

[0072] L low Use mean square error to measure the low frequency component after processing With the target low frequency component A * difference:

[0073]

[0074] Among them, N l The total number of pixels representing the low-frequency component;

[0075] L high Evaluate the enhanced high-frequency component using mean square error With the target high frequency component difference:

[0076]

[0077] Among them, N h The total number of pixels representing the low-frequency component;

[0078] L merge The structural similarity index loss and mean square error are combined to measure the fusion image Enhanced image with target difference:

[0079]

[0080] Use the Adam algorithm to train the network based on the labeled data, update the network parameters θ, and iterate until L converges or the maximum number of training steps is reached. Based on the optimized parameters, perform S1 to S5 on the new infrared image and output the enhanced image.

[0081] Compared with the prior art, the present invention has the following beneficial effects:

[0082] In the field of infrared image enhancement, existing technologies, whether traditional methods or some methods based on deep learning, find it difficult to balance noise suppression, detail enhancement and computational efficiency. Traditional methods such as histogram equalization and Gaussian filtering cannot accurately balance noise and detail processing, resulting in poor image quality improvement. Some deep learning methods have poor generalization due to insufficient utilization of the physical properties of infrared images. The present invention organically integrates these key links through a series of innovative steps such as multi-scale frequency domain decomposition, low-frequency constrained diffusion based on thermal radiation physics model, high-frequency generation diffusion introducing frequency domain attention mechanism, and dynamic fusion reconstruction.

[0083] Secondly, the present invention has significant advantages in computational efficiency. Existing methods have high computational complexity and low efficiency when processing infrared images, making it difficult to meet real-time requirements. The present invention utilizes discrete wavelet transform for multi-scale frequency domain decomposition. This method is more efficient and accurate than traditional simple filtering and frequency division, and in the subsequent diffusion model processing and fusion process, unnecessary calculation steps are reduced through reasonable parameter setting and algorithm optimization. At the same time, during the training process, the design of the joint loss function enables the network parameters to converge more efficiently. Compared with the complex and inefficient processing flow in the prior art, the present invention significantly reduces the time and computing resources required for model training and reasoning when processing large-scale infrared image data, greatly improves processing efficiency, and is more adaptable to practical application scenarios with strict time requirements such as real-time security monitoring and rapid industrial detection.

[0084] Ultimately, the present invention excels in achieving both naturalness and integrity in image enhancement. Infrared images processed using traditional methods often suffer from loss of detail, residual noise, or image distortion, resulting in an unnatural overall image. This present invention utilizes a dynamic fusion reconstruction process, performing weighted fusion based on the signal-to-noise ratio characteristics of local regions. This results in an enhanced image that achieves optimal results in all regions, with a natural transition between details and the overall background. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 This is a flow chart of a method for frequency division enhancement of infrared images based on a diffusion model implemented by the present invention;

[0086] Figure 2 This is a framework diagram of the low / high frequency constrained diffusion subnetwork in a diffusion model-based infrared image frequency division enhancement method implemented by the present invention;

[0087] Figure 3 and Figure 4Schematic diagrams of the effect of the present invention on frequency division enhancement of images with low imaging quality in two different scenes, (a) is the original image, and (b) is the enhanced image. DETAILED DESCRIPTION

[0088] The specific implementation of the present invention is further described below in combination with relevant principles and processes.

[0089] like Figure 1 The figure shows a flow chart of the infrared image frequency division enhancement method based on the diffusion model of the present invention. This method can effectively solve the problems of infrared image noise, blurred details, low contrast, etc. caused by complex environmental noise interference and imaging system limitations. The method includes the following steps:

[0090] S1: Preprocess the input original infrared image, and perform multi-scale frequency domain decomposition on the preprocessed image using discrete wavelet transform to obtain low-frequency components and high-frequency components;

[0091] S2: performing low-frequency constrained diffusion on the low-frequency component using a low-frequency constrained diffusion subnetwork to obtain a preliminarily processed low-frequency component; constructing a diffusion equation based on a thermal radiation physics model, calculating a local temperature gradient using the preliminarily processed low-frequency component, and thereby determining a diffusion coefficient in the diffusion equation; based on the preliminarily processed low-frequency component, discretizing and solving the diffusion equation using a finite difference method to obtain a processed low-frequency component;

[0092] S3: using a high-frequency generation diffusion subnetwork to perform high-frequency generation diffusion on the high-frequency component to obtain a preliminarily processed high-frequency component; introducing a frequency-domain attention mechanism into the diffusion model, obtaining the frequency-domain energy distribution of the high-frequency component obtained in step S1 through Fourier transform and adjusting the transfer probability function accordingly; processing the preliminarily processed high-frequency component based on the diffusion model to obtain an enhanced high-frequency component;

[0093] S4: Calculate the signal-to-noise ratio of the low-frequency component processed in step S2 and the high-frequency component enhanced in step S3 in each local area according to the signal-to-noise ratio characteristics of the local area of ​​the image, and dynamically determine the fusion weight of the processed low-frequency component and the enhanced high-frequency component in each local area based on the signal-to-noise ratio;

[0094] S5: Using the fusion weights determined in step S4, weighted fusion is performed on the processed low-frequency components of each local area and the enhanced high-frequency components, and the fusion results of all local areas are spliced ​​to generate an enhanced infrared image.

[0095] The method is implemented based on an infrared image frequency division enhancement model. The training method of the infrared image frequency division enhancement model includes the following steps:

[0096] 1. Data Preparation: Infrared image data for training and testing was collected. The data sources included surveillance images from security monitoring scenarios, reconnaissance images from military reconnaissance, and infrared images of equipment used in industrial inspection. The collected infrared images were uniformly resized to a resolution of 512×512 pixels to ensure data consistency. Next, normalization was performed to map pixel values ​​to the [0, 1] interval, ensuring that the data was within a suitable range for subsequent calculations and processing. The training and test sets were divided into a 7:3 ratio, with the training set used for model training and the test set used for evaluating model performance.

[0097] 2. Multi-scale frequency domain decomposition module: The multi-scale frequency domain decomposition module is constructed using discrete wavelet transform (DWT). The Daubechies wavelet basis (such as db4 wavelet basis) is selected to perform J-layer (specifically 3 layers in this example) decomposition on the input infrared image. In each layer of decomposition, the image is decomposed into approximate components (low-frequency components) and detail components (high-frequency components). After the first layer of decomposition, the size of the low-frequency component is 1 / 4 of the input image. (i.e. 256×256 pixels), the high-frequency component size is the same; after the second layer decomposition, the low-frequency component size becomes (i.e. 128×128 pixels), the size of the high-frequency component also changes accordingly; after the third layer of decomposition, the size of the low-frequency component is (i.e. 64×64 pixels), and the high-frequency components are of the same size.

[0098] 3. Low-frequency constrained diffusion sub-network: Based on the low-frequency constrained diffusion sub-network, the low-frequency components are subjected to low-frequency constrained diffusion to obtain the preliminarily processed low-frequency components. Figure 2 As shown in the figure, in the low-frequency constrained diffusion subnetwork, multiple convolutional modules are sequentially implemented. Taking the first convolutional module as an example, it first passes through a convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1 to extract features from the low-frequency components. It then enters a batch normalization layer to normalize the features extracted by the convolutional layer, accelerating network convergence. The ReLU activation function is then used to enhance the network's nonlinear representation capabilities. It then passes through a downsampling module, employing a maximum pooling operation with a kernel size of 2×2 and a stride of 2 to reduce the feature map resolution and minimize computational overhead. It then passes through a convolutional module with the same structure for further feature extraction. Furthermore, a residual connection is introduced to sum the outputs of earlier convolutional modules with those of later ones, alleviating the vanishing gradient problem and helping the network learn more complex features. It then passes through an upsampling module with an upsampling factor of 2 to restore the feature map resolution. It then passes through a convolutional module with the same structure for further feature extraction, yielding the initially processed low-frequency components.

[0099] Based on the preliminarily processed low-frequency components, the local temperature gradient is then calculated, thereby determining the diffusion coefficient.

[0100] Finally, based on the low-frequency components, the diffusion equation is solved through the discretization process of the finite difference method to obtain the processed low-frequency components.

[0101] During the discretization process, the spatial step size was set to Δx = Δy = 1, and the time step size was set to Δt = 0.01. Through multiple iterative solutions, low-frequency components were processed to suppress noise while preserving thermal structural features. During training, α and β were iteratively updated using the Adam optimizer to calculate gradients based on a joint loss function. Simultaneously, a trade-off between model convergence speed and enhancement effectiveness was considered, drawing on the results of multiple experiments and the characteristics of various infrared image datasets to determine the optimal values ​​that suppress noise while maximally preserving thermal structural features.

[0102] 4. High-frequency generation and diffusion sub-network, based on which the high-frequency generation and diffusion sub-network is used to perform high-frequency generation and diffusion on the high-frequency components to obtain the high-frequency components that are preliminarily processed. Figure 2 As shown in the figure, the high-frequency generative diffusion subnetwork has the same main network structure as the low-frequency constrained diffusion subnetwork, consisting of a series of convolutional modules. Taking the first convolutional module as an example, the convolution kernel size is 3×3, the stride is 1, and the padding is 1. It then passes through a batch normalization layer and a Reluctant Unit (ReLU) activation function. It then passes through a downsampling module, using a maximum pooling operation with a pooling kernel size of 2×2 and a stride of 2. Further feature extraction is performed through convolutional modules of the same structure. The outputs of earlier and later convolutional modules are added together via residual connections to enhance the network's learning capabilities. It then passes through an upsampling module with an upsampling factor of 2 to restore the feature map resolution. Further feature extraction is performed through convolutional modules of the same structure, yielding a preliminary processed high-frequency component.

[0103] A frequency domain attention mechanism is introduced into the diffusion model, and the frequency domain energy distribution of the high-frequency component obtained in step S1 is obtained through Fourier transform and the transfer probability function is adjusted accordingly; the preliminarily processed high-frequency component is processed based on the diffusion model to obtain an enhanced high-frequency component.

[0104] The frequency domain energy distribution of the high-frequency component obtained in step S1 is obtained through Fourier transform and the transfer probability function is adjusted accordingly. Specifically, the high-frequency component is converted to the frequency domain through fast Fourier transform, and then the frequency domain attention weight is calculated and integrated into the transfer probability function.

[0105] Finally, the high-frequency components that have been preliminarily processed are processed based on the diffusion model (specifically, a diffusion iteration module may be used for multiple iterations) to obtain enhanced high-frequency components.

[0106] Incorporating frequency-domain attention weights into the transfer probability function gives higher update probabilities to regions with high frequency-domain energy. Setting the number of iterations to 50, we diffuse the high-frequency components to highlight high-frequency detail.

[0107] 5. Local Region Signal-to-Noise Ratio Calculation Module: The processed low-frequency component and enhanced high-frequency component are divided into multiple local regions, each of which is 32×32 pixels in size. For each local region, the signal-to-noise ratio of the processed low-frequency component and the signal-to-noise ratio of the enhanced high-frequency component are calculated by dividing the mean of the processed low-frequency component or the enhanced high-frequency component within the local region by the noise variance.

[0108] 6. Dynamic Fusion Reconstruction Module: Based on the signal-to-noise ratio (SNR) calculated by the local region SNR calculation module, the fusion weights of the processed low-frequency components and the enhanced high-frequency components in each local region are dynamically determined. Using a weighted fusion approach, the processed low-frequency components and the enhanced high-frequency components in each local region are fused.

[0109] 7. Network Training: During training, infrared images from the training set were fed into the network sequentially. The joint loss function hyperparameters λ1, λ2, and λ3 were set to 0.4, 0.4, and 0.2. The Adam optimizer was used, with an initial learning rate of 0.0002 and an exponential decay strategy of 0.98, which decayed every 15 epochs. Each epoch consisted of 120 batches of data, each containing 8 infrared images.

[0110] Model Testing and Application: After network training is complete, the trained network weights are loaded into the model. Infrared images from the test set are sequentially fed into the network, and image enhancement operations are performed according to the training process. In practical application scenarios, for example, in security monitoring, infrared images are enhanced to improve target recognition; in military reconnaissance, acquired infrared reconnaissance images are enhanced to facilitate clearer analysis of target features; and in industrial inspection, infrared images of equipment are enhanced to help accurately detect thermal defects and other issues.

[0111] Depend on Figure 3 and Figure 4 It can be seen that the method of the present invention can effectively improve the quality of infrared images, effectively enhance image contrast, suppress noise, and highlight details in different scenarios, and has good adaptability and efficiency.

[0112] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A method for frequency division enhancement of infrared images based on a diffusion model, characterized in that: The following steps are involved: S1: Preprocess the input original infrared image, and perform multi-scale frequency domain decomposition on the preprocessed image using discrete wavelet transform to obtain low-frequency components and high-frequency components; S2: using a low-frequency constrained diffusion sub-network to perform low-frequency constrained diffusion on the low-frequency component to obtain a preliminarily processed low-frequency component; Constructing a diffusion equation based on a thermal radiation physics model, calculating a local temperature gradient using the preliminarily processed low-frequency component, and thereby determining a diffusion coefficient in the diffusion equation; Based on the preliminarily processed low-frequency component, the diffusion equation is discretized and solved using a finite difference method to obtain a processed low-frequency component; S3: using a high-frequency generation diffusion subnetwork to perform high-frequency generation diffusion on the high-frequency component to obtain a preliminarily processed high-frequency component; introducing a frequency-domain attention mechanism into the diffusion model, obtaining the frequency-domain energy distribution of the high-frequency component obtained in step S1 through Fourier transform and adjusting the transfer probability function accordingly; processing the preliminarily processed high-frequency component based on the diffusion model to obtain an enhanced high-frequency component; S4: Calculate the signal-to-noise ratio of the low-frequency component processed in step S2 and the high-frequency component enhanced in step S3 in each local area according to the signal-to-noise ratio characteristics of the local area of ​​the image, and dynamically determine the fusion weight of the processed low-frequency component and the enhanced high-frequency component in each local area based on the signal-to-noise ratio; S5: Using the fusion weights determined in step S4, weighted fusion is performed on the processed low-frequency components of each local area and the enhanced high-frequency components, and the fusion results of all local areas are spliced ​​to generate an enhanced infrared image.

2. The method according to claim 1, characterized in that The specific operation steps of step S1 are as follows: 1) First, for the original infrared image I raw (x, y), (x, y) are image coordinates, and the original infrared image is preprocessed using the two-point calibration method; the detector is set at a high temperature reference source T high and low temperature reference source T low The responses are R high and R low , then the preprocessed infrared image I pre (x,y) is: 2) Select Symlets wavelet basis pair I pre (x, y) performs J-layer discrete wavelet transform; the number of decomposition layers J depends on the number of pixels N in the horizontal direction of the image x and the number of pixels in the vertical direction N y OK, the calculation formula is: In each layer, the low-frequency component A is obtained by discrete wavelet transform j (x, y) and high-frequency components in the horizontal, vertical, and diagonal directions j represents the jth layer.

3. The method according to claim 2, characterized in that In steps S2 and S3: the network structures of the low-frequency constrained diffusion subnetwork and the high-frequency generative diffusion subnetwork are the same, and both are composed of a series of convolution modules; the low-frequency constrained diffusion subnetwork is used to perform low-frequency constrained diffusion on the low-frequency component to obtain a preliminarily processed low-frequency component, and the specific method is: after the low-frequency component enters the first convolution module, it first passes through the convolution layer to extract features of the low-frequency component, and then enters the batch normalization layer to normalize the features extracted by the convolution layer, and then processes it through the ReLU activation function; then enters the downsampling module, and uses the maximum pooling operation to reduce the feature map resolution; then passes through the second convolution module with the same structure as the first convolution module for further feature extraction; at the same time, a residual connection is introduced to add the output of the first convolution module to the output of the second convolution module; then, the feature map resolution is restored through the upsampling module, and then passes through the convolution module with the same structure as the first convolution module for further feature extraction to obtain a preliminarily processed low-frequency component; The method of using the high-frequency generating diffusion sub-network to perform high-frequency generating diffusion on the high-frequency component to obtain the preliminarily processed high-frequency component is the same as the method of using the low-frequency constrained diffusion sub-network to perform low-frequency constrained diffusion on the low-frequency component.

4. The method according to claim 3, characterized in that In step S2, a diffusion equation is constructed based on a thermal radiation physical model, and a local temperature gradient is calculated based on the preliminarily processed low-frequency component to determine the diffusion coefficient in the diffusion equation. Based on the preliminarily processed low-frequency component, the diffusion equation is discretized and solved using a finite difference method to obtain a processed low-frequency component. The specific steps are as follows: 1) Preliminary processing of low-frequency component A j (x,y) is normalized, the formula is: in, is the normalized low-frequency component; T env is the ambient temperature parameter, which is obtained by analyzing the characteristics of the ambient area in the infrared image; T max and T min are the maximum and minimum values ​​of the scene temperature, respectively, obtained through statistical analysis of the overall temperature distribution of the infrared image; 2) Constructing a diffusion equation based on the thermal radiation physical model, the diffusion equation is: Among them, A j (x, y, t) is the value of the low-frequency component of the jth layer at (x, y) at time t; the diffusion coefficient D(x, y, t) is related to the thermal radiation gradient field of the low-frequency component of the preliminary processing Correlation, defined as: Among them, α and β are trainable parameters, which are optimized on the training data through the back propagation algorithm; 3) Solve the diffusion equation by the discretization process of the finite difference method, specifically perform iterative calculation according to the following formula, and finally obtain the processed low-frequency component: in, It represents the k+1th iteration value of the low-frequency component preliminarily processed by the jth layer at the discrete grid (i, j); Δt, Δx, and Δy are fixed parameters of the diffusion equation, corresponding to the time step, horizontal spatial step, and vertical spatial step, respectively.

5. The method according to claim 2, characterized in that In step S3, the frequency domain attention mechanism is introduced into the diffusion model, the frequency domain energy distribution of the high-frequency component obtained in step S1 is obtained by Fourier transform, and the transfer probability function is adjusted accordingly. The specific method is: The high-frequency components in the horizontal, vertical and diagonal directions obtained in step S1 are For any direction component Perform the following processing steps: (a) Frequency domain energy distribution calculation: right Perform a two-dimensional discrete Fourier transform to obtain a frequency domain representation And calculate the frequency domain energy distribution where F{·} represents the two-dimensional discrete Fourier transform operator, Reflects the directional component In the frequency domain coordinate (f x ,f y ) at the energy intensity; (b) Frequency domain attention weight generation: Based on frequency domain energy distribution Calculate the normalized frequency domain attention weight Used to quantify the contribution of different frequency points to detail enhancement: Among them, the denominator is the sum of the full frequency domain energy, ensuring that the weight w γ Normalization, the value range is [0,1], the higher the energy, the greater the corresponding weight; (c) Transfer probability function adjustment: The frequency domain attention weight w γ Incorporate the transfer probability function of the diffusion model to adjust the pixel state update probability; let the basic transfer probability function be Indicates that at the tth iteration, the pixel (i, j) changes from state v t Transfer to v t+1 The probability of the adjusted transition probability function is expressed as in is the corresponding coordinate of pixel (i, j) in the frequency domain.

6. The method according to claim 1, characterized in that In step S4, the signal-to-noise ratio of the low-frequency component processed in step S2 and the high-frequency component enhanced in step S3 in each local area is calculated based on the signal-to-noise ratio characteristics of the local area of ​​the image. The specific method is: The low-frequency component processed in step S2 and the high-frequency component enhanced in step S3 are divided into multiple non-overlapping local regions R i , where i = 1, 2, 3, ..., N; For each local region R i , calculate the signal-to-noise ratio SNR of the processed low-frequency component respectively L,i And the signal-to-noise ratio (SNR) of the enhanced high-frequency component H,i : Among them, L(x,y) is the low-frequency component after processing in S2, H ′ (x, y) is the high-frequency component enhanced in step S3, M i For region R i The number of pixels within σ 2 is the noise variance.

7. The method according to claim 6, characterized in that In step S4, the fusion weights of the low-frequency component and the high-frequency component in each local area are dynamically determined based on the signal-to-noise ratio. The specific operation steps are as follows: For each local region R i , according to the low-frequency signal-to-noise ratio SNR of the area L,i and high-frequency signal-to-noise ratio SNR H,i , calculate the fusion weight w i : Among them, ∈ is the regularization constant to prevent the denominator from being zero; the calculated weight w i Mapped to the interval [0,1] to ensure the rationality of fusion.

8. The method according to claim 7, characterized in that Step S5 specifically includes the following steps: For each local region R i , according to the fusion weight w i The low-frequency component L(x, y) processed in step S2 and the high-frequency component H′(x, y) enhanced in step S3 are weightedly fused to obtain the local fusion result I local,i (x,y): I local,i (x,y)=w i ·L i (x,y)+(1-w i )·H′ i (x,y) Among them, L i (x,y) and H′ i (x,y) are the regions R i The corresponding processed low-frequency components and enhanced high-frequency components; Finally, the fusion results of all local regions I local,i (x, y) are stitched together to obtain the final enhanced infrared image I f (x, y); when adjacent regions overlap, a weighted average method is used to smooth the transition boundary to eliminate splicing traces.

9. The method according to claim 1, characterized in that It is based on an infrared image frequency division enhancement model. The training process of the infrared image frequency division enhancement model is based on a joint loss function. The joint loss function L is composed of the low-frequency loss L low , high frequency loss L high and fusion loss L merge Composition, namely: L=λ low L low +λ high L high +λ merge L merge Among them, λ low ,λ high and λ merge They are low frequency loss L low , high frequency loss L high and fusion loss L merge The corresponding weights satisfy λ low +λ high +λ merge =1, and are both greater than 0; L low Use mean square error to measure the low frequency component after processing With the target low frequency component A * difference: Among them, N l is the total number of pixels of the low-frequency component; L high Evaluate the enhanced high-frequency component using mean square error With the target high frequency component D * difference: Among them, N h is the total number of pixels of the high-frequency component; L merge The structural similarity index loss SSIM and mean square error are combined to measure the fusion image Enhanced image with target difference: The Adam algorithm is used to train the network based on the labeled data, update the network parameters, and iterate until L converges or the maximum number of training steps is reached. The training ends and a trained infrared image frequency division enhancement model is obtained.

Images