Infrared image super-resolution reconstruction method
By using high-resolution and low-resolution infrared detectors to acquire data, construct anti-degradation autoencoder and super-resolution reconstruction neural network model, the problem of poor effect of infrared image super-segment model in the existing technology is solved, and pixels are realized Level automatic alignment and image quality improvement improve the accuracy of infrared image super scores.
Patent Information
- Application Number
- CN202411256250.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-06
AI Technical Summary
In the existing infrared image super-scoring technology, the synthetic data set and the actual imaging infrared image have a large difference, resulting in poor effect of the trained super-scoring model; the real data set cannot be affected due to the different response rates of detectors and ISP imaging parameters of different resolutions. Accurate pixel-level pairing, resulting in unsatisfactory effects of the super-segment model.
High-resolution and low-resolution infrared detectors are used to acquire image data, generate mixed-quality images through degradation and upsampling, build anti-degradation autoencoder and super-resolution reconstruction neural network model, and use iterative recovery to achieve pixel-level automatic alignment and Image quality improvement.
The accuracy of infrared image super-scores is improved, and the self-restoration ability of high-definition infrared image and the clarity of low-quality images is improved. The trained model can effectively improve the resolution of infrared images.
Smart Images

Figure CN119090722A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of image processing and relates to an infrared image super-resolution reconstruction method. Background Art
[0002] Although high-resolution infrared images have higher temperature detail presentation capabilities, the manufacturing cost and difficulty of high-resolution infrared sensors are high, far from reaching consumer level. Therefore, using computer processing methods to super-resolve low-resolution infrared images to high-resolution images has become a valuable and promising technical means.
[0003] At present, the datasets used by mainstream processing methods are divided into two types: the first type is synthetic datasets, which use degradation simulation to degrade high-definition infrared images to obtain low-resolution infrared images, thereby forming a pair of learning samples. Due to the large difference between synthetic datasets and actual infrared images, the super-resolution model trained using this method cannot achieve the best results and is not competitive. The second type is real datasets, which configure the imaging parameters of high-definition and low-definition infrared sensors, perform pixel-level registration after data acquisition, and form a real infrared high- and low-definition image pair. However, due to the large differences in the response rate, NETD (noise equivalent temperature difference), and ISP imaging parameters of infrared detectors with different resolutions, accurate pixel-level pairing cannot be achieved between images, which results in the super-resolution model trained using these data with pairing errors failing to achieve the expected results. Summary of the invention
[0004] In view of the above problems, the present invention proposes an infrared image super-resolution reconstruction method and a preparation method, aiming to improve the accuracy of infrared image super-resolution.
[0005] In a first aspect, the present invention provides a method for super-resolution reconstruction of infrared images, comprising the following steps:
[0006] S1: A high-resolution infrared detector is used to collect infrared images to form a high-resolution image dataset HD; a low-resolution infrared detector is used to collect infrared images to form a low-resolution image dataset LD;
[0007] S2: sampling from the high-resolution image dataset HD to obtain a high-quality high-resolution image A, degrading the image A to obtain a low-quality low-resolution image B; upsampling the image B to obtain a low-quality high-resolution image C;
[0008] S3: Perform weighted fusion on the image C and the image A to obtain a mixed quality high-resolution image D, and construct a sample pair<D,A> Dataset; Using the<D,A> The dataset trains the degradation-resistant autoencoder E;
[0009] S4: sampling a low-quality low-resolution image F from the low-resolution image dataset LD, upsampling the image F and inputting it into the anti-degradation autoencoder E, and inputting the output of the anti-degradation autoencoder E into the anti-degradation autoencoder E again to obtain an output G; calculating the image difference L between two adjacent anti-degradation autoencoder outputs;
[0010] S5: If L is less than the threshold T, proceed to step S6; otherwise, proceed to step S4;
[0011] S6: Constructing image pairs<F,G> Dataset, using the<F,G> The dataset is used to train a neural network model for super-resolution reconstruction.
[0012] Furthermore, the image A is degraded in step S2 by selecting a second-order degradation model, setting a value range of the noise level at each stage and a value range of the blur degree of Gaussian blur to form a degradation parameter sampling space S, selecting a degradation parameter P from the degradation parameter space S, and degrading the image A using the degradation parameter P.
[0013] Furthermore, the method of upsampling the image B in step S2 includes nearest neighbor, bilinear interpolation, and cubic interpolation.
[0014] Furthermore, in step S3, the calculation formula for obtaining the mixed quality high-resolution image D by weighted fusion of the image C and the image A is: D=w*C+(1-w)*A, where w is a random number in [0,1].
[0015] Further, in step S3, the<D,A> The specific training of the anti-degradation autoencoder E with the dataset is as follows: the image D is used as the input of the anti-degradation autoencoder E, the error between the output image A' and the image A is calculated, and when the error is greater than the threshold 1, the parameters of the anti-degradation autoencoder E are adjusted.
[0016] Furthermore, when the error is less than or equal to a threshold value 1, the training of the anti-degradation autoencoder E is completed.
[0017] Furthermore, in step S4, the method for calculating the image difference L between two adjacent anti-degradation autoencoder outputs includes: L1 loss, L2 loss and multi-scale structure loss MS-SSIM.
[0018] Furthermore, the calculation formula for the image difference between two adjacent anti-degradation autoencoder outputs is specifically: L(n)=G(n)-G(n-1), where G(n) represents the nth output data, n≥2.
[0019] Further, step S6 adopts the<F,G> The data set training super-resolution reconstruction neural network model is as follows: image F is used as the input of the super-resolution reconstruction neural network model H, and the error between the output image G' and image G is calculated. When the error is greater than the threshold 2, the parameters of model H are adjusted.
[0020] Furthermore, when the error is less than or equal to a threshold value of 2, the training of the super-resolution reconstruction neural network model is completed.
[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention realizes robust high-definition infrared image self-recovery capability by constructing an anti-degradation autoencoder, and then uses an iterative recovery method to improve the clarity of low-quality infrared images, thereby obtaining high- and low-resolution infrared image pairs that are automatically aligned at the pixel level. Finally, such image pairs are used to train a super-resolution reconstruction neural network model to improve the accuracy of infrared image super-resolution. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0023] Figure 1 This is a schematic diagram of random degradation of high-definition infrared images of the present invention;
[0024] Figure 2 A schematic diagram of training an anti-degradation autoencoder according to the present invention;
[0025] Figure 3 A schematic diagram of generating a pair of high and low resolution infrared images for the present invention;
[0026] Figure 4 A schematic diagram of a neural network for super-resolution reconstruction training according to the present invention;
[0027] Figure 5 The low-quality and low-resolution infrared image of the present invention;
[0028] Figure 6 is an image after super-resolution processing of the present invention; DETAILED DESCRIPTION
[0029] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the specific embodiments described herein are only used to explain the present invention and are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0030] Example 1
[0031] The present invention provides an infrared image super-resolution reconstruction method, comprising:
[0032] A method for super-resolution reconstruction of infrared images comprises the following steps:
[0033] S1: A high-resolution infrared detector is used to collect infrared images to form a high-resolution image dataset HD; a low-resolution infrared detector is used to collect infrared images to form a low-resolution image dataset LD;
[0034] S2: sampling from the high-resolution image dataset HD to obtain a high-quality high-resolution image A, degrading the image A to obtain a low-quality low-resolution image B; upsampling the image B to obtain a low-quality high-resolution image C;
[0035] Specifically, the image A is degraded by: selecting a second-order degradation model, setting a value range of noise levels at each stage and a value range of blur degree of Gaussian blur to form a degradation parameter sampling space S, selecting a degradation parameter P from the degradation parameter space S, and degrading the image A using the degradation parameter P. Methods for upsampling the image B include nearest neighbor, bilinear interpolation, and cubic interpolation.
[0036] S3: Perform weighted fusion on the image C and the image A to obtain a mixed quality high-resolution image D, and construct a sample pair<D,A> Dataset; Using the<D,A> The dataset trains the degradation-resistant autoencoder E;
[0037] Specifically, the calculation formula for performing weighted fusion of the image C and the image A to obtain the mixed quality high-resolution image D is: D = w*C + (1-w) * A, where w is a random number in [0, 1].<D,A> The specific training of the anti-degradation autoencoder E with the dataset is as follows: the image D is used as the input of the anti-degradation autoencoder E, the error between the output image A' and the image A is calculated, and when the error is greater than the threshold 1, the parameters of the anti-degradation autoencoder E are adjusted. When the error is less than or equal to the threshold 1, the training of the anti-degradation autoencoder E is completed.
[0038] S4: sampling a low-quality low-resolution image F from the low-resolution image dataset LD, upsampling the image F and inputting it into the anti-degradation autoencoder E, and inputting the output of the anti-degradation autoencoder E into the anti-degradation autoencoder E again to obtain an output G; calculating the image difference L between two adjacent anti-degradation autoencoder outputs;
[0039] Specifically, the calculation method of the image difference L between two adjacent anti-degradation autoencoder outputs includes: L1 loss, L2 loss and multi-scale structure loss MS-SSIM. For example: the calculation formula of the image difference between two adjacent anti-degradation autoencoder outputs is specifically: L(n) = G(n)-G(n-1), where G(n) represents the nth output data, n≥2.
[0040] S5: If L is less than the threshold T, proceed to step S6; otherwise, proceed to step S4;
[0041] Specifically, if the image difference L output by two adjacent anti-degradation autoencoders is greater than or equal to the threshold T, the iteration continues until the threshold T is less than the threshold T.
[0042] S6: Constructing image pairs<F,G> Dataset, using the<F,G> The dataset is used to train a neural network model for super-resolution reconstruction.
[0043] Specifically, step S6 adopts the<F,G> The data set training super-resolution reconstruction neural network model is as follows: image F is used as the input of the super-resolution reconstruction neural network model H, and the error between the output image G' and image G is calculated. When the error is greater than the threshold 2, the parameters of the model H are adjusted. When the error is less than or equal to the threshold 2, the super-resolution reconstruction neural network model training is completed.
[0044] The present invention realizes robust self-recovery capability of high-definition infrared images by constructing an autoencoder with extensive anti-degradation capability, and then uses an iterative recovery method to improve the clarity of low-quality infrared images, thereby obtaining high-resolution and low-resolution infrared image pairs that are automatically aligned at the pixel level. Finally, such image pairs are used to train a super-resolution reconstruction neural network model to improve the accuracy of infrared image super-resolution.
[0045] Example 2
[0046] A method for processing an image using the super-resolution reconstruction neural network comprises the following steps:
[0047] S1: Collect a low-quality and low-resolution infrared image LQ at random, such as Figure 5 As shown;
[0048] S2: Input the LQ image into the super-resolution reconstruction network model H for calculation;
[0049] S3: The output result is a high-quality high-resolution image SR after super-resolution reconstruction.
[0050] After the super-resolution image LQ, the output result SR is as follows Figure 6 As shown in the figure, it can be seen from the comparison results that the super-resolution reconstruction network model of the present invention improves the accuracy of infrared image super-resolution.
[0051] The above embodiments only express the preferred implementation mode of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A method for super-resolution reconstruction of infrared images, comprising the following steps: S1: A high-resolution infrared detector is used to collect infrared images to form a high-resolution image dataset HD, and a low-resolution infrared detector is used to collect infrared images to form a low-resolution image dataset LD; S2: sampling from the high-resolution image dataset HD to obtain a high-quality high-resolution image A, degrading the image A to obtain a low-quality low-resolution image B, and upsampling the image B to obtain a low-quality high-resolution image C; S3: Perform weighted fusion on the image C and the image A to obtain a mixed quality high-resolution image D, and construct a sample pair<D,A> Dataset, using the<D,A> The dataset trains the degradation-resistant autoencoder E; S4: sampling a low-quality low-resolution image F from the low-resolution image dataset LD, upsampling the image F and inputting it into the anti-degradation autoencoder E, and inputting the output of the anti-degradation autoencoder E into the anti-degradation autoencoder E again to obtain an output G; calculating the image difference L between two adjacent anti-degradation autoencoder outputs; S5: If L is less than the threshold T, proceed to step S6; otherwise, proceed to step S4; S6: Constructing image pairs<F,G> Dataset, using the<F,G> The dataset is used to train a neural network model for super-resolution reconstruction.
2. The infrared image super-resolution reconstruction method according to claim 1, characterized in that: In step S2, the image A is degraded as follows: a second-order degradation model is selected, a value range of noise levels at each stage and a value range of blur degree of Gaussian blur are set to form a degradation parameter sampling space S, a degradation parameter P is selected from the degradation parameter space S, and the image A is degraded using the degradation parameter P.
3. The infrared image super-resolution reconstruction method according to claim 1, characterized in that: The method of upsampling the image B in step S2 includes nearest neighbor, bilinear interpolation, and cubic interpolation.
4. The infrared image super-resolution reconstruction method according to claim 1, characterized in that: In step S3, the calculation formula for obtaining the mixed quality high-resolution image D by weighted fusion of the image C and the image A is: D=w*C+(1-w)*A, where w is a random number in [0,1].
5. The infrared image super-resolution reconstruction method according to claim 4, characterized in that: In step S3, the<D,A> The specific training of the anti-degradation autoencoder E with the dataset is as follows: the image D is used as the input of the anti-degradation autoencoder E, the error between the output image A' and the image A is calculated, and when the error is greater than the threshold 1, the parameters of the anti-degradation autoencoder E are adjusted.
6. According to the infrared image super-resolution reconstruction method of claim 5, when the error is less than or equal to the threshold 1, the training of the anti-degradation autoencoder E is completed.
7. The infrared image super-resolution reconstruction method according to claim 1, characterized in that: The method for calculating the image difference L between two adjacent anti-degradation autoencoder outputs in step S4 includes: L1 loss, L2 loss and multi-scale structure loss MS-SSIM.
8. The infrared image super-resolution reconstruction method according to claim 7, characterized in that: The calculation formula for the image difference between two adjacent anti-degradation autoencoder outputs is specifically: L(n)=G(n)-G(n-1), where G(n) represents the nth output data, n≥2.
9. The infrared image super-resolution reconstruction method according to claim 1, wherein step S6 adopts the<F,G> The data set training super-resolution reconstruction neural network model is as follows: image F is used as the input of the super-resolution reconstruction neural network model H, and the error between the output image G' and image G is calculated. When the error is greater than the threshold 2, the parameters of model H are adjusted.
10. According to the infrared image super-resolution reconstruction method of claim 9, when the error is less than or equal to the threshold value 2, the super-resolution reconstruction neural network model training is completed.
