A thermal imaging super-resolution method, system, device and storage medium with adaptive contrast

Abstract

The present invention discloses a thermal imaging super-resolution method, system, device and storage medium with adaptive contrast, which is applied to the field of image processing technology, including: performing 2x bicubic interpolation on the thermal imaging image, and using a filter window with a filter window of to split it into a base layer and a detail layer; normalizing the base layer, and finding the optimal contrast control value based on a particle swarm algorithm, combining the geometric mean of the base layer, and using an S-curve to adjust the contrast of the base layer; using a window with a size of to traverse the detail layer, calculating the ratio of the pixel value in the detail layer to the pixel mean of the window where the pixel value is located, and based on the ratio, fusing the detail layer with the base layer after contrast adjustment to obtain an enhanced final image. The present invention realizes adaptive adjustment of image contrast while improving image resolution using infrared super-resolution technology, and further enhances the dynamic range and feature information of the original image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a thermal imaging super-resolution method, system, device and storage medium for adaptive contrast. Background Art

[0002] The imaging process of infrared images is to reasonably compress the dynamic range of the original 14-bit infrared image and retain and enhance the local details, so that the image can retain the effective perception of the overall scene in the 8-bit image while highlighting the possible target scene details. The infrared super-resolution technology aims to overcome or compensate for the blur and poor image quality caused by the limitations of the acquisition system and the acquisition environment itself, improve the image resolution, and provide important support for subsequent image processing such as feature extraction and information recognition.

[0003] The existing infrared super-resolution technology relies on the neural network learning and reconstruction process. There are two ways to collect training samples. One is to build a model of infrared image degradation, and the other is to collect training samples with two devices of different resolutions. Since the imaging of infrared equipment is greatly affected by temperature and hardware, there is a certain gap between the degradation model built for deep learning and the real scene. If data is collected by devices with different resolutions, more data alignment operations will be added, which cannot be produced in batches and is often inefficient. In addition, the focus of the existing infrared super-resolution technology is mainly on building a network structure learning and reconstruction process. Commonly used network structures include: GAN, CNN, Transformer, which only processes the picture resolution and sharpness items, but does not do much processing on the contrast. This leads to scenes with low dynamic range performance before super-resolution, and the dynamic range performance is still low after super-resolution, such as forests, mountains and other scenes, which are difficult to adapt. For example, the S-curve in the S-curve Retinex infrared image algorithm is not adaptive; the CLAHE used in the infrared image super-resolution reconstruction algorithm based on ERSGCNN is also not threshold adaptive.

[0004] Therefore, how to provide a thermal imaging super-resolution method, system, device and storage medium that can adaptively adjust the image contrast while focusing on using infrared super-resolution technology to improve image resolution to enhance the dynamic range and adaptive contrast of feature information of the original image is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention proposes a thermal imaging super-resolution method, system, device and storage medium with adaptive contrast.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A thermal imaging super-resolution method with adaptive contrast, comprising:

[0008] Step 1: Perform 2x bicubic interpolation on the thermal image and use the filter window as The filter window is divided into a base layer and a detail layer;

[0009] Step 2: Normalize the base layer, find the optimal contrast control value based on the particle swarm algorithm, combine the geometric mean of the base layer, and use the S-curve to adjust the contrast of the base layer;

[0010] Step 3: Use the size The window traverses the detail layer, calculates the ratio of the pixel value in the detail layer to the pixel mean value in the window where the pixel value is located, and based on the ratio, fuses the detail layer and the base layer after contrast adjustment to obtain the enhanced final image.

[0011] Optionally, in step 1, the filter window is The filter window is divided into a base layer and a detail layer as follows:

[0012]

[0013] in, As the base layer; is the image obtained by 2x bicubic interpolation; is the filter window size; For the detail layer.

[0014] Optionally, in step 2, the base layer is normalized, specifically:

[0015] Normalize the base layer to interval.

[0016] Optionally, in step 2, the optimal contrast control value is found based on the particle swarm algorithm, specifically:

[0017] Define the fitness function as follows:

[0018]

[0019] in, is the fitness function; is the information entropy of the data; Grayscale The proportion of pixels in the image; for Blocks The mean of is the variance of pixels in a block. pixels; is the pixel value; is the mean value of the pixel value; is the sum of edge strengths; is the contrast adjustment result under the contrast control value;

[0020] Use the fitness function to iterate a preset number of times and calculate the fitness function value of each particle in each iterative evolution ,when Better than the best individual history , then update ,when Better than the global historical best , then update , after a preset number of iterations, the optimal contrast control value is obtained.

[0021] Optionally, in step 2, the geometric mean of the base layer is as follows:

[0022]

[0023] in, is the geometric mean of the base layer; As the base layer.

[0024] Optionally, in step 2, the contrast of the base layer is adjusted using an S-curve as follows:

[0025]

[0026] in, is the contrast adjustment result under the optimal contrast control value; is the optimal contrast control value; As the base layer; is the geometric mean of the base layer.

[0027] Optionally, in step 3, the detail layer and the contrast-adjusted base layer are fused based on the ratio to obtain an enhanced final image, as follows:

[0028]

[0029] in, is the final image after enhancement; is the ratio; is a hyperparameter; for the detail layer; This is the base layer after contrast adjustment.

[0030] The present invention also provides an adaptive contrast thermal imaging super-resolution system using an adaptive contrast thermal imaging super-resolution method, comprising:

[0031] Splitting module: used to perform 2x bicubic interpolation on thermal imaging images and use the filter window to The filter window is divided into a base layer and a detail layer;

[0032] Base layer adjustment module: used to normalize the base layer, find the optimal contrast control value based on the particle swarm algorithm, and adjust the contrast of the base layer using the S-curve in combination with the geometric mean of the base layer;

[0033] Fusion Enhancement Module: Used to utilize The window traverses the detail layer, calculates the ratio of the pixel value in the detail layer to the pixel mean value in the window where the pixel value is located, and based on the ratio, fuses the detail layer and the base layer after contrast adjustment to obtain the enhanced final image.

[0034] The present invention also provides an electronic device, comprising:

[0035] Memory for storing computer programs;

[0036] A processor is used to implement the steps of an adaptive contrast thermal imaging super-resolution method when executing a computer program.

[0037] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a thermal imaging super-resolution method with adaptive contrast are implemented.

[0038] It can be seen from the above technical solutions that, compared with the prior art, the present invention proposes a thermal imaging super-resolution method, system, device and storage medium with adaptive contrast. The particle swarm algorithm is used to find the contrast control value of the base layer under the optimal fitness function value to perform S-curve contrast adjustment on the base layer, and based on the ratio of the pixel value in the detail layer to the pixel mean of the window where the pixel value is located, the detail layer and the base layer after contrast adjustment are fused to obtain the enhanced final image, which realizes the adaptive adjustment of image contrast while improving the image resolution using infrared super-resolution technology, and further enhances the dynamic range and feature information of the original image. At the same time, the present invention can be used for batch generation of training data for deep learning, which is convenient for network training iteration, and the network can be fine-tuned according to the scene change data sample, and it has strong portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Embodiment 1:

[0043] Embodiment 1 of the present invention discloses a thermal imaging super-resolution method with adaptive contrast, such as Figure 1 As shown, including:

[0044] Step 1: The effective range of the data output from the thermal imaging host computer is 14 bits. Perform 2x bicubic interpolation on the thermal imaging image and use the filter window as The filter window is divided into a base layer and a detail layer.

[0045] The filter window is The filter window is divided into a base layer and a detail layer as follows:

[0046]

[0047] in, As the base layer; is the image obtained after 2 times bicubic interpolation; is the filter window size; For the detail layer.

[0048] Step 2: Normalize the base layer, find the optimal contrast control value based on the particle swarm algorithm, combine the geometric mean of the base layer, and use the S-curve to adjust the contrast of the base layer.

[0049] Normalize the base layer as follows:

[0050] Normalize the base layer to interval.

[0051] Find the optimal contrast control value based on the particle swarm algorithm, specifically:

[0052] Define the fitness function as follows:

[0053]

[0054] in, is the fitness function; is the information entropy of the data; Grayscale The proportion of pixels in the image; for Blocks The mean of is the variance of pixels in a block. pixels; is the pixel value; is the mean value of the pixel value; is the sum of edge strengths; is the contrast adjustment result under the contrast control value;

[0055] Use the fitness function to iterate a preset number of times and calculate the fitness function value of each particle in each iterative evolution ,when Better than the best individual history , then update ,when Better than the global historical best , then update , after a preset number of iterations, the optimal contrast control value is obtained.

[0056] The geometric mean of the base layer is as follows:

[0057]

[0058] in, is the geometric mean of the base layer; As the base layer.

[0059] Use the S-curve to adjust the contrast of the base layer as follows:

[0060]

[0061] in, is the contrast adjustment result under the optimal contrast control value; is the optimal contrast control value; As the base layer; is the geometric mean of the base layer.

[0062] Step 3: Use the size The window traverses the detail layer, calculates the ratio of the pixel value in the detail layer to the pixel mean value in the window where the pixel value is located, and based on the ratio, fuses the detail layer and the base layer after contrast adjustment to obtain the enhanced final image.

[0063] Based on the ratio, the detail layer and the contrast-adjusted base layer are fused to obtain the enhanced final image, as follows:

[0064]

[0065] in, is the final image after enhancement; is the ratio; It is a hyperparameter used to fine-tune the fusion ratio and control the noise performance to a certain extent. for the detail layer; This is the base layer after contrast adjustment.

[0066] Embodiment 2:

[0067] Embodiment 2 of the present invention further provides an adaptive contrast thermal imaging super-resolution system using an adaptive contrast thermal imaging super-resolution method, comprising:

[0068] Splitting module: used to perform 2x bicubic interpolation on thermal imaging images and use the filter window to The filter window is divided into a base layer and a detail layer;

[0069] Base layer adjustment module: used to normalize the base layer, find the optimal contrast control value based on the particle swarm algorithm, and adjust the contrast of the base layer using the S-curve in combination with the geometric mean of the base layer;

[0070] Fusion Enhancement Module: Used to utilize The window traverses the detail layer, calculates the ratio of the pixel value in the detail layer to the pixel mean value in the window where the pixel value is located, and based on the ratio, fuses the detail layer and the base layer after contrast adjustment to obtain the enhanced final image.

[0071] Embodiment 3:

[0072] Embodiment 3 of the present invention further provides an electronic device, including:

[0073] Memory for storing computer programs;

[0074] A processor is used to implement the steps of an adaptive contrast thermal imaging super-resolution method when executing a computer program.

[0075] Embodiment 4:

[0076] Embodiment 4 of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a thermal imaging super-resolution method with adaptive contrast are implemented.

[0077] The embodiment of the present invention discloses a thermal imaging super-resolution method, system, device and storage medium with adaptive contrast. The particle swarm algorithm is used to find the contrast control value of the base layer under the optimal fitness function value to perform S-curve contrast adjustment on the base layer, and based on the ratio of the pixel value in the detail layer to the pixel mean of the window where the pixel value is located, the detail layer and the base layer after contrast adjustment are fused to obtain the enhanced final image, so as to achieve adaptive adjustment of image contrast while improving image resolution using infrared super-resolution technology, and further enhance the dynamic range and feature information of the original image. At the same time, the present invention can be used for batch generation of training data for deep learning, which is convenient for network training iteration, and the network can be fine-tuned according to scene change data samples, and it has strong portability.

[0078] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0079] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A thermal imaging super-resolution method with adaptive contrast, characterized in that: include: Step 1: Perform a 2x bicubic interpolation on the thermal imaging image and split it into a base layer and a detail layer using a filter window with a filter window of k; Step 2: normalize the base layer, find the optimal contrast control value based on the particle swarm algorithm, combine the geometric mean of the base layer, and use the S-curve to adjust the contrast of the base layer; Step 3: traverse the detail layer using a window of size k×k, calculate the ratio of the pixel value in the detail layer to the average value of the pixel in the window where the pixel value is located, and based on the ratio, fuse the detail layer and the base layer after the contrast adjustment to obtain the enhanced final image; In step 2, the optimal contrast control value is found based on the particle swarm algorithm, specifically: Define the fitness function as follows: f edge =∑Laplacian(Ic k ); Among them, F is the fitness function; f en is the information entropy of the data; h z is the proportion of grayscale z pixels in the image; f rms is the pixel variance f(rms a ,rms b ) mean; f(rms a , rms b ) is the variance of pixels in a block, and there are m pixels in a block. q ×n q pixels; x ij is the pixel value; is the mean value of the pixel value; f edge is the sum of edge strengths; Ic k is the contrast adjustment result under the contrast control value; The fitness function is iterated a preset number of times to calculate the fitness function value F(Ic k ), when F(Ic k ) is better than the individual historical optimal p Best , then update p Best , when F(Ic k ) is better than the global historical optimal g Best , then update g Best , after the preset number of iterations, the optimal contrast control value is obtained.

2. The method for thermal imaging super-resolution with adaptive contrast according to claim 1, characterized in that: In step 1, the filter window with a filter window of k is used to split the layer into a base layer and a detail layer as follows: Ib k =Blur(I,k); Id k =I-Ib k ; Among them, Ib k is the base layer; I is the image obtained by 2x bicubic interpolation; k is the filter window size; Id k For the detail layer.

3. The method for thermal imaging super-resolution with adaptive contrast according to claim 1, characterized in that: In step 2, the base layer is normalized, specifically: The base layer is normalized to the interval [0,1].

4. The method for thermal imaging super-resolution with adaptive contrast according to claim 1, characterized in that: In step 2, the geometric mean of the base layer is as follows: Wherein, P is the geometric mean of the base layer; Ib k As the base layer.

5. The method for thermal imaging super-resolution with adaptive contrast according to claim 1, characterized in that: In step 2, the contrast of the base layer is adjusted using an S-curve as follows: in, is the contrast adjustment result under the optimal contrast control value; t * is the optimal contrast control value; Ib k is the base layer; P is the geometric mean of the base layer.

6. The method for thermal imaging super-resolution with adaptive contrast according to claim 1, characterized in that: In step 3, based on the ratio, the detail layer and the base layer after the contrast adjustment are fused to obtain the enhanced final image, as follows: Among them, Ie k is the final image after enhancement; r ij is the ratio; is a hyperparameter; Id k is the detail layer; is the base layer after the contrast adjustment.

7. An adaptive contrast thermal imaging super-resolution system using an adaptive contrast thermal imaging super-resolution method according to any one of claims 1 to 6, characterized in that: include: Splitting module: used to perform 2x bicubic interpolation on the thermal imaging image and split it into a base layer and a detail layer using a filter window with a filter window of k; Base layer adjustment module: used to normalize the base layer, find the optimal contrast control value based on the particle swarm algorithm, and adjust the contrast of the base layer using an S-curve in combination with the geometric mean of the base layer; Fusion enhancement module: used to traverse the detail layer using a window of size k×k, calculate the ratio of the pixel value in the detail layer to the pixel mean of the window where the pixel value is located, and based on the ratio, fuse the detail layer and the base layer after the contrast adjustment to obtain the enhanced final image.

8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of an adaptive contrast thermal imaging super-resolution method as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the adaptive contrast thermal imaging super-resolution method according to any one of claims 1 to 6 are implemented.

Images