Self-adaptive high dynamic range infrared image reconstruction algorithm

Through the adaptive high dynamic range infrared image reconstruction algorithm, the high dynamic infrared industrial camera and deep learning technology are used to solve the problem of information gradient recovery of infrared images under single exposure conditions, and efficient HDR image reconstruction is achieved, improving the image Details and contrast.

CN119067864APending Publication Date: 2024-12-03西安中科立德红外科技有限公司

Patent Information

Application Number
CN202411050532.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to effectively reconstruct infrared images with high dynamic range, especially under single exposure conditions, which leads to insufficient exposure or excessive area information gradient recovery difficulties, affecting image details and contrast.

Method used

Adaptive high dynamic range infrared image reconstruction algorithm is used to capture images through customized high dynamic infrared industrial cameras, dynamic range limit, grayscale nonlinear mapping and low-level quantization are performed, and branches are estimated using masks, global feature branches and shallow extraction branches The features are acquired, combined with the adaptive grayscale module, and the HDR infrared image is finally reconstructed through a dense residual network.

Benefits of technology

High dynamic range reconstruction of infrared images under single exposure conditions is achieved, especially in underexposed or excessive conditions, which improves the detail recovery and visual perception of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067864A_ABST
    Figure CN119067864A_ABST
Patent Text Reader

Abstract

The invention provides an adaptive high dynamic range infrared image reconstruction algorithm. The adaptive high dynamic range infrared image reconstruction algorithm comprises the following steps: step 1, shooting an HDR infrared image by using a customized high dynamic infrared industrial camera; the method comprises the following steps: making an LDR image by sequentially carrying out dynamic range limitation, gray nonlinear mapping and low-order quantization operation on an infrared image, and pairing data of the HDR image and the LDR image into a data set; step 2, sending the LDR infrared image into a reconstruction network, respectively obtaining a feature edge dynamic range feature xME, a global feature xGF and a shallow feature xF through a mask estimation branch, a global feature branch and a shallow extraction branch, connecting channels of the three features, and inputting the three features into an adaptive gray module; according to the self-adaptive high-dynamic-range infrared image reconstruction algorithm provided by the invention, the high-dynamic-range reconstruction of the infrared image under single exposure is realized, and the algorithm has a better effect on weak conditions of insufficient exposure / overexposure.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of infrared image processing, and in particular to an adaptive high dynamic range infrared image reconstruction algorithm. Background Art

[0002] Since Daguerre of France made the first practical silver plate camera in 1839, the camera has been developed for nearly two centuries, but the recording capability of common consumer camera sensors is still far inferior to the dynamic range of the real world. The image loses details in overexposed and underexposed areas. At the same time, the advancement of display technology has made the low dynamic range (LDR) images of the past no longer meet the needs of visual perception. In order to give digital photos the ability to contain more scene information, make images closer to real-world scenes, and thus meet people's needs, a high dynamic range (HDR) technology that can record a wide range of intensity information is proposed. Compared with traditional LDR images, HDR images have deeper data depth and more intensity levels. Therefore, they can retain more details in environments with a large dynamic range such as overexposure and underexposure, which is beneficial to downstream visual tasks, including segmentation, object detection, and also provides more aesthetically attractive pictures. At present, the methods for obtaining HDR data are divided into two categories: improving camera sensors to capture higher dynamic images; and reconstructing from multiple exposures and single exposure images. Traditional HDR methods are not sufficient to cope with complex LDR image situations, so the present invention focuses on a more practical and commercial single-exposure image reconstruction method based on deep learning.

[0003] Although single exposure is more in line with the requirements of practical applications than multiple exposures, due to the inherent information loss of only the regional information under a single exposure, it is difficult to enrich the information gradient of overexposed or underexposed areas and restore the slight differences in detail information during the reconstruction process. Therefore, single-image HDR reconstruction is more challenging.

[0004] HDR reconstruction for infrared images is different from RGB images. It only has a single data channel, which aggravates the impact of information loss and makes image information recovery more difficult. However, infrared images do not need to consider the impact of color ghosting, and can avoid image alignment and feature alignment operations, shifting the focus of the algorithm to adjusting the image grayscale and enhancing details and contrast.

[0005] Therefore, it is necessary to provide an adaptive high dynamic range infrared image reconstruction algorithm to solve the above technical problems. Summary of the invention

[0006] With the development of science and technology, people are no longer satisfied with low dynamic range image display, and then pursue higher visual enjoyment. High dynamic range images can effectively increase the authenticity of image data, significantly improve visual perception, and facilitate many downstream tasks such as image segmentation and image detection. Therefore, high dynamic image reconstruction technology is becoming more and more important. In view of the fact that infrared imaging is affected by imaging mechanism defects, detection equipment limitations, and easy overexposure / underexposure under conditions such as high heat source interference or space environment, low dynamic range infrared images lack detail information, low contrast and poor visual perception, which is beneficial to subsequent infrared image processing tasks. The present invention proposes a new method to reconstruct a high dynamic image from a single low dynamic range infrared input image, by estimating the lost information in bright image parts, such as bright spots, due to saturated camera sensors. Current industrial infrared cameras can already support us in collecting infrared high dynamic images to produce data sets that solve the problem.

[0007] To solve the above technical problems, the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention comprises the following steps:

[0008] Step 1: Use a customized high dynamic infrared industrial camera to shoot HDR infrared images; produce LDR images by performing dynamic range limitation, grayscale nonlinear mapping and low-bit quantization operations on the infrared images in sequence, and pair the HDR image with the LDR image data set;

[0009] Step 2: Send the LDR infrared image into the reconstruction network, and obtain the feature edge dynamic range feature xME, global feature xGF and shallow feature xF through the mask estimation branch, global feature branch and shallow extraction branch respectively. Connect the three channels and input them into the adaptive grayscale module to enrich the grayscale information under different radiation intensity adaptation, and finally reconstruct the HDR infrared image through the dense residual network;

[0010] Step 3: Save the parameter results obtained in step 2 as a weight file xx.pth, load the weight file into the reconstruction network, and input the LDR infrared image to obtain the HDR reconstructed image.

[0011] Preferably, in step one, an HDR reconstructed infrared image data set is produced.

[0012] Preferably, in step 2, three feature branches are selected and set, an improved adaptive grayscale module, and a perceptual loss is selected.

[0013] Preferably, the step one comprises: A. using a customized high dynamic infrared industrial camera to shoot an HDR infrared image; B. sequentially performing dynamic range limitation, grayscale nonlinear mapping and low-bit quantization on the infrared image.

[0014] Preferably, the method further includes dividing the image information to be reconstructed into two parts, a high frequency part and a low frequency part, training a convolutional neural network to process the two parts respectively, and fusing the processed two parts of the image.

[0015] Preferably, the image information to be reconstructed is divided into two parts, a high frequency part and a low frequency part: an infrared image to be processed is obtained, and an infrared sample image is obtained by performing image acquisition at different angles on a target scene through a high dynamic infrared industrial camera, wherein the infrared sample image includes at least two first infrared images, and image reconstruction is performed on the at least two first infrared images obtained through a reconstruction network to obtain a second infrared image, wherein the content of the first infrared image is the same as that of the second infrared image, and the resolution of the first infrared image is smaller than that of the second infrared image, and feature extraction processing is performed on the first infrared sample image and the second infrared sample image, respectively.

[0016] Preferably, the training convolutional neural network processes two parts separately: initializing the convolutional neural network, making a data set for training the convolutional neural network to obtain a low-frequency information processing model and a high-frequency information processing model, using the obtained high-frequency information processing model to perform super-resolution reconstruction on the high-frequency part, and using the obtained low-frequency information processing model to resample the low-frequency part to obtain an image of an enlarged size.

[0017] Preferably, the two processed image parts are fused: the image obtained by super-resolution reconstruction of the high-frequency part is fused with the image of the low-frequency part with a magnified size to obtain an image reconstruction result.

[0018] Preferably, the adaptive high dynamic range infrared image reconstruction algorithm is applied to an adaptive high dynamic range infrared image reconstruction system, which comprises: a data acquisition module, a network construction module, a training module and a super-resolution reconstruction module, wherein the data acquisition module is used to acquire target HDR infrared images and produce a data set; the network construction module is used to construct a super-resolution network; the training module trains a deep neural network based on the data set; the super-resolution reconstruction module uses the trained parameters to perform infrared video super-resolution reconstruction on the corrected pictures, and combines the reconstructed frame images into a video.

[0019] Preferably, it also includes an acquisition unit, which is used to control the high dynamic infrared industrial camera to acquire images of the target scene at different angles according to the received image acquisition instruction.

[0020] Compared with the related art, the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention has the following beneficial effects:

[0021] The present invention provides an adaptive high dynamic range infrared image reconstruction algorithm. The present invention firstly sends an LDR infrared image into a reconstruction network, obtains a feature edge dynamic range feature xME, a global feature xGF and a shallow feature xF respectively through a mask estimation branch, a global feature branch and a shallow extraction branch, connects the three channels and inputs them into an adaptive grayscale module to enrich the grayscale information under different radiation intensity adaptation, and finally reconstructs the HDR infrared image through a dense residual network. The method realizes the high dynamic range reconstruction of infrared images under single exposure, and has a better effect on weak underexposure / overexposure conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of the overall steps of the first embodiment of the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention;

[0023] Figure 2 It is the algorithm network structure flow chart of the present invention;

[0024] Figure 3 It is a structural diagram of the MB flow in the network of the present invention;

[0025] Figure 4 It is a FSB flow structure diagram in the network of the present invention;

[0026] Figure 5 AGB flow chart of the network of the present invention;

[0027] Figure 6 A flowchart of a second embodiment of an adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention;

[0028] Figure 7 This is a structural block diagram of the third embodiment of the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0030] First embodiment

[0031] Please refer to Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 ,in, Figure 1 A flowchart of the overall steps of the first embodiment of the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention; Figure 2 It is the algorithm network structure flow chart of the present invention; Figure 3 It is a structural diagram of the MB flow in the network of the present invention; Figure 4It is a FSB flow structure diagram in the network of the present invention; Figure 5 The figure is a structural diagram of the AGB process in the network of the present invention. The adaptive high dynamic range infrared image reconstruction algorithm comprises the following steps:

[0032] Step 1: Use a customized high dynamic infrared industrial camera to shoot HDR infrared images; produce LDR images by performing dynamic range limitation, grayscale nonlinear mapping and low-bit quantization operations on the infrared images in sequence, and pair the HDR image with the LDR image data set;

[0033] Step 2: Send the LDR infrared image into the reconstruction network, and obtain the feature edge dynamic range feature xME, global feature xGF and shallow feature xF through the mask estimation branch, global feature branch and shallow extraction branch respectively. Connect the three channels and input them into the adaptive grayscale module to enrich the grayscale information under different radiation intensity adaptation, and finally reconstruct the HDR infrared image through the dense residual network;

[0034] In this embodiment, the network structure in step 2 is as follows: Figure 2 As shown:

[0035] First, the LDR infrared image is used as the input image x, and the input image x is input into the Mask Estimation Branch (MEB) and the Global Features Branch (GFB) to obtain the features xME and xGF respectively. Then, the two output features are connected with the feature xF channel of the shallow extraction branch and input into the Adaptive Grayscale Block (AGB) to adjust the image grayscale level xAG, and finally input into the Reconstruction Block (RB) to obtain the output infrared image y. The overall process is shown in the following formula:

[0036]

[0037] In the formula, This is a channel connection operation.

[0038] A shallow feature extraction branch fConv. It uses n 3×3 convolutions to extract shallow features of the input image x, as shown in the following formula: F =f Conv (x)

[0039] Map the input image to feature space. The shallow feature extraction branch processes local details, maintaining and expanding high-frequency details.

[0040] B Mask estimation branch. The mask estimation module fME consists of two mask blocks (MB) and a Sigmod activation function. It performs high grayscale masking and low grayscale masking operations on the input image to simulate overexposure and underexposure environments and segment the image with edge dynamic range. The process is shown in the following formula:

[0041] x ME =f ME (x) = Sigmod(f M (f M (x)))

[0042] MB consists of three 3×3 convolutions, two GeLU activation functions and a 1×1 convolution residual structure. Its module structure is as follows: Figure 3 shown.

[0043]

[0044] In the formula, Conv_1 / 2 / 3 represent different 3×3 convolution kernels, GeLU represents the GeLU activation function, It is a grayscale superposition calculation, Conv1d represents 1D convolution; the polarization amount is omitted in the formula.

[0045] C global feature branch. First, it passes through 7 convolution kernels with a step size of 2, and then a global average pooling (GAP) operation is performed to obtain an abstract feature of size C×1×1 (C is the number of channels for shallow feature extraction). The image size is restored to C×W×H using a copy operation. The flowchart is as follows: Figure 4 shown.

[0046] x 1 =ReLU(Conv_4(x)+b 4 )

[0047] x GF =Replication(GAP(x 7 ))

[0048] Where conv is a 3×3 convolution with a stride of 2; ReLU is the ReLU activation function, and b is the bias. 1 =ReLU(Conv_4(x)+b 4 ) is a formula for the Conv and ReLU parts, and the remaining 6 downsampling convolution parts can be inferred by analogy, thereby obtaining the output feature x7, and then entering the following formula:

[0049] x GF =Replication(GAP(x 7 ))

[0050] Get the global branch features. The global branch is responsible for the highest level of image range characteristics, providing global overall information, which can be fused with shallow features in subsequent processing to further improve the performance of the model.

[0051] D Adaptive grayscale module. After the xME, xGF and xF features are connected through channels, they are input into the adaptive grayscale module. The adaptive grayscale module is modified from a single selective kernel (SKC). Considering the different influence ranges of infrared imaging radiation sources with different intensities, the influence range is adaptively adjusted to enrich the possible dynamic range grayscale information. Figure 5 As shown in the figure, 3×3Conv is a 3×3 convolution, 5×5Conv is a 5×5 convolution, and the step size is 1; Ffc is a fully connected operation; is a channel-by-channel multiplication operation; λ is the maximum radiation source intensity.

[0052]

[0053] x 9 =Conv_11(x)+b 11 ,x 10 =Conv 5×5 (x)+b 12

[0054]

[0055] x fc =F fc (x gp )

[0056]

[0057] E reconstruction module. It consists of 4 convolutional layers, each layer has a 3×3 convolution kernel and ReLU function, and the first 3 layers have dense residual connections. Therefore, the grayscale conditions of infrared images under various conditions of adaptability are integrated to reconstruct HDR infrared images.

[0058] y=f R (x AG )

[0059] F loss. The method uses perceptual loss, selects the reconstructed image y and the real HDR image ygt in the shallow, middle and deep layers (3, 8 and 11 layers) of the VGG network, and uses L2 processing to obtain them. Perceptual loss usually extracts low-frequency information such as edges, colors and brightness in the shallower layers, and some high-frequency information such as detailed textures in the deeper layers. The deep layers extract some key discriminative features, which can further improve the reconstruction performance in the field of dynamic range reconstruction, which has strong feature dependence.

[0060]

[0061] Where φj is the VGG feature extraction, j is the number of extracted layers, here j = 3, 8, 11.

[0062] Step 3: Save the parameter results obtained in step 2 as a weight file xx.pth, load the weight file into the reconstruction network, and input the LDR infrared image to obtain the HDR reconstructed image.

[0063] In the step 1, the HDR reconstructed infrared image data set is produced.

[0064] The three feature branch selections and settings in step 2, the improved adaptive grayscale module, and the selection of perceptual loss.

[0065] The step 1 includes: A. using a customized high dynamic infrared industrial camera to shoot an HDR infrared image; B. sequentially performing dynamic range limitation, grayscale nonlinear mapping and low-bit quantization on the infrared image.

[0066] The HDR image and the LDR image data are paired as training sets according to the naming method of “… / Data / Train / HDR / xxx” and “… / Data / Train / LDR / xxx”, where xxx is the image number and the numbers of the same scene should be the same. In addition, the same method can be used to create a small number of test sets, and the naming method of the test sets is similar.

[0067] Compared with the related art, the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention has the following beneficial effects:

[0068] The present invention first sends the LDR infrared image into the reconstruction network, and obtains the feature edge dynamic range feature xME, the global feature xGF and the shallow feature xF through the mask estimation branch, the global feature branch and the shallow extraction branch respectively. The three channels are connected and input into the adaptive grayscale module to enrich the grayscale information under different radiation intensities. Finally, the HDR infrared image is reconstructed through the dense residual network. The method realizes the high dynamic range reconstruction of infrared images under single exposure, and has better effect on weak underexposure / overexposure conditions.

[0069] Second embodiment

[0070] Please refer to Figure 6Based on the adaptive high dynamic range infrared image reconstruction algorithm provided by the first embodiment of the present application, the second embodiment of the present application proposes another adaptive high dynamic range infrared image reconstruction algorithm. The second embodiment is only a preferred embodiment of the first embodiment, and the implementation of the second embodiment will not affect the independent implementation of the first embodiment.

[0071] Specifically, the difference of the adaptive high dynamic range infrared image reconstruction algorithm provided by the second embodiment of the present application is that it also includes dividing the image information to be reconstructed into two parts, high frequency and low frequency, training a convolutional neural network to process the two parts separately, and fusing the two processed image parts.

[0072] The method divides the image information to be reconstructed into two parts, high frequency and low frequency: obtaining an infrared image to be processed, acquiring images of a target scene at different angles through a high dynamic infrared industrial camera to obtain an infrared sample image, wherein the infrared sample image includes at least two first infrared images, and reconstructing the at least two first infrared images through a reconstruction network to obtain a second infrared image, wherein the content of the first infrared image is the same as that of the second infrared image, and the resolution of the first infrared image is smaller than that of the second infrared image, and performing feature extraction processing on the first infrared sample image and the second infrared sample image respectively.

[0073] In this embodiment, when the high dynamic infrared industrial camera captures an image of a target scene, it first captures the image at the current angle, and then rotates the preset angle to capture the image of the target scene again to obtain two first infrared images.

[0074] In this embodiment, the high dynamic infrared industrial camera is controlled by a control system and connected to a mobile phone app through a Bluetooth wireless module, and the position is adjusted through the mobile terminal. It should be noted that the high dynamic infrared industrial camera moves linearly along a preset direction, including linear movement of the x, y, and z axes, to capture images at different angles.

[0075] The training convolutional neural network processes two parts respectively: initializing the convolutional neural network, making a data set for training the convolutional neural network to obtain a low-frequency information processing model and a high-frequency information processing model, using the obtained high-frequency information processing model to perform super-resolution reconstruction on the high-frequency part, and using the obtained low-frequency information processing model to resample the low-frequency part to obtain an image with an enlarged size.

[0076] In this embodiment, the image to be processed is subjected to wavelet decomposition to obtain low-frequency components and high-frequency components of the image to be processed, and the pixel value of the low-frequency component is increased according to a nonlinear stretching rule to obtain the low-frequency component after contrast enhancement, including: determining a low-frequency stretching range based on the wavelet coefficient range of the low-frequency component; and stretching the wavelet coefficients of the low-frequency component within the low-frequency stretching range to update the wavelet coefficients of the low-frequency component.

[0077] In this embodiment, the maximum value and the minimum value of the wavelet coefficient of the low-frequency component are obtained, the first difference between the wavelet coefficient of the low-frequency component within the low-frequency stretching range and the minimum value of the wavelet coefficient, and the second difference between the maximum value and the minimum value of the wavelet coefficient are obtained, and the wavelet coefficient after stretching processing is determined according to the first difference and the second difference.

[0078] In this embodiment, the high frequency component is subjected to noise reduction processing to obtain a noise-reduced high frequency component.

[0079] The two processed image parts are fused: the image obtained by super-resolution reconstruction of the high-frequency part is fused with the image of the low-frequency part with a magnified size to obtain an image reconstruction result.

[0080] In this embodiment, the high-frequency component and the low-frequency component of the first infrared image are extracted to obtain the first high-frequency component and the first low-frequency component; the high-frequency component and the low-frequency component of the second infrared image are extracted to obtain the second high-frequency component and the second low-frequency component; wherein the resolution of the first infrared image is lower than the resolution of the second infrared image; the edge strength of each pixel position in the second high-frequency component is determined; the local difference metric value of each pixel position in the high-frequency components of the first infrared image and the second infrared image is determined to obtain the first local difference metric value and the second local difference metric value of each pixel position respectively; the first high-frequency component and the second high-frequency component are weightedly fused pixel by pixel to obtain a fused high-frequency component; based on the fused high-frequency component and the fused low-frequency component, a fused image is obtained; wherein the fused low-frequency component is obtained by fusing the first low-frequency component and the second low-frequency component.

[0081] Compared with the related art, the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention has the following beneficial effects:

[0082] The infrared image of the present invention is not limited to infrared data from a single source. It is divided into high-frequency and low-frequency parts and reconstructed using super-resolution to make the details clearer. At the same time, by fusing multiple infrared image features, the texture information of visible light data is used to enhance the effect of the infrared image, which also ensures the accuracy of determining the image reconstruction features and improves the accuracy of subsequent recognition of the details of the infrared image.

[0083] Third embodiment

[0084] Please refer to Figure 7 Based on the adaptive high dynamic range infrared image reconstruction algorithm provided by the first embodiment of the present application, the third embodiment of the present application proposes another adaptive high dynamic range infrared image reconstruction algorithm. The third embodiment is only a preferred embodiment of the first embodiment, and the implementation of the third embodiment will not affect the independent implementation of the first embodiment.

[0085] Specifically, the difference of the adaptive high dynamic range infrared image reconstruction algorithm provided in the third embodiment of the present application is that it is applied to an adaptive high dynamic range infrared image reconstruction system, and the adaptive high dynamic range infrared image reconstruction system includes: a data acquisition module, a network construction module, a training module and a super-resolution reconstruction module, the data acquisition module is used to acquire the target HDR infrared image and produce a data set; the network construction module is used to build a super-resolution network; the training module trains the deep neural network based on the data set; the super-resolution reconstruction module uses the trained parameters to perform infrared video super-resolution reconstruction on the corrected picture, and combines the reconstructed frame images into a video.

[0086] In this embodiment, the network construction module constructs a super-resolution network, including a multi-scale feature alignment module, a channel attention fusion module, and an upsampling module;

[0087] In this embodiment, a sample infrared image and an image feature label corresponding to the sample infrared image are obtained; feature extraction is performed on the sample infrared image to obtain a plurality of sub-image features, and the local features corresponding to each of the sub-image features are determined; feature fusion is performed on the plurality of sub-image features to obtain a multi-layer fusion feature, and image reconstruction features are obtained based on the multi-layer fusion feature and the plurality of local features.

[0088] In this embodiment, it also includes an image splitting module, which separates the image to be reconstructed into a low-frequency part and a high-frequency part, and transmits them to a super-resolution reconstruction module. The training module trains the deep neural network based on the data set to obtain a low-frequency processing model and a high-frequency processing model and transmits the obtained modules to the super-resolution reconstruction module. The super-resolution reconstruction module transmits the low-frequency part to the low-frequency processing model to obtain a low-frequency part with an enlarged size, and transmits the high-frequency part to the high-frequency processing model to obtain a high-frequency part obtained through super-resolution reconstruction. The reconstruction module fuses the high-frequency part reconstructed by super-resolution and the low-frequency part with the enlarged size to obtain an image reconstruction result of the image information to be reconstructed.

[0089] It also includes an acquisition unit, which is used to control the high dynamic infrared industrial camera to acquire images of the target scene at different angles according to the received image acquisition instruction.

[0090] In this embodiment, the high dynamic infrared industrial camera moves linearly along a preset direction, including linear movement of the x, y, and z axes, to collect images at different angles;

[0091] The preset direction can be horizontal, but it is necessary to ensure that the high dynamic infrared industrial camera rotates along a fixed rotation axis relative to the target scene;

[0092] In another embodiment, the preset direction may be a vertical direction, but it is only necessary to ensure that the high dynamic infrared industrial camera rotates along a fixed rotation axis relative to the target scene;

[0093] In another embodiment, the preset direction may be any other direction, but it is only necessary to ensure that the high dynamic infrared industrial camera rotates along a fixed rotation axis relative to the target scene.

[0094] Compared with the related art, the adaptive high dynamic range infrared image reconstruction algorithm provided by the present invention has the following beneficial effects:

[0095] The present invention divides the infrared image information to be reconstructed into two parts, high frequency and low frequency, for processing. Super-resolution reconstruction is used for high frequency information to make the details clearer, and resampling is used for low frequency information to retain the original grayscale information. By decomposing the image to be processed, the contrast is enhanced nonlinearly at low frequency while reducing high frequency noise. Finally, the image is reconstructed to enhance the image contrast, which is beneficial to the downstream processing tasks of infrared images.

[0096] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An adaptive high dynamic range infrared image reconstruction algorithm, characterized in that: The following steps are involved: Step 1: Use a customized high dynamic infrared industrial camera to shoot HDR infrared images; produce LDR images by performing dynamic range limitation, grayscale nonlinear mapping and low-bit quantization operations on the infrared images in sequence, and pair the HDR image with the LDR image data set; Step 2: Send the LDR infrared image into the reconstruction network, and obtain the feature edge dynamic range feature xME, global feature xGF and shallow feature xF through the mask estimation branch, global feature branch and shallow extraction branch respectively. Connect the three channels and input them into the adaptive grayscale module to enrich the grayscale information under different radiation intensity adaptation, and finally reconstruct the HDR infrared image through the dense residual network; Step 3: Save the parameter results obtained in step 2 as a weight file xx.pth, load the weight file into the reconstruction network, and input the LDR infrared image to obtain the HDR reconstructed image.

2. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 1, characterized in that: In the step 1, the HDR reconstructed infrared image data set is produced.

3. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 1, characterized in that: The three feature branch selections and settings in step 2, the improved adaptive grayscale module, and the selection of perceptual loss.

4. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 1, characterized in that: The step 1 includes: A. using a customized high dynamic infrared industrial camera to shoot an HDR infrared image; B. sequentially performing dynamic range limitation, grayscale nonlinear mapping and low-bit quantization on the infrared image.

5. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 1, characterized in that: It also includes dividing the image information to be reconstructed into two parts, high frequency and low frequency, training a convolutional neural network to process the two parts separately, and fusing the two processed image parts.

6. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 5, characterized in that: The method divides the image information to be reconstructed into two parts, high frequency and low frequency: obtaining an infrared image to be processed, acquiring images of a target scene at different angles through a high dynamic infrared industrial camera to obtain an infrared sample image, wherein the infrared sample image includes at least two first infrared images, and reconstructing the at least two first infrared images through a reconstruction network to obtain a second infrared image, wherein the content of the first infrared image is the same as that of the second infrared image, and the resolution of the first infrared image is smaller than that of the second infrared image, and performing feature extraction processing on the first infrared sample image and the second infrared sample image respectively.

7. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 5, characterized in that: The training convolutional neural network processes two parts respectively: initializing the convolutional neural network, making a data set for training the convolutional neural network to obtain a low-frequency information processing model and a high-frequency information processing model, using the obtained high-frequency information processing model to perform super-resolution reconstruction on the high-frequency part, and using the obtained low-frequency information processing model to resample the low-frequency part to obtain an image with an enlarged size.

8. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 5, characterized in that: The two processed image parts are fused: the image obtained by super-resolution reconstruction of the high-frequency part is fused with the image of the low-frequency part with a magnified size to obtain an image reconstruction result.

9. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 1 is applied to an adaptive high dynamic range infrared image reconstruction system, characterized in that: The adaptive high dynamic range infrared image reconstruction system includes: a data acquisition module, a network construction module, a training module and a super-resolution reconstruction module, wherein the data acquisition module is used to acquire target HDR infrared images and produce a data set; the network construction module is used to build a super-resolution network; the training module trains a deep neural network based on the data set; the super-resolution reconstruction module uses the trained parameters to perform infrared video super-resolution reconstruction on the corrected pictures, and combines the reconstructed frame images into a video.

10. The adaptive high dynamic range infrared image reconstruction algorithm according to claim 9, characterized in that: It also includes an acquisition unit, which is used to control the high dynamic infrared industrial camera to acquire images of the target scene at different angles according to the received image acquisition instruction.

Citation Information

Patent Citations

  • Image high dynamic range reconstruction method based on deep learning

    CN111292264A

  • High dynamic range image reconstruction method based on global feature guidance

    CN111709900A

  • High dynamic range image artifact removing method based on attention mechanism

    CN114998138A

  • High dynamic range image generation method, apparatus and device, and readable storage medium

    CN115103118A

Cited By

  • Extreme overexposure image restoration method, system, medium, program product and equipment

    CN119784647A

  • Single-exposure HDR image reconstruction method based on high and low frequency double branches

    CN121190324A