Image non-uniformity correction method and device, infrared image device and storage medium

Abstract

The image non-uniformity correction method and device, the infrared image device and the computer readable storage medium provided by the embodiments of the present application, the method comprises: acquiring a same group of in-focus images and out-of-focus images collected for a target scene; respectively decomposing the in-focus images and the out-of-focus images through wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images; determining a fixed image noise component contained in a high-frequency component in the out-of-focus sub-band images based on the high-frequency component; determining a corresponding image component estimation value according to a sub-band coefficient of the in-focus sub-band images and the corresponding fixed image noise component; reconstructing the image component estimation value through inverse wavelet transform to obtain a non-uniformity corrected image of the target scene, thus avoiding information loss and redundancy of the image signal in the decomposition process, so that the non-uniformity corrected image of the target scene is obtained more quickly and more clearly.

Description

Technical Field

[0001] This application relates to the field of infrared imaging technology, and in particular to an image non-uniformity correction method, apparatus, infrared imaging device, and storage medium. Background Technology

[0002] Infrared focal plane array detectors (IRFPAs) have become the main components of infrared imaging technology, but they suffer from severe non-uniformity, manifesting as fixed pattern noise (FPN) in infrared images. This significantly restricts the application of IRFPAs, necessitating non-uniformity correction (NUC). Correction methods mainly fall into two categories: one is the calibration method, typically represented by the two-point temperature calibration algorithm. The biggest drawback of the calibration method is the need for repeated system calibration; the other is the scene-based method, which uses information from the actual scene to achieve non-uniformity correction, enabling adaptive correction and becoming the main direction of NUC technology development.

[0003] In recent years, numerous scene-based non-uniformity correction techniques have emerged. Generally, these algorithms are implemented through two main approaches: one is statistical, with representative techniques including time-domain high-pass filtering, statistical constant methods, and neural networks; the other is registration-based, with representative techniques including panoramic image accumulation, inter-frame registration, and algebraic correction. Among statistical scene-based correction algorithms, time-domain high-pass filtering is widely used and studied due to its relatively low computational and storage requirements. However, it struggles to balance convergence speed and stability; excessive pursuit of high convergence speed can easily lead to "ghosting." To address the ghosting problem in time-domain high-pass filtering algorithms, a spatial low-pass time-domain high-pass method (SLPF-NUC) was proposed based on the time-domain high-pass filtering method (THPF-NUC). Furthermore, a time-domain high-pass filtering method based on bilateral filtering (BFTH-NUC) was proposed. Bilateral filters can better preserve image edges, effectively reducing ghosting and further improving convergence speed. However, the spatial mean filter used cannot effectively protect the edge components in the image scene, while the bilateral filter can protect the scene edge to a certain extent. However, the residual image of the filtered image still contains some non-negligible scene edge information, which inevitably leads to incorrect updates of the correction parameters and affects the final correction accuracy of the algorithm. Summary of the Invention

[0004] To address the existing technical problems, this application provides a method, apparatus, infrared imaging device, and computer-readable storage medium for image non-uniformity correction that can preserve image details and display clear images.

[0005] To achieve the above objectives, the technical solution of this application embodiment is implemented as follows:

[0006] An image non-uniformity correction method includes the following steps: acquiring a set of in-focus and out-of-focus images of a target scene; decomposing the in-focus and out-of-focus images respectively using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images; determining the fixed image noise component contained in the high-frequency component based on the high-frequency component in the out-of-focus sub-band image; determining the corresponding image component estimate value according to the sub-band coefficient of the in-focus sub-band image and the corresponding fixed image noise component; and reconstructing the image component estimate value by inverse wavelet transform to obtain the non-uniformity corrected image of the target scene.

[0007] Optionally, the step of decomposing the in-focus image and the out-of-focus image using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images, and determining the fixed image noise component contained in the high-frequency component based on the high-frequency component in the out-of-focus sub-band image, includes: performing two-dimensional decomposition of the in-focus image using wavelet transform to obtain four sets of in-focus sub-band images, performing two-dimensional decomposition of the out-of-focus image using wavelet transform to obtain four sets of out-of-focus sub-band images; and calculating the fixed image noise component contained in each of the three sets of out-of-focus sub-band images according to the high-frequency sub-band coefficients of the three sets of out-of-focus sub-band images corresponding to the three directions.

[0008] Optionally, the step of performing two-dimensional decomposition of the in-focus image using wavelet transform to obtain four sets of in-focus sub-band images, and performing two-dimensional decomposition of the out-of-focus image using wavelet transform to obtain four sets of out-of-focus sub-band images, includes: performing one-dimensional wavelet decomposition on the one-dimensional data of each row in the in-focus image using wavelet transform to obtain high-frequency information and low-frequency information; performing one-dimensional wavelet decomposition on the one-dimensional data of each column in the decomposed high-frequency information and low-frequency information to obtain a first set of in-focus sub-band images obtained through row low-pass and column low-pass, a second set of in-focus sub-band images obtained through row low-pass and column high-pass, a third set of in-focus sub-band images obtained through row high-pass and column low-pass, and a fourth set of in-focus sub-band images obtained through row high-pass and column high-pass. The image is a real-focus sub-band image. One-dimensional wavelet decomposition is performed on the one-dimensional data of each row in the defocused image using wavelet transform to obtain high-frequency and low-frequency information. One-dimensional wavelet decomposition is then performed on the one-dimensional data of each column in the decomposed high-frequency and low-frequency information to obtain a first set of defocused sub-band images obtained through row low-pass and column low-pass filters, a second set of defocused sub-band images obtained through row low-pass and column high-pass filters, a third set of defocused sub-band images obtained through row high-pass and column low-pass filters, and a fourth set of defocused sub-band images obtained through row high-pass and column high-pass filters. In the real-focus sub-band image and the defocused sub-band image, the first set of real-focus sub-band images and the first set of defocused sub-band images are low-frequency components, while the others are high-frequency components.

[0009] Optionally, the step of calculating the fixed image noise component contained in each of the three sets of defocus sub-band images based on the high-frequency sub-band coefficients of the three sets of defocus sub-band images corresponding to the three directions respectively includes: based on the high-frequency sub-band coefficients of the second, third, and fourth sets of defocus sub-band images (which are high-frequency components) and the confidence level of the fixed image noise, determining the second fixed image noise component contained in the second set of defocus sub-band images according to the following calculation formula. The third fixed image noise component contained in the third group of defocused sub-band images The fourth fixed image noise component contained in the fourth group of defocused sub-band images

[0010]

[0011]

[0012]

[0013] in, It is the second fixed image noise component contained in the second group of defocused sub-band images corresponding to the first defocused image. It refers to the third fixed image noise component contained in the third group of defocused sub-band images corresponding to the first defocused image. It is the fourth fixed image noise component contained in the fourth group of defocused sub-band images corresponding to the first defocused image. It is the confidence level of the fixed-pattern noise component contained in the high-frequency component. These are the high-frequency sub-band coefficients of the second group of defocused sub-band images, the third group of defocused sub-band images, and the fourth group of defocused sub-band images, respectively. M is the high-pass filter parameter, and j is the wavelet decomposition level.

[0014] Optionally, the step of calculating the fixed image noise component contained in each of the three sets of defocused sub-band images based on the high-frequency sub-band coefficients of the three sets of defocused sub-band images corresponding to the three directions respectively further includes: determining the observed value variance and the true value variance corresponding to the defocused image based on the high-frequency sub-band coefficients of the second, third, and fourth sets of defocused sub-band images, which are high-frequency components; determining the variance of the fixed image noise component contained in the high-frequency sub-band coefficients of the high-frequency components in the defocused sub-band images based on the maximum difference between the observed value variance and the true value variance; and calculating the confidence level of the fixed image noise based on the variance of the fixed image noise component.

[0015] Optionally, determining the corresponding image component estimate based on the sub-band coefficients of the real-focus sub-band image and the corresponding fixed image noise component includes: determining a second image component estimate based on the difference between the high-frequency sub-band coefficients of the second set of real-focus sub-band images and the second fixed image noise component; determining a third image component estimate based on the difference between the high-frequency sub-band coefficients of the third set of real-focus sub-band images and the third fixed image noise component; and determining a fourth image component estimate based on the difference between the high-frequency sub-band coefficients of the fourth set of real-focus sub-band images and the fourth fixed image noise component.

[0016] Optionally, acquiring the same set of in-focus and out-of-focus images for the target scene includes: acquiring infrared video image data for the target scene; sequentially acquiring a sequence of in-focus infrared image frames and a sequence of out-of-focus infrared image frames carrying time information from the infrared video image data; and obtaining multiple sets of in-focus and out-of-focus images of the target scene based on the in-focus and out-of-focus infrared image frame sequences; after reconstructing the estimated values ​​of the image components through inverse wavelet transform to obtain the non-uniformity corrected image of the target scene, the process includes: generating the non-uniformity corrected infrared video image of the target scene based on the non-uniformity corrected images corresponding to the multiple sets of in-focus and out-of-focus images respectively.

[0017] This application embodiment also provides an image non-uniformity correction device, comprising: an acquisition module, configured to acquire a set of in-focus images and out-of-focus images of a target scene; a determination module, configured to decompose the in-focus images and the out-of-focus images respectively by wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images, and determine the fixed image noise component contained in the high-frequency component based on the high-frequency component in the out-of-focus sub-band image; further configured to determine the corresponding image component estimate value according to the sub-band coefficient of the in-focus sub-band image and the corresponding fixed image noise component; and a reconstruction module, configured to reconstruct the image component estimate value by inverse wavelet transform to obtain the non-uniformity corrected image of the target scene.

[0018] This application also provides an infrared imaging device, including a memory, a processor, and an image acquisition module connected to the processor. The memory is used to store a program. The image acquisition module is used to acquire in-focus and out-of-focus images of the same group for a target scene and send them to the processor. The processor is used to implement the image non-uniformity correction method described in any of the above embodiments when executing the program.

[0019] Optionally, the infrared imaging device includes an infrared focal plane array detector in its image acquisition module.

[0020] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image non-uniformity correction method as described in any of the above embodiments.

[0021] The image non-uniformity correction method provided in the above embodiments of this application decomposes the same group of in-focus and out-of-focus images of the target scene using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images. Based on the high-frequency components in the out-of-focus sub-band images, the fixed image noise components contained in the out-of-focus images are determined. Based on the sub-band coefficients of the in-focus sub-band images and the fixed image noise components, the estimated values ​​of the image components are determined. The estimated values ​​of the image components are reconstructed using inverse wavelet transform to obtain the non-uniformity corrected image of the target scene. Thus, compared to the low-pass filters such as mean filtering, bilateral filtering, and guided filtering used in the spatial low-pass time-pass NUC (SLPF-NUC), wavelet transform can more accurately decompose and represent the detailed information in the image, and provides selective images that conform to the direction of the human visual system, which is beneficial for quickly extracting structural and detailed information from the original image. Compared to the situation where some target edges and detailed information remain after separation of in-focus images, the high-frequency information such as non-uniformity and noise contained in defocus images is easier to separate, with almost no residual target edge details. Therefore, the embodiments of this application use the high-frequency components of the defocus image to obtain the FPN (Fixed Pattern Noise) component contained in the defocus image, so that the infrared image device can more accurately determine the FPN components present when acquiring the target scene, effectively protecting the edge components of the target scene and reducing the occurrence of ghosting. The use of wavelet transform decomposition and inverse transform reconstruction gives the decomposed image a complete reconstruction capability, avoiding information loss and redundancy of the image signal during the decomposition process, thereby obtaining the non-uniformity corrected image of the target scene more quickly and clearly.

[0022] The image non-uniformity correction device, infrared imaging device, and computer-readable storage medium described in the embodiments of this application have the same specific technical features as the image non-uniformity correction method and have the same beneficial technical effects as the image non-uniformity correction method, and will not be repeated here. Attached Figure Description

[0023] Figure 1 This is a schematic flowchart of an image non-uniformity correction method in one embodiment of this application;

[0024] Figure 2 This is a schematic diagram illustrating the specific process of an image non-uniformity correction method in another embodiment of this application;

[0025] Figure 3This is a real-focus image of a target scene acquired in one embodiment of this application;

[0026] Figure 4 This is an off-focus image captured for a target scene in one embodiment of this application;

[0027] Figure 5 This is an image showing the effect obtained by using an image non-uniformity correction method in one embodiment of this application.

[0028] Figure 6 This is a display effect diagram of non-uniformity correction performed on an out-of-focus image in one embodiment of this application;

[0029] Figure 7 This is a display effect diagram of non-uniformity correction without introducing a fixed image noise component confidence level in one embodiment of this application;

[0030] Figure 8 This is a schematic diagram of the structure of an image non-uniformity correction device in one embodiment of this application;

[0031] Figure 9 This is a schematic diagram of the structure of an infrared imaging device in one embodiment of this application. Detailed Implementation

[0032] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to limit the ways in which this application may be implemented. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0034] In the description of this application, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0035] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0036] The inventors of this application discovered in their research that the temporal high-pass filtering algorithm (THPF) is a statistical filter in the time domain. Its basic idea is to assume that non-uniformity exists within background clutter and changes relatively slowly over time in the image, while the target signal of interest to the human eye moves relatively quickly relative to the background clutter on the image plane. Therefore, a temporal low-pass filtering method can decompose the noisy image into two parts in the time domain: the target signal (high-frequency component) and the background clutter (low-frequency component). Since the fundamental reason affecting the convergence speed of the THPF algorithm and its ghosting problem is that a large amount of irrelevant scene information is involved in the calculation of the non-uniformity correction parameters, by excluding as much scene information, especially strong objects, as possible from the original image and then including the remaining part in the calculation of the non-uniformity correction parameters, the influence of non-random motion and strong objects in the scene on the correction process can be minimized, effectively reducing the ghosting effect. Based on this idea, the embodiments of this application use wavelet transform to separate the high and low frequencies of the input image signal.

[0037] Please see Figure 1 An image non-uniformity correction method provided in this application includes the following steps:

[0038] S11: Acquire in-focus and out-of-focus images of the same group for the target scene.

[0039] Here, the target scene refers to an objective scene that the user expects to capture and that can be used to meet relevant needs. The in-focus image refers to an infrared image of the target scene captured when the infrared imaging device is in a sharp state. The out-of-focus image refers to an infrared image of the target scene captured when the infrared imaging device is in an out-of-focus state. The same set of in-focus and out-of-focus images refers to images captured simultaneously at the same time, one in sharp state and the other in out-of-focus state, representing the same frame. Acquiring the same set of in-focus and out-of-focus images for the target scene can mean that the infrared imaging device captures the same set of in-focus and out-of-focus images for the target scene. The infrared imaging device can be simultaneously adjusted to a sharp state and an out-of-focus state under the control of a motor control device to capture in-focus and out-of-focus images of the same frame.

[0040] S12: Decompose the in-focus image and the out-of-focus image respectively by wavelet transform to obtain the corresponding in-focus sub-band image and out-of-focus sub-band image. Based on the high-frequency components in the out-of-focus sub-band image, determine the fixed image noise component contained in the high-frequency components.

[0041] Here, the wavelet transform is simply replacing the trigonometric functions of the Fourier transform with wavelet functions. When decomposing an image, the wavelet transform includes the positional information of each sub-band image, and the accuracy of the image decomposition is related to the scale and direction of the decomposition. Generally, the more layers and directions of wavelet decomposition, the more accurate the image decomposition. Therefore, to obtain a more refined image decomposition, the number of sub-band images obtained will be greater. A sub-band image refers to a component image divided into several frequency-limited sub-bands from an original image. A focused sub-band image refers to a focused image divided into several frequency-limited focused component images. A defocused sub-band image refers to a defocused image divided into several frequency-limited defocused component images. High-frequency components in a defocused sub-band image refer to sub-band images containing high-frequency components, such as high-frequency components corresponding to the horizontal direction or the vertical direction. The fixed pattern noise (FPN) component in the defocused image refers to the factors that cause non-uniformity issues in the defocused image, such as vertical stripes and motion blur. After wavelet decomposition, the contour information of the defocused image mainly exists in the low-frequency sub-band coefficients; while the decomposed high-frequency sub-band contains the detailed features of the image, such as object contours and texture information. Due to its unique high-frequency and directional nature, the striped fixed pattern noise component in the infrared image mainly exists in the high-frequency sub-band coefficients after wavelet transform decomposition. Therefore, during the temporal filtering process of each defocused sub-band image, only a portion of the high-frequency coefficients containing the fixed pattern noise component can be used to estimate the correction parameters. In this way, other detailed components of the image can be preserved as much as possible, and the problems of image blurring in static scenes and motion blur in moving images will be solved. Therefore, the step of decomposing the in-focus image and the out-of-focus image using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images, and determining the fixed image noise component contained in the high-frequency component based on the high-frequency component in the out-of-focus sub-band image, can refer to: the infrared imaging device decomposing the in-focus image to obtain the corresponding in-focus sub-band image and decomposing the out-of-focus image to obtain the corresponding out-of-focus sub-band image, and determining the fixed image noise component contained in the high-frequency component based on the high-frequency component in the out-of-focus sub-band image.

[0042] S13: Determine the corresponding image component estimate based on the sub-band coefficient of the real-focus sub-band image and the corresponding fixed image noise component.

[0043] Here, the sub-band coefficients of the focused sub-band image refer to the wavelet coefficients generated by convolving the focused sub-band image in two directions using low-pass filters and / or high-pass filters. The image component estimates are the estimates of each sub-band image in the real image group that is in the same category as the focused image and the out-of-focus image, and are determined based on the calculation relationship between the sub-band coefficients of the focused sub-band image and the fixed image noise components contained in the out-of-focus image. The fixed image noise component corresponding to the sub-band coefficients of the focused sub-band image refers to the fixed image noise component contained in the high-frequency component in the out-of-focus image that is at the same position as the focused sub-band image. Determining the image component estimates based on the sub-band coefficients of the focused sub-band image and the fixed image noise components can be achieved by the infrared imaging device determining the estimated values ​​of the corresponding sub-band images of the real image based on the sub-band coefficients of the focused sub-band image and the fixed image noise components contained in the high-frequency component in the out-of-focus image that is at the same position as the focused sub-band image.

[0044] S14: The estimated values ​​of the image components are reconstructed by inverse wavelet transform to obtain the non-uniformity corrected image of the target scene.

[0045] Here, the inverse wavelet transform is the inverse function of the wavelet transform. Reconstructing the image component estimates refers to combining the image components estimated based on the sub-band coefficients of the focused sub-band image. The non-uniformity-corrected image refers to a real image of the target scene, grouped with the focused and out-of-focus images, after removing fixed image noise components such as image blur, motion blur, and stripes from the focused image. Reconstructing the image component estimates using inverse wavelet transform to obtain the non-uniformity-corrected image of the target scene can refer to the infrared imaging device combining the image component estimates corresponding to the sub-band coefficients of the focused sub-band image using inverse wavelet transform to obtain a real image of the target scene. The specific formula is as follows:

[0046]

[0047] Where j is the wavelet decomposition level, G is the template operation of the high-pass filter, H is the template operation of the low-pass filter, and G' and H' are the conjugate inversions of G and H, respectively. The estimated value for the image components, This is the defocused image of the previous layer.

[0048] The image non-uniformity correction method provided in the above embodiments of this application decomposes the same group of in-focus and out-of-focus images of the target scene using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images. Based on the high-frequency components in the out-of-focus sub-band images, the fixed image noise components contained in the high-frequency components are determined. According to the sub-band coefficients of the in-focus sub-band images and the corresponding fixed image noise components, the corresponding image component estimates are determined. The image component estimates are reconstructed using inverse wavelet transform to obtain the non-uniformity corrected image of the target scene. Thus, compared to the low-pass filters such as mean filtering, bilateral filtering, and guided filtering used in the Spatial Low-Pass Time-High-Pass (SLPF-NUC) method, wavelet transform can more accurately decompose and represent the detailed information in the image, and provides selective images that conform to the direction of the human visual system, which is beneficial for quickly extracting structural and detailed information from the original image. Compared to the situation where some target edges and detailed information remain after separation of in-focus images, the high-frequency information such as non-uniformity and noise contained in defocus images is easier to separate, with almost no residual target edge details. Therefore, the embodiments of this application utilize the FPN components contained in the high-frequency components of defocus images, so that the determination of the FPN components present when the infrared imaging device acquires the target scene is more accurate, which can effectively protect the edge components of the target scene and reduce the occurrence of ghosting. The use of wavelet transform decomposition and inverse transform reconstruction gives the decomposed image a complete reconstruction capability, avoids information loss and redundancy in the image signal during the decomposition process, and thus obtains the non-uniformity corrected image of the target scene more quickly and clearly.

[0049] In some embodiments, the step of decomposing the in-focus image and the out-of-focus image using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images, and determining the fixed image noise component contained in the high-frequency component based on the high-frequency component in the out-of-focus sub-band image, includes: performing two-dimensional decomposition of the in-focus image using wavelet transform to obtain four sets of in-focus sub-band images, performing two-dimensional decomposition of the out-of-focus image using wavelet transform to obtain four sets of out-of-focus sub-band images; and calculating the fixed image noise component contained in each of the three sets of out-of-focus sub-band images according to the high-frequency sub-band coefficients of the three sets of out-of-focus sub-band images corresponding to the three directions.

[0050] Here, the two-dimensional decomposition refers to filtering the image in the horizontal and vertical directions to achieve multi-resolution decomposition of the image sequence. The four sets of in-focus sub-band images refer to four sets of sub-band images obtained in four directions after filtering the in-focus image in both the horizontal and vertical directions. The four sets of out-of-focus sub-band images refer to four sets of sub-band images obtained in four directions after filtering the out-of-focus image in both the horizontal and vertical directions. The two-dimensional decomposition of the in-focus image using wavelet transform to obtain four sets of in-focus sub-band images, and vice versa, can refer to the infrared imaging device performing two-dimensional decomposition of the in-focus image using wavelet transform to obtain four sets of in-focus sub-band images in four directions, and performing two-dimensional decomposition of the out-of-focus image using wavelet transform to obtain four sets of out-of-focus sub-band images in four directions. The three sets of out-of-focus sub-band images corresponding to the three directions refer to three high-frequency components of the four sets of in-focus sub-band images. The step of calculating the fixed image noise component contained in each of the three sets of defocused sub-band images based on the high-frequency sub-band coefficients of the three sets of defocused sub-band images corresponding to the three directions can refer to: the infrared imaging device selecting the high-frequency sub-band coefficients of the three sets of high-frequency components corresponding to the defocused image in the three directions, and calculating the fixed image noise component contained in each set of high-frequency components respectively.

[0051] The non-uniformity image correction method described in this application uses two-dimensional wavelets to decompose the in-focus image and the out-of-focus image respectively. This allows for more accurate decomposition of the in-focus image and the out-of-focus image to represent the detailed information in the in-focus image and the out-of-focus image. The decomposed image has a complete reconstruction capability, avoiding information loss and redundancy in the image signal during the decomposition process, so as to obtain the non-uniformity corrected image of the target scene more quickly and clearly.

[0052] In some embodiments, the step of performing two-dimensional decomposition of the in-focus image using wavelet transform to obtain four sets of in-focus sub-band images, and performing two-dimensional decomposition of the out-of-focus image using wavelet transform to obtain four sets of out-of-focus sub-band images, includes: performing one-dimensional wavelet decomposition on the one-dimensional data of each row in the in-focus image using wavelet transform to obtain high-frequency information and low-frequency information; performing one-dimensional wavelet decomposition on the one-dimensional data of each column in the decomposed high-frequency information and low-frequency information to obtain a first set of in-focus sub-band images obtained through row low-pass and column low-pass, a second set of in-focus sub-band images obtained through row low-pass and column high-pass, a third set of in-focus sub-band images obtained through row high-pass and column low-pass, and a fourth set of in-focus sub-band images obtained through row high-pass and column high-pass. Four sets of in-focus sub-band images; one-dimensional wavelet decomposition is performed on the one-dimensional data of each row in the defocused image using wavelet transform to obtain high-frequency and low-frequency information. One-dimensional wavelet decomposition is then performed on the one-dimensional data of each column in the decomposed high-frequency and low-frequency information to obtain a first set of defocused sub-band images obtained through row low-pass and column low-pass, a second set of defocused sub-band images obtained through row low-pass and column high-pass, a third set of defocused sub-band images obtained through row high-pass and column low-pass, and a fourth set of defocused sub-band images obtained through row high-pass and column high-pass. Among the in-focus and defocused sub-band images, the first set of in-focus and defocused sub-band images are low-frequency components, while the others are high-frequency components.

[0053] Here, the one-dimensional wavelet decomposition of each row of the real-focus image is performed by wavelet transform to obtain high-frequency and low-frequency information. The one-dimensional data of each column of the high-frequency and low-frequency information obtained by the decomposition is then subjected to one-dimensional wavelet decomposition to obtain a first set of real-focus sub-band images obtained by row low-pass and column low-pass, a second set of real-focus sub-band images obtained by row low-pass and column high-pass, a third set of real-focus sub-band images obtained by row high-pass and column low-pass, and a fourth set of real-focus sub-band images obtained by row high-pass and column high-pass. One-dimensional wavelet decomposition is performed on the one-dimensional data of each row in the defocused image using wavelet transform to obtain high-frequency and low-frequency information. Then, one-dimensional wavelet decomposition is performed on the one-dimensional data of each column in the decomposed high-frequency and low-frequency information to obtain a first set of defocused sub-band images obtained through row low-pass and column low-pass filters, a second set of defocused sub-band images obtained through row low-pass and column high-pass filters, a third set of defocused sub-band images obtained through row high-pass and column low-pass filters, and a fourth set of defocused sub-band images obtained through row high-pass and column high-pass filters. The low-frequency sub-band coefficients of the first group of real-focus sub-band images, These are the three high-frequency sub-band coefficients of the second group of real-focus sub-band images, the third group of real-focus sub-band images, and the fourth group of real-focus sub-band images, respectively. The low-frequency sub-band coefficients of the first group of defocused sub-band images, The high-frequency sub-band coefficients of the second, third, and fourth sets of defocused sub-band images are expressed as follows:

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] in, The low-frequency sub-band coefficients of the first group of real-focus sub-band images of the previous real-focus image. These are the three high-frequency sub-band coefficients of the second, third, and fourth sets of real-focus sub-band images from the previous layer of real-focus image. The low-frequency sub-band coefficients of the first group of defocused sub-band images in the previous defocused image layer. These are the three high-frequency sub-band coefficients of the second, third, and fourth sets of defocused sub-band images of the previous layer's defocused image.

[0063] The non-uniformity image correction method described in this application performs one-dimensional wavelet decomposition on the one-dimensional data of each row in the in-focus and out-of-focus images using wavelet transform. Then, it performs one-dimensional wavelet decomposition on the one-dimensional data of each column in the decomposed high-frequency and low-frequency information to obtain three sets of in-focus sub-band images containing high-frequency components, one set of in-focus sub-band images containing low-frequency components, three sets of out-of-focus sub-band images containing high-frequency components, and one set of out-of-focus sub-band images containing low-frequency components. This facilitates the subsequent determination of the fixed noise components contained in the out-of-focus image. The decomposed image possesses excellent reconstruction capabilities, avoiding information loss and redundancy in the image signal during the decomposition process, thus enabling a faster and clearer acquisition of the non-uniformity corrected image of the target scene.

[0064] In some embodiments, the step of calculating the fixed image noise component contained in each of the three sets of defocus sub-band images based on the high-frequency sub-band coefficients of the three sets of defocus sub-band images corresponding to the three directions respectively includes: based on the high-frequency sub-band coefficients of the second, third, and fourth sets of defocus sub-band images (which are high-frequency components) and the confidence level of the fixed image noise, determining the second fixed image noise component contained in the second set of defocus sub-band images according to the following calculation formula. The third fixed image noise component contained in the third group of defocused sub-band images The fourth fixed image noise component contained in the fourth group of defocused sub-band images

[0065]

[0066]

[0067]

[0068] in, It is the second fixed image noise component contained in the second group of defocused sub-band images corresponding to the first defocused image. It refers to the third fixed image noise component contained in the third group of defocused sub-band images corresponding to the first defocused image. It is the fourth fixed image noise component contained in the fourth group of defocused sub-band images corresponding to the first defocused image. It is the confidence level of the fixed-pattern noise component contained in the high-frequency component. These are the high-frequency sub-band coefficients of the second group of defocused sub-band images, the third group of defocused sub-band images, and the fourth group of defocused sub-band images, respectively. M is the high-pass filter parameter, and j is the wavelet decomposition level.

[0069] After performing two-dimensional wavelet decomposition on the defocused image, the selection of which image subbands to estimate the FPN has a significant impact on the subsequent reconstruction of the non-uniformity-corrected image. For example, after wavelet decomposition, the subband components corresponding to a striped FPN should mainly exist in the vertical subbands, but this does not rule out the possibility of the FPN existing in other subbands. Therefore, by introducing FPN confidence... This parameter describes the extent to which each high-frequency component in the out-of-focus image contains an FPN. The preceding out-of-focus image refers to the previous frame of the current out-of-focus image in the time domain.

[0070] The non-uniform image correction method described in this application introduces FPN confidence level. This parameter describes the degree of FPN contained in different out-of-focus images, so as to accurately determine the fixed noise components contained in the out-of-focus images and to perform reasonable and effective image attenuation processing on the three sets of real-focus sub-band images containing high-frequency components in the real-focus images.

[0071] In some embodiments, the step of calculating the fixed image noise component contained in each of the three sets of defocus sub-band images based on the high-frequency sub-band coefficients of the three sets of defocus sub-band images corresponding to the three directions respectively further includes: determining the observed value variance and the true value variance corresponding to the defocus image based on the high-frequency sub-band coefficients of the second set of defocus sub-band images, the third set of defocus sub-band images, and the fourth set of defocus sub-band images, which are high-frequency components; determining the variance of the fixed image noise component contained in the high-frequency sub-band coefficients of the high-frequency components in the defocus sub-band images based on the maximum difference between the observed value variance and the true value variance; and calculating the confidence level of the fixed image noise based on the variance of the fixed image noise component.

[0072] Here, the high-frequency sub-band coefficients of the second group of defocused sub-band images, the third group of defocused sub-band images, and the fourth group of defocused sub-band images, which are based on high-frequency components in the defocused sub-band images, are... Determine the variance of the observed values ​​corresponding to the defocused image. and the variance of the true value Specifically, this could be: the variance of the observed values. It can be obtained directly by calculating the variance of the high-frequency subband coefficients of the defocused subband image, as shown in the following formula:

[0073]

[0074] Meanwhile, a general assumption is introduced: for any target scene image, the contained image scene has multi-directional characteristics. Thus, the variances of the high-frequency sub-band coefficients of each sub-band image after wavelet decomposition should be relatively similar. Therefore, we can estimate the true variance of the defocused image by averaging the high-frequency sub-band coefficients from different directions within the same scale. The formula is as follows:

[0075]

[0076] Then, based on the variance of the observed values... and the variance of the true value The relationships between them are as follows:

[0077]

[0078] The variance of the fixed image noise component contained in the high-frequency sub-band coefficients of the defocused sub-band image is determined as follows:

[0079]

[0080] Where, δ 2 For variance operators, The variance of the fixed image noise component is used. In the formula for calculating the variance of the fixed image noise component corresponding to the defocused image, the operation of taking the maximum value is to avoid negative numbers that would affect the subsequent estimation of the FPN component, thus preventing the filtering out of image details. The variance of the fixed image noise component represents the degree of FPN contained in the sub-band image, and it has the same trend as the FPN confidence. Therefore, the variance of the estimated image component can be used as a parameter to adjust the FPN confidence, thereby achieving the purpose of adaptively adjusting the filtering degree. Finally, based on the variance of the fixed image noise component, the confidence of the fixed image noise is calculated, that is, by introducing an activation function (Sigmoid function) to normalize the value to the range of [0,1]. The normalization formula is as follows:

[0081]

[0082] Where ε is the shape parameter of the Sigmoid function, the larger the ε is, the steeper the shape of the Sigmoid function; conversely, the smaller the ε is, the flatter the shape.

[0083] Thus, for off-focus images with more FPN components, It should be closer to 1. This allows the time-domain high-frequency filter to more completely remove the FPN components. And for defocused images containing fewer FPN components, It should be closer to 0. This maximizes the preservation of detail in the image. In short, different defocused images have different levels of filtering depending on the degree to which they contain FPN components. Therefore, the estimation of image components will be more accurate, and even if there is some error in a certain sub-band, the impact on the non-uniformity corrected image of the target scene will be smaller, thus greatly solving the image motion blur problem.

[0084] The non-uniformity image correction method described in this application first determines the observed variance and true variance of the defocused image using the high-frequency subband coefficients of three sets of defocused subband images containing high-frequency components. Then, it determines the variance of the fixed image noise component included in the high-frequency subband coefficients of the defocused subband image. Finally, based on the variance of the fixed image noise component, it calculates the confidence level of the fixed image noise. Thus, by determining the corresponding confidence level of the fixed image noise according to the different degrees of FPN components contained in the defocused image, it is easier to accurately determine the fixed noise component contained in the defocused image and to perform reasonable and effective image attenuation processing on the in-focus image. Therefore, the estimation of the image component estimates will be more accurate.

[0085] In some embodiments, determining the image component estimate based on the sub-band coefficients of the real-focus sub-band image and the fixed image noise component includes: determining a second image component estimate based on the difference between the high-frequency sub-band coefficients of the second set of real-focus sub-band images and the second fixed image noise component; determining a third image component estimate based on the difference between the high-frequency sub-band coefficients of the third set of real-focus sub-band images and the third fixed image noise component; and determining a fourth image component estimate based on the difference between the high-frequency sub-band coefficients of the fourth set of real-focus sub-band images and the fourth fixed image noise component.

[0086] Here, determining the estimated value of the second image component based on the difference between the high-frequency subband coefficient of the second set of focused subband images and the noise component of the second fixed image can refer to the infrared imaging device determining the estimated value of the second image component based on the difference between the high-frequency subband coefficient of the second set of focused subband images and the noise component of the second fixed image; determining the estimated value of the third image component based on the difference between the high-frequency subband coefficient of the third set of focused subband images and the noise component of the third fixed image can refer to the infrared imaging device determining the estimated value of the third image component based on the difference between the high-frequency subband coefficient of the third set of focused subband images and the noise component of the third fixed image; determining the estimated value of the fourth image component based on the difference between the high-frequency subband coefficient of the fourth set of focused subband images and the noise component of the fourth fixed image can refer to the infrared imaging device determining the estimated value of the fourth image component based on the difference between the high-frequency subband coefficient of the fourth set of focused subband images and the noise component of the fourth fixed image. The specific formulas are as follows:

[0087]

[0088]

[0089]

[0090] in, This represents the estimated value of the second image component, i.e., the horizontal high-frequency subband coefficients of the j-th layer after removing non-uniformities. This is the estimate of the third image component, i.e., the vertical high-frequency subband coefficients of the j-th layer after removing non-uniformities. This is the estimated value of the fourth image component, i.e., the diagonal high-frequency subband coefficients of the j-th layer after removing non-uniformity. Meanwhile, since the first set of real-focus subband images in the real-focus image consists of low-frequency components and does not contain FPN components, high-pass filtering is not required for the first set of real-focus subband images to avoid filtering out the image's contour information. in, This is the estimated value of the first image component, i.e., the low-frequency subband coefficient of the j-th layer.

[0091] The non-uniformity image correction method described in this application uses the high-frequency subband coefficients of the in-focus image to subtract the FPN components contained in the corresponding high-frequency components of the out-of-focus image, thereby obtaining multiple corresponding image component estimates. This allows for reasonable and effective image attenuation processing of the high-frequency components in the in-focus image, which is beneficial for improving the sharpness of the image after non-uniformity correction.

[0092] In some embodiments, acquiring a set of in-focus and out-of-focus images of the same target scene includes: acquiring infrared video image data of the target scene; sequentially acquiring a sequence of in-focus infrared image frames and a sequence of out-of-focus infrared image frames carrying time information from the infrared video image data; and obtaining multiple sets of in-focus and out-of-focus images of the target scene based on the sequence of in-focus infrared image frames and the sequence of out-of-focus infrared image frames.

[0093] After reconstructing the estimated values ​​of the image components through inverse wavelet transform to obtain the non-uniformity corrected image of the target scene, the process includes: generating a non-uniformity corrected infrared video image of the target scene based on the non-uniformity corrected images corresponding to multiple sets of the in-focus and out-of-focus images.

[0094] Here, the infrared video image data refers to infrared video data collected for a target scene, and the video contains a continuously collected sequence of image frames for the target scene. The in-focus infrared image frame sequence refers to the sequence of infrared image frames in the infrared video image data for the target scene collected when the infrared imaging device is in a clear state. The out-of-focus infrared image frame sequence refers to the sequence of infrared image frames in the infrared video image data for the target scene collected when the infrared imaging device is in an out-of-focus state. The process of acquiring the infrared video image data for the target scene; sequentially acquiring the in-focus and out-of-focus infrared image frame sequences carrying time information from the infrared video image data; and obtaining multiple sets of in-focus and out-of-focus images of the target scene based on the in-focus and out-of-focus infrared image frame sequences can refer to: the infrared imaging device acquiring the infrared video image data for the target scene; sequentially acquiring the in-focus and out-of-focus infrared image frame sequences carrying time information from the infrared video image data; and obtaining multiple sets of in-focus and out-of-focus images of the target scene based on the in-focus and out-of-focus infrared image frame sequences. In this context, the set of in-focus images and out-of-focus images correspond to the same time information. Each set of in-focus images and each set of out-of-focus images corresponds to a set of non-uniformity-corrected images. Generating a non-uniformity-corrected infrared video image of the target scene based on multiple sets of non-uniformity-corrected images corresponding to the in-focus and out-of-focus images can mean that the infrared imaging device generates the non-uniformity-corrected infrared video image of the target scene based on multiple sets of non-uniformity-corrected images corresponding to the in-focus and out-of-focus images.

[0095] The non-uniformity image correction method described in this application further determines the non-uniformity-corrected image corresponding to each set of in-focus and out-of-focus images at each time information from infrared video image data acquired for a target scene. Based on multiple sets of non-uniformity-corrected images corresponding to multiple time information, a non-uniformity-corrected infrared video image of the target scene is generated. This describes a feasible scheme for performing non-uniformity correction on infrared video image data acquired for a target scene to generate a non-uniformity-corrected infrared video image of the target scene, thereby obtaining the non-uniformity-corrected infrared video image of the target scene more quickly and clearly.

[0096] For a comprehensive description of the embodiments of this application, please refer to [link / reference needed]. Figure 2 This is a schematic flowchart illustrating the image non-uniformity correction method according to another embodiment of this application. The image non-uniformity correction method includes the following steps:

[0097] S21: The infrared imaging device is modulated into a clear state to acquire a sequence of in-focus images of the target scene under the control of the motor controller, and is also modulated into a defocus state to acquire a sequence of out-of-focus images of the target scene.

[0098] S22: Initialize the high-pass filter parameter M and wavelet transform level j of the time-domain high-pass filter for infrared imaging equipment;

[0099] S23: The infrared imaging device acquires the focused image and the out-of-focus image of the j-th layer in the same group from the focused image sequence and the out-of-focus image sequence;

[0100] S24: The infrared imaging device performs a two-dimensional wavelet transform on the in-focus image to obtain three corresponding high-frequency sub-band coefficients and one low-frequency sub-band coefficient, and performs a two-dimensional wavelet transform on the out-of-focus image to obtain three corresponding high-frequency sub-band coefficients and one low-frequency sub-band coefficient.

[0101] S25: The infrared imaging device determines the confidence level of the FPN of the high-frequency component in the defocused image based on the three high-frequency sub-band coefficients of the defocused image;

[0102] S26: The infrared imaging device determines the FPN components contained in the three high-frequency components of the defocused image based on the three high-frequency subband coefficients of the defocused image, the confidence level of the FPN, and the high-pass filter parameter M.

[0103] S27: The infrared imaging device determines the estimated value of the corresponding high-frequency image component based on the difference between each high-frequency component of the focused image and the FPN component contained in the corresponding high-frequency component in the defocused image, and determines the estimated value of the corresponding low-frequency image component based on a low-frequency sub-band coefficient of the focused image.

[0104] S28: The infrared imaging device uses inverse wavelet transform to reconstruct the three high-frequency image component estimates and one low-frequency image component estimate to obtain the non-uniformity corrected image of the target scene.

[0105] To gain a more intuitive understanding of the image non-uniformity correction method described in the embodiments of this application, please refer to... Figure 3-5 ,in, Figure 3 This is a real-focus image of a target scene acquired in one embodiment of this application; Figure 4 This is an out-of-focus image in the same group as the in-focus image, acquired for a target scene in one embodiment of this application; Figure 5 This image shows the result of an image non-uniformity correction method used in one embodiment of this application. To further illustrate the beneficial effect of using a defocused image to determine the corresponding FPN in order to remove non-uniformity from the same group of in-focus images, please refer to [link to relevant documentation]. Figure 6The image shown is a display effect diagram of non-uniformity correction without capturing out-of-focus images in one embodiment of this application. To further illustrate the beneficial effect of using out-of-focus images to determine the confidence level of the corresponding FPN in this embodiment of the application to remove non-uniformity from the same group of in-focus images, please refer to [link to relevant documentation]. Figure 7 The image shows the display effect of non-uniformity correction without introducing FPN confidence level in one embodiment of this application. A comparison shows that... Figure 7 The display effect of the captured out-of-focus image is significantly better than... Figure 6 The display effect of the out-of-focus image, i.e. Figure 6 The rendered image will retain obvious scene images, exhibiting "ghosting"; while Figure 5 The display effect of acquiring out-of-focus images and further determining the confidence level of the corresponding FPN is significantly better than that of... Figure 7 The display effect of acquiring out-of-focus images but not determining the confidence level of the corresponding FPN, i.e. Figure 7 In the resulting image, FPN information at some frequencies was not completely removed, resulting in residual vertical lines.

[0106] Therefore, combining Figure 5 As shown in the effect diagram, the image non-uniformity correction method provided in the above embodiments of this application decomposes the same group of in-focus and out-of-focus images of the target scene using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images. Based on the high-frequency components in the out-of-focus sub-band images, the fixed image noise components contained in the out-of-focus images are determined. Based on the sub-band coefficients of the in-focus sub-band images and the fixed image noise components, the estimated values ​​of the image components are determined. The estimated values ​​of the image components are reconstructed using inverse wavelet transform to obtain the non-uniformity corrected image of the target scene. Thus, compared to the low-pass filters such as mean filtering, bilateral filtering, and guided filtering used in the Spatial Low-Pass Time-High-Pass (SLPF-NUC) method, wavelet transform can more accurately decompose and represent the detailed information in the image, and provides selective images that conform to the direction of the human visual system, which is beneficial for quickly extracting structural and detailed information from the original image. Compared to the situation where some target edges and detailed information remain after separation of in-focus images, the high-frequency information such as non-uniformity and noise contained in defocus images is easier to separate, with almost no residual target edge details. Therefore, the embodiments of this application use the high-frequency components of the defocus image to obtain the FPN components contained in the defocus image, making the determination of the FPN components present when the infrared image device acquires the target scene more accurate, effectively protecting the edge components of the target scene and reducing the occurrence of ghosting. The use of wavelet transform decomposition and inverse transform reconstruction gives the decomposed image a complete reconstruction capability, avoiding information loss and redundancy of the image signal during the decomposition process, thereby obtaining the non-uniformity corrected image of the target scene more quickly and clearly.

[0107] This application also provides an image non-uniformity correction device. Please refer to [link to relevant documentation]. Figure 8 The image non-uniformity correction device includes: an acquisition module 81, used to acquire a set of in-focus and out-of-focus images of the same group for a target scene; a determination module 82, used to decompose the in-focus and out-of-focus images respectively by wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images, and to determine the fixed image noise component contained in the out-of-focus image based on the high-frequency components in the out-of-focus sub-band image; and to determine the image component estimate value according to the sub-band coefficient of the in-focus sub-band image and the fixed image noise component; and a reconstruction module 83, used to reconstruct the image component estimate value by inverse wavelet transform to obtain the non-uniformity corrected image of the target scene.

[0108] Optionally, the determining module 82 is further configured to perform two-dimensional decomposition of the real-focus image by wavelet transform to obtain four sets of real-focus sub-band images, and perform two-dimensional decomposition of the defocus image by wavelet transform to obtain four sets of defocus sub-band images; and calculate the fixed image noise component contained in each set of defocus sub-band images in the three sets of defocus sub-band images according to the high-frequency sub-band coefficients of the three sets of defocus sub-band images corresponding to the three directions respectively.

[0109] Optionally, the determining module 82 is further configured to perform one-dimensional wavelet decomposition on the one-dimensional data of each row in the in-focus image using wavelet transform to obtain high-frequency information and low-frequency information; perform one-dimensional wavelet decomposition on the one-dimensional data of each column in the decomposed high-frequency and low-frequency information to obtain a first set of in-focus sub-band images obtained through row low-pass and column low-pass, a second set of in-focus sub-band images obtained through row low-pass and column high-pass, a third set of in-focus sub-band images obtained through row high-pass and column low-pass, and a fourth set of in-focus sub-band images obtained through row high-pass and column high-pass; and perform one-dimensional wavelet decomposition on the one-dimensional data of each row in the out-of-focus image using wavelet transform. The data undergoes one-dimensional wavelet decomposition to obtain high-frequency and low-frequency information. One-dimensional wavelet decomposition is then performed on each column of the obtained high-frequency and low-frequency information to obtain a first set of out-of-focus sub-band images obtained through row low-pass and column low-pass filters, a second set of out-of-focus sub-band images obtained through row low-pass and column high-pass filters, a third set of out-of-focus sub-band images obtained through row high-pass and column low-pass filters, and a fourth set of out-of-focus sub-band images obtained through row high-pass and column high-pass filters. In the in-focus sub-band images and the out-of-focus sub-band images, the first set of in-focus sub-band images and the first set of out-of-focus sub-band images are low-frequency components, while the others are high-frequency components.

[0110] Optionally, the determining module 82 is further configured to determine the second fixed image noise component contained in the second group of defocused sub-band images, the third group of defocused sub-band images, and the fourth group of defocused sub-band images, which are high-frequency components in the defocused sub-band images, and the confidence level of the fixed image noise, respectively, according to the following calculation formula. The third fixed image noise component contained in the third group of defocused sub-band images The fourth fixed image noise component contained in the fourth group of defocused sub-band images

[0111]

[0112]

[0113]

[0114] in, It is the second fixed image noise component contained in the second group of defocused sub-band images corresponding to the first defocused image. It refers to the third fixed image noise component contained in the third group of defocused sub-band images corresponding to the first defocused image. It is the fourth fixed image noise component contained in the fourth group of defocused sub-band images corresponding to the first defocused image. It is the confidence level of the fixed-pattern noise component contained in the high-frequency component. These are the high-frequency sub-band coefficients of the second group of defocused sub-band images, the third group of defocused sub-band images, and the fourth group of defocused sub-band images, respectively. M is the high-pass filter parameter, and j is the wavelet decomposition level.

[0115] Optionally, the determining module 82 is further configured to: determine the observed value variance and the true value variance corresponding to the defocused image based on the high-frequency sub-band coefficients of the second group of defocused sub-band images, the third group of defocused sub-band images, and the fourth group of defocused sub-band images that are high-frequency components in the defocused sub-band images; determine the variance of the fixed image noise component included in the high-frequency sub-band coefficients of the high-frequency components in the defocused sub-band images based on the maximum difference between the observed value variance and the true value variance; and calculate the confidence level of the fixed image noise based on the variance of the fixed image noise component.

[0116] Optionally, the determining module 82 is further configured to determine a second image component estimate based on the difference between the high-frequency subband coefficient of the second set of real-focus subband images and the noise component of the second fixed image; determine a third image component estimate based on the difference between the high-frequency subband coefficient of the third set of real-focus subband images and the noise component of the third fixed image; and determine a fourth image component estimate based on the difference between the high-frequency subband coefficient of the fourth set of real-focus subband images and the noise component of the fourth fixed image.

[0117] Optionally, the acquisition module 81 is further configured to acquire infrared video image data collected for the target scene; sequentially acquire a real-focus infrared image frame sequence and a defocus infrared image frame sequence carrying time information from the infrared video image data; and obtain multiple sets of real-focus images and defocus images of the target scene based on the real-focus infrared image frame sequence and the defocus infrared image frame sequence; the reconstruction module 83 is further configured to generate a non-uniformity-corrected infrared video image of the target scene based on the non-uniformity-corrected images corresponding to the multiple sets of real-focus images and defocus images respectively.

[0118] This application also provides an infrared imaging device; please refer to [link / reference]. Figure 9 The infrared imaging device includes a memory 91, a processor 92, and an image acquisition module 93 connected to the processor 92. The memory 91 is used to store programs. The image acquisition module 93 is used to acquire in-focus and out-of-focus images of the same group for a target scene and send them to the processor 92. The processor 92 is used to implement the image non-uniformity correction method described in any of the above embodiments when executing the program.

[0119] In some embodiments, the image acquisition module 93 includes an infrared focal plane array detector.

[0120] Many materials can be selected for the infrared focal plane array detector, such as ferroelectric materials, vanadium oxide, and graphene. However, the materials suitable for multicolor infrared focal plane array devices with tunable response bands are mainly four material systems: mercury cadmium telluride (HgCdTe), quantum well (QWIP), novel type II superlattices, and quantum dot (QDIP).

[0121] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the image non-uniformity correction method as described in any of the above embodiments, and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0122] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image non-uniformity correction method, characterized by, Includes the following steps: Acquire in-focus and out-of-focus images of the same group for the target scene. The in-focus and out-of-focus images refer to images captured at the same time in the same frame image, respectively in the sharp state and in the out-of-focus state. The in-focus image and the out-of-focus image are decomposed by wavelet transform respectively to obtain the corresponding in-focus sub-band image and out-of-focus sub-band image. Based on the high-frequency components in the out-of-focus sub-band image, the fixed image noise components contained in the high-frequency components are determined. Based on the sub-band coefficients of the real-focus sub-band image and the corresponding fixed image noise components, determine the corresponding image component estimates; The image component estimates are reconstructed by inverse wavelet transform to obtain the non-uniformity corrected image of the target scene.

2. The image non-uniformity correction method of claim 1, wherein, The process involves decomposing the in-focus image and the out-of-focus image using wavelet transform to obtain corresponding in-focus sub-band images and out-of-focus sub-band images. Based on the high-frequency components in the out-of-focus sub-band images, the process of determining the fixed image noise components contained in the high-frequency components includes: The in-focus image is decomposed into four sets of in-focus sub-band images by wavelet transform, and the out-of-focus image is decomposed into four sets of out-of-focus sub-band images by wavelet transform. Based on the high-frequency sub-band coefficients of the three sets of defocused sub-band images corresponding to the three directions, the fixed image noise component contained in each set of defocused sub-band images is calculated.

3. The image non-uniformity correction method of claim 2, wherein, The two-dimensional decomposition of the in-focus image using wavelet transform yields four sets of in-focus sub-band images, and the two-dimensional decomposition of the out-of-focus image using wavelet transform yields four sets of out-of-focus sub-band images, including: One-dimensional wavelet decomposition is performed on the one-dimensional data of each row in the real-focus image by wavelet transform to obtain high-frequency information and low-frequency information. One-dimensional wavelet decomposition is then performed on the one-dimensional data of each column in the obtained high-frequency information and low-frequency information to obtain a first set of real-focus sub-band images obtained by row low-pass and column low-pass, a second set of real-focus sub-band images obtained by row low-pass and column high-pass, a third set of real-focus sub-band images obtained by row high-pass and column low-pass, and a fourth set of real-focus sub-band images obtained by row high-pass and column high-pass. One-dimensional wavelet decomposition is performed on the one-dimensional data of each row in the defocused image by wavelet transform to obtain high-frequency information and low-frequency information. One-dimensional wavelet decomposition is then performed on the one-dimensional data of each column in the decomposed high-frequency information and low-frequency information to obtain the first set of defocused sub-band images obtained by row low-pass and column low-pass, the second set of defocused sub-band images obtained by row low-pass and column high-pass, the third set of defocused sub-band images obtained by row high-pass and column low-pass, and the fourth set of defocused sub-band images obtained by row high-pass and column high-pass. In the focused sub-band image and the out-of-focus sub-band image, the first set of focused sub-band images and the first set of out-of-focus sub-band images are low-frequency components, while the others are high-frequency components.

4. The image non-uniformity correction method of claim 3, wherein, The step of calculating the fixed image noise component contained in each of the three sets of defocused sub-band images based on the high-frequency sub-band coefficients corresponding to the three directions respectively includes: Based on the high-frequency sub-band coefficients of the second, third, and fourth sets of defocused sub-band images (which contain high-frequency components) and the confidence level of the fixed image noise, the second fixed image noise component contained in the second set of defocused sub-band images is determined according to the following calculation formula. The third fixed image noise component contained in the third group of defocused sub-band images The fourth fixed image noise component contained in the fourth group of defocused sub-band images : in, It is the second fixed image noise component contained in the second group of defocused sub-band images corresponding to the first defocused image. It refers to the third fixed image noise component contained in the third group of defocused sub-band images corresponding to the first defocused image. It is the fourth fixed image noise component contained in the fourth group of defocused sub-band images corresponding to the first defocused image. It is the confidence level of the fixed-pattern noise component contained in the high-frequency component. , , These are the high-frequency sub-band coefficients of the second group of defocused sub-band images, the third group of defocused sub-band images, and the fourth group of defocused sub-band images, respectively. These are high-pass filter parameters. It represents the number of wavelet decomposition layers.

5. The image non-uniformity correction method as described in claim 4, characterized in that, The step of calculating the fixed image noise component contained in each of the three sets of defocused sub-band images based on the high-frequency sub-band coefficients corresponding to the three directions respectively, further includes: Based on the high-frequency subband coefficients of the second group of defocused subband images, the third group of defocused subband images, and the fourth group of defocused subband images that are high-frequency components, the observed value variance and the true value variance corresponding to the defocused image are determined. Based on the maximum difference between the observed variance and the true variance, determine the variance of the fixed image noise component contained in the high-frequency sub-band coefficients of the high-frequency component in the defocused sub-band image; The confidence level of the fixed image noise is calculated based on the variance of the fixed image noise component.

6. The image non-uniformity correction method as described in claim 4, characterized in that, The step of determining the corresponding image component estimate based on the sub-band coefficient of the real-focus sub-band image and the corresponding fixed image noise component includes: The estimated value of the second image component is determined based on the difference between the high-frequency subband coefficient of the second set of real-focus subband images and the noise component of the second fixed image. The estimated value of the third image component is determined based on the difference between the high-frequency subband coefficient of the third set of real-focus subband images and the noise component of the third fixed image. The estimated value of the fourth image component is determined based on the difference between the high-frequency subband coefficient of the fourth set of real-focus subband images and the noise component of the fourth fixed image.

7. The image non-uniformity correction method according to any one of claims 1 to 6, characterized in that, The acquisition of in-focus and out-of-focus images of the same group captured for the target scene includes: Acquire infrared video image data collected for the target scene; Sequentially obtain a real-focus infrared image frame sequence and a defocus infrared image frame sequence carrying time information from the infrared video image data, and obtain multiple sets of real-focus images and defocus images of the target scene based on the real-focus infrared image frame sequence and the defocus infrared image frame sequence. After reconstructing the estimated image components using inverse wavelet transform to obtain the non-uniformity-corrected image of the target scene, the process includes: Based on the non-uniformity-corrected images corresponding to the multiple sets of in-focus and out-of-focus images, a non-uniformity-corrected infrared video image of the target scene is generated.

8. An image non-uniformity correction device, characterized in that, include: The acquisition module is used to acquire both in-focus and out-of-focus images of the same group for the target scene. The determination module is used to decompose the real-focus image and the defocus image respectively by wavelet transform to obtain the corresponding real-focus sub-band image and defocus sub-band image, and to determine the fixed image noise component contained in the high-frequency component based on the high-frequency component in the defocus sub-band image. It is also used to determine the corresponding image component estimate based on the sub-band coefficient of the real-focus sub-band image and the corresponding fixed image noise component; The reconstruction module is used to reconstruct the estimated values ​​of the image components through inverse wavelet transform to obtain the non-uniformity corrected image of the target scene.

9. An infrared imaging device, characterized in that, The device includes a memory, a processor, and an image acquisition module connected to the processor. The memory is used to store a program. The image acquisition module is used to acquire in-focus and out-of-focus images of the same group for a target scene and send them to the processor. The processor is used to implement the image non-uniformity correction method according to any one of claims 1 to 7 when executing the program.

10. The infrared imaging device as described in claim 9, characterized in that, The image acquisition module includes an infrared focal plane array detector.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image non-uniformity correction method as described in any one of claims 1 to 7.

Images