High-signal-to-noise-ratio long-wave infrared multispectral imaging system and imaging method thereof

Abstract

The invention provides a high-signal-to-noise-ratio long-wave infrared multispectral imaging system and an imaging method thereof, which are used for detecting and identifying infrared weak and small targets under a dense fog condition. A dark channel prior method is adopted to carry out defogging processing on each acquired spectrum image so as to eliminate imaging quality degradation caused by atmospheric scattering and absorption; a block matching technology is used for carrying out block matching on a defogged image, similar blocks in a multispectral image are extracted, a collaborative filtering technology is combined for processing the similar blocks, noise is removed, the contrast ratio of the image is enhanced, and the signal-to-noise ratio is doubled. According to the multispectral imaging system, the imaging quality can be effectively improved by utilizing spectral feature differences of different objects and combining an image processing technology, and the multispectral imaging system has high anti-interference performance, is beneficial to foggy day target detection and recognition and has wide application prospects.

Description

Technical Field

[0001] The invention belongs to the technical field of optical engineering, and specifically relates to a high signal-to-noise ratio long-wave infrared multi-spectral imaging system and an imaging method thereof. Background Art

[0002] Infrared imaging technology is widely used in military and civilian fields such as reconnaissance and early warning, monitoring and security, medical diagnosis, and traffic surveillance, and is of great significance to improving all-weather information warfare and defense capabilities. However, in dense fog, the quality of traditional long-wave infrared imaging is seriously degraded due to water vapor absorption and reduced emissivity. The target signal is weak, the edge features are blurred, and the signal-to-noise ratio is low. These problems seriously hinder the accuracy of target detection, especially in the detection of weak targets at long distances, which constitutes a huge technical obstacle. Therefore, how to improve the imaging contrast and signal-to-noise ratio under foggy conditions is a difficult problem that needs to be solved urgently in current infrared imaging technology.

[0003] In response to the above problems, traditional infrared imaging technology mainly relies on enhancement and reconstruction methods to restore target details and suppress background noise. However, this type of method only relies on intensity information. In dense fog conditions, due to the extremely low contrast between the target and the background, it is difficult to segment, which can easily lead to missed judgments and misjudgments. Multispectral imaging technology can highlight the target signal in the spatial domain of the characteristic spectrum by capturing the radiation information of the target in different spectral bands. Compared with plenoptic images, this technology can provide richer spectral information, thereby achieving more accurate target detection and recognition. At present, the development of multispectral imaging technology in the visible light band is relatively mature, but there are few related studies in the infrared band, and there is a lack of corresponding imaging system design solutions. In addition, there is still a lack of research foundation on how to reasonably divide the imaging band and use the spectral difference between the target and the background to achieve enhanced targets. In view of the loss of edge and detail information of infrared imaging targets under harsh conditions such as dense fog, and the difficulty in distinguishing between targets and backgrounds, there is a relative lack of algorithm research on image dehazing and denoising using multispectral information, and traditional visible light image processing methods are not applicable. Summary of the invention

[0004] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a high signal-to-noise ratio long-wave infrared multi-spectral imaging system and an imaging method thereof, which utilize spectral information to suppress background noise, and are used to solve the problem of low signal-to-noise ratio of long-wave infrared imaging caused by water vapor absorption and scattering under foggy conditions, so as to improve the accuracy of infrared target detection and recognition.

[0005] The technical solution of the present invention is as follows:

[0006] On the one hand, the present invention provides a high signal-to-noise ratio long-wave infrared multi-spectral imaging system for detecting and identifying weak infrared targets under dense fog conditions, which is characterized by comprising:

[0007] Infrared multispectral imaging system, used to collect multispectral images of target scenes at different wavelengths through a multispectral imaging system composed of a long-wave infrared camera and an adjustable filter;

[0008] A defogging processing module, which uses a dark channel prior method to perform defogging on the multispectral image to eliminate the degradation of imaging quality caused by atmospheric scattering and absorption;

[0009] The multi-channel block matching module uses the block matching technology to perform block matching on the dehazed image, extract similar blocks in the multi-spectral image, and generate a multi-channel block matching group;

[0010] The image enhancement module combines collaborative filtering technology to process similar blocks, remove noise and enhance the contrast of the image, thereby improving the signal-to-noise ratio and obtaining a haze-free image restoration map.

[0011] Further, the long-wave multi-spectral imaging system comprises: an adjustable filter for selectively filtering specific wavelengths in the multiple spectral images; a long-wave infrared camera used in conjunction with the adjustable filter to capture images processed by the filter;

[0012] Furthermore, the defogging processing module includes: a dark channel prior map acquisition unit, which is used to calculate and obtain a dark channel prior map based on the collected multispectral image; an infrared atmospheric light value estimation unit, which is used to estimate an approximate estimate of the infrared atmospheric light based on the dark channel prior map; and a multispectral dark channel prior reconstruction unit, which is used to reconstruct a fog-free image based on the infrared fog transmittance and the multispectral image.

[0013] Furthermore, the multi-channel block matching module includes: a threshold filtering unit for increasing the distance matching efficiency and success rate; and a distance calculation unit for calculating the distance between blocks.

[0014] Furthermore, the image enhancement module includes: a transform domain-based threshold collaborative filtering unit, which is used to perform three-dimensional linear transformation, threshold filtering and three-dimensional linear inverse transformation on the multi-channel block matching group image to remove noise; a transform domain-based quadratic Wiener filtering unit, which is used to perform Wiener filtering on the denoised coarsely reconstructed multi-channel matching group image to generate a finely reconstructed denoised block matching group image; a weighted average aggregation unit, which is used to perform weighted average aggregation on the finely reconstructed denoised block matching group image to obtain a high-quality haze-free image restoration image.

[0015] On the other hand, the present invention also provides a high signal-to-noise ratio long-wave infrared multi-spectral imaging method, which is characterized in that it comprises the following steps:

[0016] a) Multispectral image acquisition: A multispectral imaging system is constructed using a long-wave infrared camera and a tunable filter to collect multiple narrow spectrum images of different wavelengths;

[0017] b) Obtain infrared dark channel prior map: According to any spectral channel of the foggy image, calculate the dark channel prior map I in the window of the pixel point dark , the formula is as follows:

[0018]

[0019] In the formula, I c represents any spectral channel of the foggy image I, and Ω(x) represents the window at pixel x;

[0020] c) Estimation of infrared atmospheric light value: taking the first certain proportion of the dark channel prior image as the approximate estimated value of infrared atmospheric light;

[0021] d) Estimation of infrared fog transmittance: Combine the atmospheric scattering model and dark channel prior map to calculate the transmittance The expression is as follows:

[0022]

[0023] In the formula, ω is a constant value, 0<ω≤1;

[0024] e) Multi-spectral dark channel prior reconstruction: Reconstruct the fog-free image based on the transmittance, infrared atmospheric light value and empirical value to improve the image brightness of the infrared multi-spectral dark channel prior reconstruction;

[0025] f) Multi-channel block matching algorithm: threshold filtering is applied to the multi-spectral dark channel image to generate a multi-channel block matching group;

[0026] g) Transform domain-based threshold collaborative filtering: Perform a three-dimensional linear transformation on the multi-channel block matching group image, convert the image from the spatial domain to the frequency domain, perform threshold filtering to remove noise, and then convert the image back to the spatial domain through a three-dimensional linear inverse transformation to obtain the denoised block matching group image, and use a smoothing algorithm to equalize the effects of the overlapping parts to obtain a coarsely reconstructed denoised image;

[0027] h) Quadratic Wiener filtering based on transform domain: Perform three-dimensional linear transformation on the denoised coarsely reconstructed multi-channel matching group image, convert the image from the spatial domain to the frequency domain, apply Wiener filtering to generate a finely reconstructed denoised block matching group image, and then convert the image back to the spatial domain through a three-dimensional linear inverse transformation;

[0028] i) Weighted average aggregation: The overlapping parts in the finely reconstructed denoised block matching group image are aggregated using weighted filtering to obtain a finely reconstructed denoised image.

[0029] Furthermore, the infrared atmospheric light approximate estimation value is achieved by taking the first 4.1% of the dark channel prior image.

[0030] Furthermore, the empirical value is used to improve the brightness of an image reconstructed a priori by the infrared multi-spectral dark channel.

[0031] Furthermore, in step g), the three-dimensional linear transformation includes a three-dimensional discrete Fourier transform and a three-dimensional discrete cosine transform, and the threshold filtering is to filter out small coefficients according to a preset threshold coefficient to remove noise.

[0032] Furthermore, in step h), the Wiener filter coefficients are generated jointly by the coarse reconstruction image and the dark channel reconstruction image.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1) Use tunable filters to obtain multiple spectral images. According to the spectral response characteristics of different targets, flexibly adjust the filter band to obtain accurate and rich spectral information and improve imaging quality.

[0035] 2) Use the dark channel method to process multi-spectral images to achieve dehazing, effectively eliminate the influence of atmospheric scattering and absorption, achieve contrast enhancement, and highlight target features.

[0036] 3) Combining block matching and collaborative filtering technology to achieve denoising, while retaining the edge and detail information of the target, significantly reducing image noise. According to the embodiment of the present invention, the signal-to-noise ratio can be increased to twice the original.

[0037] The present invention not only improves the performance of the multispectral imaging system, but also provides an effective defogging and denoising solution for the field of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flow chart of the high signal-to-noise ratio long-wave infrared multi-spectral imaging method of the present invention;

[0039] Figure 2 Schematic diagram of infrared dark channel prior calculation;

[0040] Figure 3 This is a schematic diagram of multi-spectral fast matching;

[0041] Figure 4 It is a schematic diagram of multi-spectral system filtering;

[0042] Figure 5 This is a comparison chart between plenoptic image and multispectral imaging results;

[0043] Table 1 is a comparison table of signal-to-noise ratio between plenoptic image and multispectral imaging; DETAILED DESCRIPTION

[0044] The present invention is further described below by way of embodiments in conjunction with the accompanying drawings, but the protection scope of the present invention shall not be limited thereto.

[0045] The present embodiment provides a high signal-to-noise ratio long-wave infrared multi-spectral imaging system for detecting and identifying weak infrared targets under dense fog conditions, including: an infrared multi-spectral imaging system for capturing multiple spectral images of a target scene; a defogging processing module, which uses a dark channel prior method to perform defogging on the multiple spectral images to eliminate the degradation of imaging quality caused by atmospheric scattering and absorption; a multi-channel block matching module, which uses block matching technology to perform block matching on the defogged image to extract similar blocks in the multi-spectral image; an image enhancement module, which combines collaborative filtering technology to process similar blocks, remove noise and enhance the contrast of the image, thereby improving the signal-to-noise ratio. Among them, the long-wave multi-spectral imaging system includes: an adjustable filter for selectively filtering specific wavelengths in the multiple spectral images; a long-wave infrared camera, which is used in conjunction with the adjustable filter to capture the image processed by the filter. The defogging processing module includes: a dark channel prior map acquisition unit, which is used to obtain a dark channel prior map; an infrared atmospheric light value estimation unit, which is used to estimate the approximate estimated value of infrared atmospheric light, where α is taken as 0.041; an infrared fog transmittance estimation unit, which is used to estimate the infrared fog transmittance, where ω is taken as 0.95; and a multi-spectral dark channel prior reconstruction unit, which is used to reconstruct a fog-free image. The multi-channel block matching module includes: a threshold filtering unit, which is used to increase the distance matching efficiency and success rate; a distance calculation unit, which is used to calculate the distance between blocks, and the distance calculation method includes but is not limited to grayscale-based mean square error, feature-based mean square error, and transform domain coefficient-based mean square error. The image enhancement module includes: a transform domain-based threshold collaborative filtering unit, which is used to denoise the block matching group image; a transform domain-based quadratic Wiener filtering unit, which is used to generate a finely reconstructed denoised block matching group image; and a weighted average aggregation unit, which is used to aggregate to obtain a finely reconstructed denoised image.

[0046] Example:

[0047] The present invention models the infrared band multispectral imaging process and adopts the atmospheric scattering model as follows:

[0048] I(x)=J(x)t(x)+A(1-t(x))+N(x)

[0049] Where x is the pixel point of the multispectral image, x∈X, C is the number of spectral bands, I(x) is the foggy image, J(x) is the fog-free image, A is the atmospheric light value in the infrared band, t(x) is the transmittance, and N(x) is the noise.

[0050] Figure 1 Flow chart of the high signal-to-noise ratio long-wave infrared multi-spectral imaging method of the present invention, as shown in Figure 1 As shown,

[0051] Step 1: Multispectral image acquisition: A multispectral imaging system is constructed using a long-wave infrared camera and a tunable filter to collect C narrow spectrum images I of different wavelengths.

[0052] Step 2: Obtain infrared dark channel prior map, such as Figure 2 shown.

[0053]

[0054] Among them, I dark represents the dark channel prior map, I c represents any spectral channel of the foggy image I, and Ω(x) represents the window at pixel x.

[0055] Step 3: Estimate the infrared atmospheric light value. Take the first α% of the dark channel prior image as the approximate estimate of the infrared atmospheric light. In the embodiment, α is set to 4.1.

[0056]

[0057] Step 4: Estimate infrared fog transmittance. Since the dark channel value is almost zero, combined with formula (1), the transmittance can be obtained The expression is:

[0058]

[0059] Wherein, ω represents a constant value, 0<ω≤1, and is 0.95 in the embodiment.

[0060] Step 5: Multispectral dark channel prior reconstruction. Reconstructing the haze-free image for:

[0061]

[0062] Among them, t0 is an empirical value, which is used to improve the image brightness of the infrared multi-spectral dark channel prior reconstruction.

[0063] After the above algorithm processing, the multi-spectral dark channel prior reconstruction map can be expressed as:

[0064]

[0065] Next, we further remove noise to improve the reconstruction effect.

[0066] Step 6: Multi-channel block matching algorithm. Simple threshold filtering is used for the multi-spectral dark channel image to increase the distance matching efficiency and success rate, and then the multi-channel block matching group is generated according to the following formula: like Figure 3 shown.

[0067]

[0068] in, for, for, Represents the distance between the two. The distance can be calculated using different distance calculation methods, including but not limited to grayscale-based mean square error, feature-based mean square error, transform domain coefficient-based mean square error, etc. The threshold coefficient is preset, and the specific value of the coefficient can be determined based on engineering experience or statistical priors.

[0069] Step 7: Threshold collaborative filtering based on transform domain, such as Figure 4 shown.

[0070]

[0071] in, It is a block matching group image stacked by multi-channel block matching algorithm. Indicates a three-dimensional linear transformation of the block matching group image, converting the image from the spatial domain to the frequency domain. Y indicates threshold filtering, filtering out small coefficients and removing noise. The three-dimensional linear inverse transform is used to transform the block matching group graph from the frequency domain back to the spatial domain. This is the block matching group image after threshold collaborative filtering denoising. For the overlapping parts, the smoothing algorithm is used to equalize the effect, and the rough reconstruction denoised image can be obtained by aggregation.

[0072] Step 8: Secondary Wiener filtering based on transform domain. The same principle and method as above are used to obtain the denoising rough reconstruction multi-channel matching group

[0073]

[0074] Then, the denoised coarsely reconstructed multi-channel matching group image is filtered by Wiener to generate the finely reconstructed denoised block matching group image.

[0075]

[0076] in, represents the Wiener filter coefficient, which is generated by the coarse reconstruction image and the dark channel reconstruction image. represents a three-dimensional linear transformation, converting the coarse reconstruction from the spatial domain to the frequency domain, represents the three-dimensional inverse linear transform, which converts the Wiener filtered image from the frequency domain back to the spatial domain. Represents the finely reconstructed denoised block matching group graph.

[0077] Step 9: Weighted Average Aggregation

[0078] For the overlapping parts, weighted filtering can be used to aggregate and obtain a finely reconstructed denoised image.

[0079]

[0080] in, represents the weighted weight, Represents the overlapping feature function. If there is overlap, it takes 1, and if there is no overlap, it takes 0. The finely reconstructed denoised image after weighted filtering and smoothing It is a high-quality restored image of the haze-free image.

[0081] According to the above steps to process the multispectral image, the results are as follows Figure 5 And as shown in Table 1.

[0082] Table 1

[0083]

[0084] It can be seen that the multispectral imaging system of the present invention, by utilizing the differences in spectral characteristics of different objects and combining image processing technology, can effectively improve the imaging quality, has strong anti-interference ability, and is helpful for target detection and recognition in foggy weather.

Claims

1. A high signal-to-noise ratio long-wave infrared multi-spectral imaging system for infrared weak target detection and recognition under dense fog conditions, characterized in that: include: Infrared multispectral imaging system, used to collect multispectral images of target scenes at different wavelengths through a multispectral imaging system composed of a long-wave infrared camera and an adjustable filter; A defogging processing module, which uses a dark channel prior method to perform defogging on the multispectral image to eliminate the degradation of imaging quality caused by atmospheric scattering and absorption; The multi-channel block matching module uses the block matching technology to perform block matching on the dehazed image, extract similar blocks in the multi-spectral image, and generate a multi-channel block matching group; The image enhancement module combines collaborative filtering technology to process similar blocks, remove noise and enhance the contrast of the image, thereby improving the signal-to-noise ratio and obtaining a haze-free image restoration map.

2. A high signal-to-noise ratio long-wave infrared multi-spectral imaging system according to claim 1, characterized in that: The long-wave multi-spectral imaging system comprises: an adjustable filter for selectively filtering specific wavelengths in the multiple spectral images; and a long-wave infrared camera used in conjunction with the adjustable filter to capture images processed by the filter.

3. The high signal-to-noise ratio long-wave infrared multi-spectral imaging system according to claim 1, characterized in that: The defogging processing module includes: a dark channel prior map acquisition unit, which is used to calculate and acquire a dark channel prior map based on the collected multispectral image; an infrared atmospheric light value estimation unit, which is used to estimate an approximate estimated value of infrared atmospheric light based on the dark channel prior map; and a multispectral dark channel prior reconstruction unit, which is used to reconstruct a fog-free image based on the infrared fog transmittance and the multispectral image.

4. The method for designing a high signal-to-noise ratio long-wave infrared multi-spectral imaging system according to claim 1, characterized in that: The multi-channel block matching module includes: a threshold filtering unit for increasing the distance matching efficiency and success rate; and a distance calculation unit for calculating the distance between blocks.

5. The high signal-to-noise ratio long-wave infrared multi-spectral imaging system according to claim 1, characterized in that: The image enhancement module includes: a threshold collaborative filtering unit based on a transform domain, which is used to perform three-dimensional linear transformation, threshold filtering and three-dimensional linear inverse transformation on a multi-channel block matching group image to remove noise; a quadratic Wiener filtering unit based on a transform domain, which is used to perform Wiener filtering on a denoised coarsely reconstructed multi-channel matching group image to generate a finely reconstructed denoised block matching group image; and a weighted average aggregation unit, which is used to perform weighted average aggregation on a finely reconstructed denoised block matching group image to obtain a high-quality haze-free image restoration image.

6. A high signal-to-noise ratio long-wave infrared multispectral imaging method, characterized in that: The following steps are involved: a) Multispectral image acquisition: A multispectral imaging system is constructed using a long-wave infrared camera and a tunable filter to collect multiple narrow spectrum images of different wavelengths; b) Obtain infrared dark channel prior map: According to any spectral channel of the foggy image, calculate the dark channel prior map I in the window of the pixel point dark , the formula is as follows: In the formula, I c represents any spectral channel of the foggy image I, and Ω(x) represents the window at pixel x; c) Estimation of infrared atmospheric light value: taking the first certain proportion of the dark channel prior image as the approximate estimated value of infrared atmospheric light; d) Estimation of infrared fog transmittance: Combine the atmospheric scattering model and dark channel prior map to calculate the transmittance The expression is as follows: In the formula, ω is a constant value, 0<ω≤1; e) Multi-spectral dark channel prior reconstruction: Reconstruct the fog-free image based on the transmittance, infrared atmospheric light value and empirical value to improve the image brightness of the infrared multi-spectral dark channel prior reconstruction; f) Multi-channel block matching algorithm: threshold filtering is applied to the multi-spectral dark channel image to generate a multi-channel block matching group; g) Transform domain-based threshold collaborative filtering: Perform a three-dimensional linear transformation on the multi-channel block matching group image, convert the image from the spatial domain to the frequency domain, perform threshold filtering to remove noise, and then convert the image back to the spatial domain through a three-dimensional linear inverse transformation to obtain the denoised block matching group image, and use a smoothing algorithm to equalize the effects of the overlapping parts to obtain a coarsely reconstructed denoised image; h) Quadratic Wiener filtering based on transform domain: Perform three-dimensional linear transformation on the denoised coarsely reconstructed multi-channel matching group image, convert the image from the spatial domain to the frequency domain, apply Wiener filtering to generate a finely reconstructed denoised block matching group image, and then convert the image back to the spatial domain through a three-dimensional linear inverse transformation; i) Weighted average aggregation: The overlapping parts in the finely reconstructed denoised block matching group image are aggregated using weighted filtering to obtain a finely reconstructed denoised image.

7. The high signal-to-noise ratio long-wave infrared multi-spectral imaging method according to claim 6, characterized in that: The infrared atmospheric light approximation is achieved by taking the first 4.1% of the dark channel prior map.

8. The high signal-to-noise ratio long-wave infrared multi-spectral imaging method according to claim 6, characterized in that: The empirical value is used to improve the image brightness of the infrared multi-spectral dark channel prior reconstruction.

9. The high signal-to-noise ratio long-wave infrared multi-spectral imaging method according to claim 6, characterized in that: In step g), the three-dimensional linear transformation includes a three-dimensional discrete Fourier transform and a three-dimensional discrete cosine transform, and the threshold filtering is to filter out small coefficients according to a preset threshold coefficient to remove noise.

10. The high signal-to-noise ratio long-wave infrared multi-spectral imaging method according to claim 6, characterized in that: In step h), the Wiener filter coefficients are generated jointly by the coarse reconstruction image and the dark channel reconstruction image.

Images