Image defogging method

Abstract

The invention relates to an image defogging method, which comprises the following steps of: acquiring a color image of a target scene in fog and a plurality of infrared images of different wavebands; performing weighted fusion on the plurality of infrared images of different wavebands to obtain a fused image; iteratively segmenting the color image by taking the maximum difference value between the pixel mean value and the pixel variance in the region as a target to obtain a target region with a set size; taking a pixel mean value of pixel points closest to pure white in the target area as an atmospheric light value; estimating a transmissivity image based on the atmospheric light value and the color image, and performing guided filtering on the transmissivity image by using the infrared image of a set wave band; and calculating according to the atmospheric light value and the transmissivity image to obtain a fogless image of the color image, and performing YUV channel brightness fusion on the fogless image and the fusion image to obtain a defogged image. The method can adapt to different foggy scenes under the condition that parameters are not set, and a better defogging effect is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image defogging method. BACKGROUND

[0002] In the process of outdoor optical system imaging in foggy weather, the infrared imaging system is affected by solid-liquid particles in the atmosphere, resulting in a decline in the quality of the observed image, causing phenomena such as high overall brightness value, low contrast, and reduced dynamic range. The image degradation problem prevents clear and stable high-quality images or sequences from being obtained, and makes subsequent recognition, tracking, segmentation, and other algorithms unable to work effectively. Especially with the rapid development of autonomous driving and assisted driving, new requirements are put forward for the imaging quality in foggy weather.

[0003] The defogging technology can be currently divided into an enhancement method based on image processing and an enhancement method based on image restoration. Among them, the enhancement method based on image processing mainly includes histogram equalization, filtering method, wavelet change, Retinex algorithm, and atmospheric modulation transfer function method. These methods achieve good results to some extent, but also have many problems. For example, for targets with uneven illumination or complex distance changes, halo phenomenon and blocking effect are easily caused, such methods have poor universality, and there is a problem of large amount of calculation. The defogged image only starts from the visual angle and does not conform to the actual physical law to some extent.

[0004] The method based on image restoration starts from the physical process of causing image quality in foggy weather, and inverses the physical degradation process to achieve the purpose of defogging. According to the basic research on atmospheric particles optics, the atmospheric physical scattering model is obtained, and the optimal parameters of the atmospheric physical scattering model are solved to inverse the fog-free image. Therefore, the goal of this method is to find the optimal parameters of the model, and most of them need to provide additional prior information such as scene depth information or additional means to obtain depth auxiliary information, fog concentration information, or scene polarization information. Due to the difficulty and complexity of obtaining additional information, defogging methods based only on image information have rapidly developed. The dark channel method is a statistical prior information proposed by He Kai-ming, and has become the most effective defogging prior information. However, the dark channel method is based on the statistics of three-channel information of a large number of color images, and is not suitable for infrared images. The existing defogging method for infrared images depends on the given parameters to perform superpixel segmentation and then estimate the atmospheric light value A, and then estimate the transmittance using double thresholds. The initially given parameters need to be adjusted according to different scenes. Therefore, it is necessary to find a more effective method that can adapt to different foggy scenes without setting parameters to achieve better defogging effect. SUMMARY

[0005] The technical problem solved by the present application is to provide an image defogging method which can adapt to different foggy scenes without setting parameters and achieve better defogging effect.

[0006] The technical solution adopted by the present application to solve the technical problem is to provide an image defogging method, comprising the following steps:

[0007] Collecting a color image and a plurality of infrared images of different wave bands of a target scene in fog;

[0008] Weighted fusion of the plurality of infrared images of different wave bands to obtain a fused image;

[0009] Iterative segmentation of the color image to obtain a target region of a set size, with the difference between the mean value of pixels in the region and the pixel variance being maximized as the target;

[0010] Taking the mean value of the pixel points closest to the pure white color in the target region as the atmospheric light value;

[0011] Estimating a transmittance image based on the atmospheric light value and the color image, and guiding filtering the transmittance image using the infrared image of a set wave band;

[0012] Calculating a defogging image of the color image according to the atmospheric light value and the transmittance image, and then performing YUV channel brightness fusion on the defogging image and the fused image to obtain a defogging image.

[0013] Further, the iterative segmentation of the color image to obtain a target region of a set size, with the difference between the mean value of pixels in the region and the pixel variance being maximized as the target, comprises:

[0014] Equally dividing the color image into a set number of sub-regions;

[0015] Calculating the difference between the mean value of pixels and the pixel variance of each sub-region, equally dividing the sub-region with the largest difference into a set number of new sub-regions, repeating the step until the new sub-regions meet the set size, and outputting the new sub-region with the largest difference between the mean value of pixels and the pixel variance as the target region.

[0016] Further, the set number is 4.

[0017] Further, the set size is 1 / 16 or 1 / 32 of the color image.

[0018] Further, the transmittance image is estimated according to the following formula:

[0019]

[0020] wherein I c(y) is the color image, A c is the atmospheric light value, and ω is an empirical constant.

[0021] Further, the weighted fusion of the plurality of infrared images of different wavebands comprises:

[0022] detecting feature descriptors of the color image and each of the infrared images and performing feature matching, thereby aligning each of the infrared images with the color image;

[0023] fusing the aligned infrared images by means of weighted averaging.

[0024] Further, the pixel mean value of the pixel point closest to pure white is obtained by the following method:

[0025] calculating the Euclidean distance of each pixel point in the region from pure white;

[0026] selecting the pixel point with the smallest Euclidean distance and calculating the pixel mean value thereof.

[0027] Further, the plurality of infrared images of different wavebands comprises infrared images of 650-750 nm waveband, 750-850 nm waveband, and greater than 850 nm waveband.

[0028] Further, the infrared image of the set waveband comprises an infrared image of 750-850 nm waveband.

[0029] Advantages

[0030] Compared with the prior art, the present application has the following advantages and positive effects: the present application extracts a target region of a set size with the largest difference between the pixel mean value and the pixel variance by using the iterative segmentation method, and takes the pixel mean value of the pixel point closest to pure white in the target region as the atmospheric light value, which reduces the influence of some strong light environments such as car high beam, street lamp, etc., and obtains a more stable and accurate atmospheric light value compared with directly selecting the pixel mean value of the dark channel high luminance pixel point as the atmospheric light value; the present application uses a color image to estimate the atmospheric light value for guiding the infrared image to perform image defogging, which can more accurately estimate the atmospheric light value, and at the same time refines the estimated transmittance image with the aid of the long-wave red image, retains more image edge details, and obtains better edge retention capability and defogging effect. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart of an embodiment of the present application. DETAILED DESCRIPTION

[0032] The application will be further described in connection with specific embodiments. It should be understood that these embodiments are only used to illustrate the present application and not used to limit the scope of the present application. Furthermore, it should be understood that after reading the content of the present application, those skilled in the art can make various modifications or changes to the present application, and these equivalent forms also fall within the scope defined by the appended claims.

[0033] Embodiments of the present application relate to a dehazing method based on image fusion, as shown in the following steps: Figure 1

[0034] Acquire a color image and a plurality of infrared images of different wave bands of a target scene in fog;

[0035] Fuse the infrared images of different wave bands to obtain a fused image;

[0036] Segment the color image iteratively to obtain a target region of a set size, with the goal of maximizing the difference between the mean value of pixels and the variance of pixels in the region;

[0037] Take the mean value of the pixel points closest to the pure white color in the target region as the atmospheric light value;

[0038] Estimate the transmittance image based on the atmospheric light value and the color image, and guide filter the transmittance image using the infrared image of a set wave band;

[0039] Calculate the dehazed image of the color image based on the atmospheric light value and the transmittance image, and then perform YUV channel brightness fusion on the dehazed image and the fused image to obtain a dehazed image.

[0040] The foggy image can be acquired using a narrow-band filter and a CMOS camera, and the color image, the 650nm-750nm wave band image, the 750nm-850nm wave band image, and the 850nm and longer wave band image are selected respectively.

[0041] Next, the foggy color image Ic(y), the 650nm-750nm wave band image, the 750nm-850nm wave band image, and the 850nm and longer wave band image are fused. The fusion steps include:

[0042] Use the ORB (Oriented FAST and Rotated BRIEF) feature detector to detect key points and calculate descriptors;

[0043] Then use the BFMatcher (Brute-Force Matcher) to match the descriptors;

[0044] ​A homography matrix is calculated using the RANSAC algorithm, which can transform the remaining images to align with the reference image;

[0045] A color image is selected as the reference, and the remaining images are perspective transformed using the homography matrix to obtain the aligned images.

[0046] The three-band infrared images are fused by weighted average.

[0047] The quadtree search is used to determine the atmospheric light value when the color image is iteratively segmented, but other segmentation methods can also be selected. The specific operation is as follows:

[0048] The image is segmented into four sub-regions with consistent area, and the pixel mean M n and the variance D n are calculated respectively.

[0049] S n = M n -D n

[0050] The kth sub-region satisfies:

[0051] {k|s k = max(s n )}

[0052] The k sub-regions are further segmented into four smaller regions with equal area, and the above process is repeated until the image is divided into small enough regions. Usually, 1 / 16 or 1 / 32 of the original area can be selected as the threshold. At this time, the target region obtains the pixel mean of the point closest to (255, 255, 255) in the region as the atmospheric light value A. Compared with directly selecting the pixel mean of the 0.1% brightness in the dark channel as the atmospheric light value, this method can avoid the influence of some strong light environments such as car high beam, street lamp, etc., and the obtained atmospheric light value is more stable and accurate.

[0053] When estimating the transmittance image, it can be assumed that the transmittance of a local region Ω(x) is constant, denoted as t(x). According to the dark channel prior theory, the transmittance estimation formula can be obtained as:

[0054]

[0055] where ω is a constant to preserve a small part of the fogging effect. If there is no fogging effect in the actual image, it may cause misjudgment in the perception of the depth of field. According to experience, the constant 0.95 is more appropriate.

[0056] The image of 750nm-850nm waveband is taken as the guide image to guide filter the transmittance image. In the fog environment, this waveband has clearer edges than the color image, and the guide image as the guide filter can have better edge preservation effect.

[0057] The fog-free image of the color image is calculated according to the atmospheric light value A and the transmittance image t(x), and the formula is:

[0058]

[0059] The color dehazing image is fused with the infrared weighted fusion image in YUV channel brightness, specifically, the chroma components (U, V) of the color image are kept unchanged to retain the color information of the color image, and the brightness information of the infrared image is weighted and averaged with the brightness component (Y) of the color image:

[0060] Y fuse = aY ir + (1-a)Y color

[0061] The YUV channel is converted into the RGB channel to obtain the final image.

Claims

1. An image defogging method, characterized in that: The following steps are involved: Collect color images of target scenes in fog and infrared images of multiple different bands; Performing weighted fusion on the plurality of infrared images of different bands to obtain a fused image; Iteratively segmenting the color image with the goal of maximizing the difference between the pixel mean and the pixel variance within the region to obtain a target region of a set size; The average value of the pixels closest to pure white in the target area is taken as the atmospheric light value; estimating a transmittance image based on the atmospheric light value and the color image, and performing guided filtering on the transmittance image using the infrared image of a set wavelength band; A fog-free image of the color image is calculated according to the atmospheric light value and the transmittance image, and then the fog-free image and the fused image are subjected to YUV channel brightness fusion to obtain a defogged image.

2. The method according to claim 1, characterized in that The method of iteratively segmenting the color image with the goal of maximizing the difference between the pixel mean and the pixel variance in the region to obtain a target region of a set size includes: Dividing the color image into a set number of sub-regions; Calculate the difference between the pixel mean and the pixel variance of each sub-region, select the sub-region with the largest difference and divide it into a set number of new sub-regions, repeat this step until the new sub-region meets the set size, and output the sub-region with the largest difference between the pixel mean and the pixel variance as the target region.

3. The method according to claim 2, characterized in that The set number is 4.

4. The method according to claim 2, characterized in that The set size is 1 / 16 or 1 / 32 of the color image.

5. The method according to claim 1, wherein The transmittance image is estimated according to the following formula: Among them, I c (y) is the color image, A c is the atmospheric light value, and ω is an empirical constant.

6. The method according to claim 1, characterized in that The weighted fusion of the plurality of infrared images of different bands includes: detecting feature descriptors of the color image and each of the infrared images and performing feature matching, thereby aligning each of the infrared images with the color image; The aligned infrared images are fused by weighted averaging.

7. The method according to claim 1, characterized in that The pixel mean value of the pixel closest to pure white is obtained by the following method: Calculate the Euclidean distance between each pixel in the area and pure white; Select the pixel with the smallest Euclidean distance and calculate its pixel mean.

8. The method according to claim 1, characterized in that The plurality of infrared images in different wavelength bands include infrared images in a wavelength band of 650nm-750nm, a wavelength band of 750nm-850nm, and a wavelength band greater than 850nm.

9. The method according to claim 1, characterized in that The infrared image in the set wavelength band includes the infrared image in the wavelength band of 750nm-850nm.

Images