A real-time polarization dehazing method based on atmospheric light estimation and dynamic range adjustment

Abstract

The present invention discloses a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment, which relates to the technical field of image defogging processing, and comprises the following steps: pre-acquiring multiple polarization images of a scene, parsing and obtaining polarization images in multiple directions, calibrating the atmospheric light area Rect based on the application scenario, and estimating the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , estimate the atmospheric light intensity A; then estimate the atmospheric transmittance t, and obtain the average atmospheric light intensity A mean The estimation is performed and the dynamic range is adjusted based on the atmospheric light intensity A. The present invention can effectively solve the image clarity problem in foggy and hazy weather, and can be directly applied to practical projects, and has a wide range of application prospects in the fields of foggy weather monitoring, navigation, etc.

Description

Technical Field

[0001] The present invention relates to the technical field of image defogging processing, and in particular to a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment. Background Art

[0002] Under the influence of haze weather, the visibility of optical imaging systems and the contrast of collected images are greatly reduced, which seriously affects people's observation of the scene and has an adverse impact on various aspects such as video surveillance, transportation, and shipping. Therefore, how to restore and improve the quality of optical imaging under haze weather conditions, that is, to study image defogging technology, is of great significance.

[0003] At present, defogging technology is mainly divided into two categories: 1) Image processing defogging technology, which improves the quality of images by enhancing or restoring the images collected by the optical system; 2) Optical defogging technology, which reduces the impact of haze on imaging quality by transforming the optical imaging system and optimizing the imaging algorithm.

[0004] Among them, the optical defogging technology transforms the optical imaging system, collects multiple images of the same scene with different optical characteristics, obtains the target reflected light information and atmospheric light information from the multiple images and separates them, and then fuses a restored defogging image through an algorithm. Since the atmospheric light formed by scattering particles such as haze scattering sunlight has partial polarization characteristics, polarization optical imaging defogging is suitable for various types of haze weather in various scenes and has a wide range of applications. Israeli scholar Schechner first proposed the use of polarization imaging defogging method. Based on the premise that atmospheric light in foggy days has partial polarization characteristics, a polarization defogging model was constructed. By obtaining two polarization images in the best and worst states, image defogging is achieved. However, this method selects the best and worst images and the atmospheric light intensity at infinity, all relying on manual judgment, with low estimation accuracy, poor real-time performance, and low signal-to-noise ratio of defogging images. Since then, many research teams at home and abroad have conducted in-depth research and optimization on polarization dehazing algorithms, including polarization dehazing algorithms based on dark channel priors, polarization dehazing algorithms based on atmospheric light polarization angle estimation, etc. The algorithms are highly complex, which is not conducive to engineering transformation, and the dehazing effect needs further verification.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In view of the problems in the related art, the present invention proposes a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment to overcome the above technical problems existing in the existing related art.

[0007] The technical solution of the present invention is achieved in this way:

[0008] A real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment comprises the following steps:

[0009] Step S1, pre-acquire multiple polarization images of the scene, and analyze and obtain polarization images in multiple directions, including an intensity image I, a polarization degree image P, and a polarization angle image A of the target;

[0010] Step S2: calibrate the atmospheric light area Rect based on the application scenario and estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , where the application scenarios include fixed scenarios and dynamic scenarios;

[0011] Step S3, based on the atmospheric light polarization degree P A and the atmospheric light polarization angle θ A , estimate the intensity A of atmospheric polarized light p , and estimate the atmospheric light intensity A;

[0012] Step S4, estimating the atmospheric transmittance t again, which includes: based on the atmospheric light intensity A and the atmospheric light intensity A at infinity ∞ The atmospheric transmittance t1 is obtained and the atmospheric transmittance t2 is calculated based on the intensity image I information, and then the atmospheric transmittance t is obtained by weighted fusion. f , and use guided filtering to calculate the atmospheric transmittance t f Perform smooth optimization to obtain the final atmospheric transmittance t;

[0013] Step S5, based on the acquired atmospheric light intensity A and atmospheric transmittance t, estimate the defogging image J, which includes acquiring the mean value A of the atmospheric light intensity A mean Make an estimate and adjust the dynamic range based on the atmospheric light intensity A, expressed as:

[0014]

[0015] Among them, A mean =mean(mean(A)), which is the mean value of atmospheric light intensity A.

[0016] Furthermore, the step of pre-acquiring a plurality of polarization images of a scene comprises the following steps:

[0017] A polarization simultaneous imaging device is used to collect multiple polarization images of a scene, wherein; the polarization simultaneous imaging device includes a split-beam polarization imaging device and a split-focus plane polarization imaging device, wherein;

[0018] The spectroscopic polarization imaging analyzes and obtains polarization images in multiple directions, including: intensity image I, linear polarization degrees Q and U, polarization degree image P, polarization angle image A and 90 degree direction polarization image I (90).

[0019] The split-focus plane polarization imaging analyzes and obtains polarization images in multiple directions, including: intensity image I, linear polarization degrees Q and U, polarization degree image P and polarization angle image A.

[0020] Furthermore, the calibration of the atmospheric light area Rect based on the application scenario includes the following steps:

[0021] Step S201, calibrate the current application scene as a fixed scene, manually select the atmospheric light area Rect, and estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , respectively expressed as:

[0022] P A =mean(mean(P(Rect)))

[0023] θ A =mean(mean(A(Rect)));

[0024] A ∞ =mean(mean(I(Rect)))

[0025] Among them, mean() means to find the average by row or column.

[0026] Furthermore, the calibration of the atmospheric light area Rect based on the application scenario further includes the following steps:

[0027] Step S202, calibrating the current application scene as a dynamic scene and automatically selecting an atmospheric light area, includes the following steps:

[0028] Step S203: normalize the intensity image I in advance to obtain I normal , calculate its semi-inverse image I inverse , expressed as:

[0029] I inverse =max(I normal ,1-I normal );

[0030] Step S204, obtain the possible atmospheric region R1, and the constraint R1 is expressed as:

[0031] R1=|I normal -I inverse |<ε;

[0032] Among them, ε is the difference threshold;

[0033] Step S205, in the atmosphere region R1, taking the point max pos with the maximum gray value as the center, taking DW and DH as the width and height of the rectangular region respectively, to obtain the atmosphere light region R2, which is expressed as:

[0034] maxpos=[x,y]=Find(I(R1)=max(max(I(R1))));

[0035] R2=[x- DW / 2:x+ DW / 2,y- DH / 2:y+ DH / 2];

[0036] Step S206, estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , respectively expressed as:

[0037] P A =mean(mean(P(Rect)))

[0038] θ A =mean(mean(A(Rect))).

[0039] A ∞ =mean(mean(I(Rect)))

[0040] Furthermore, the estimated atmospheric light intensity A is expressed as:

[0041]

[0042]

[0043] Where a1 = cos 2 (θ A ), a2 = sin 2 (θ A ).

[0044] Furthermore, the estimating of the atmospheric transmittance t comprises the following steps:

[0045] Step S401, pre-obtain the atmospheric transmittance t1, expressed as:

[0046]

[0047] Step S402, obtaining the atmospheric transmittance t2, expressed as:

[0048]

[0049] Step S403: Perform weighted fusion to obtain the atmospheric transmittance t f , expressed as:

[0050] t f =w1*t1+w2*t2;

[0051] Among them, w1, w2 are weights;

[0052] Step S404: Use guided filtering to calculate the atmospheric transmittance t f Perform smooth optimization to obtain the final atmospheric transmittance t.

[0053] Furthermore, the estimating the dehazed image J further includes the following steps:

[0054] Step S6: adjusting and enhancing the dehazed image J based on a histogram equalization method.

[0055] Beneficial effects of the present invention:

[0056] 1. The real-time polarization defogging method of the present invention has a main step of linear operation, strong real-time performance, and is convenient for practical engineering applications;

[0057] 2. The atmospheric transmittance estimation method of the present invention adopts a weighted fusion method to estimate the atmospheric transmittance more accurately, thus overcoming the problem that the transmittance cannot be estimated in the absence of sky areas;

[0058] 3. The defogging image estimation method of the present invention uses the mean atmospheric light intensity to replace the original atmospheric light intensity, making the defogging image estimation more stable.

[0059] 4. Use atmospheric light intensity and histogram equalization methods to adjust the dynamic range of the dehazed image to achieve better visual effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 is a flow chart of a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to an embodiment of the present invention;

[0062] Figure 2is a schematic diagram of automatically selecting an atmospheric light region according to a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to an embodiment of the present invention;

[0063] Figure 3 is a schematic diagram of an atmospheric transmittance calculation process of a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to an embodiment of the present invention;

[0064] Figure 4 is a schematic diagram of defogging effect according to a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to an embodiment of the present invention;

[0065] Figure 5 The present invention is a scene schematic diagram of a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to an embodiment of the present invention, wherein (a) is an image before defogging and (b) is an image after being processed by the defogging method. DETAILED DESCRIPTION

[0066] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0067] According to an embodiment of the present invention, a real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment is provided.

[0068] like Figure 1 As shown, the real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to an embodiment of the present invention includes the following steps:

[0069] Step S1, using a polarization simultaneous imaging device to obtain multiple polarization images of the scene, analyzing to obtain polarization images in multiple directions, and calculating to obtain an intensity image I, a polarization degree image P, and a polarization angle image A of the target. The polarization simultaneous imaging device is currently mainly divided into a split-beam polarization imaging device and a split-focus plane polarization imaging device. The calculation steps are as follows:

[0070] In this technical solution, for the spectroscopic polarization imaging, the polarization images at 0, 60 and 120 degrees are analyzed and represented by I(0), I(60) and I(120) respectively, and the intensity image I, the linear polarization degrees Q and U, the polarization degree image P, the polarization angle image A and the 90 degree polarization image I(90) are calculated and represented as:

[0071] and

[0072] In addition, for the focal plane polarization imaging, the present technical solution obtains polarization images of 0 degrees, 45 degrees, 90 degrees and 135 degrees by analysis, which are represented by I(0), I(45), I(90) and I(135) respectively, and calculates the intensity image I, linear polarization degrees Q and U, polarization degree image P and polarization angle image A, which are represented by:

[0073] and

[0074] Step S2: calibrate the atmospheric light area Rect based on the application scenario and estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , where the application scenarios include fixed scenarios and dynamic scenarios;

[0075] In this technical solution, in step S201, the current application scene is calibrated as a fixed scene, and the atmospheric light area Rect is pre-selected manually, which can greatly improve the algorithm operation speed and estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , respectively expressed as:

[0076] P A =mean(mean(P(Rect)))

[0077] θ A =mean(mean(A(Rect)));

[0078] A ∞ =mean(mean(I(Rect)))

[0079] Among them, mean() means to find the average by row or column.

[0080] In addition, in the technical solution, step S202, the current application scene is calibrated as a dynamic scene. It is difficult to meet the requirements by adopting the manual selection method. The semi-reflection method is used to automatically select the atmospheric light area. The basic principle is that due to the scattering effect of atmospheric light, the atmospheric light area in the intensity image is very different from the area corresponding to the semi-reflection image. The atmospheric light area is automatically obtained, including the following steps:

[0081] Step S203: normalize the intensity image I in advance to obtain I normal , calculate its semi-inverse image I inverse , expressed as:

[0082] I inverse =max(I normal ,1-I normal);

[0083] Step S204, obtain the possible atmospheric region R1, and the constraint R1 is expressed as:

[0084] R1=|I normal -I inverse |<ε;

[0085] Among them, ε is the difference threshold;

[0086] Step S205, as Figure 2 As shown, in the atmospheric region R1, the point max pos with the maximum gray value is taken as the center, and DW and DH are respectively used as the width and height of the rectangular region to obtain the atmospheric light region R2, which is expressed as:

[0087] maxpos=[x,y]=Find(I(R1)==max(max(I(R1))));

[0088] R2=[x- DW / 2:x+ DW / 2,y- DH / 2:y+ DH / 2];

[0089] Specifically, in this technical solution, ε is taken as 0.01, and DW and DH are taken as 100.

[0090] Step S206, estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , respectively expressed as:

[0091] P A =mean(mean(P(Rect)))

[0092] θ A =mean(mean(A(Rect))).

[0093] A ∞ =mean(mean(I(Rect)))

[0094] Step S3, based on the atmospheric light polarization degree P A and the atmospheric light polarization angle θ A , estimate the intensity A of atmospheric polarized light p , and estimate the atmospheric light intensity A, expressed as:

[0095]

[0096]

[0097] Where a1 = cos 2 (θ A ), a2 = sin 2 (θ A ).

[0098] Step S4, estimating the atmospheric transmittance t again, which includes: based on the atmospheric light intensity A and the atmospheric light intensity A at infinity ∞ The atmospheric transmittance t1 is obtained and the atmospheric transmittance t2 is calculated based on the intensity image I information, and then the atmospheric transmittance t is obtained by weighted fusion. f , and use guided filtering to calculate the atmospheric transmittance t f Perform smooth optimization to obtain the final atmospheric transmittance t, specifically:

[0099] This technical solution includes the following steps:

[0100] Step S401, pre-obtain the atmospheric transmittance t1, expressed as:

[0101]

[0102] Step S402, obtaining the atmospheric transmittance t2, expressed as:

[0103]

[0104] Step S403: Perform weighted fusion to obtain the atmospheric transmittance t f , expressed as:

[0105] t f =w1*t1+w2*t2;

[0106] Among them, w1, w2 are weights;

[0107] Step S404: Use guided filtering to calculate the atmospheric transmittance t f Perform smooth optimization to obtain the final atmospheric transmittance t.

[0108] In addition, this technical solution uses guided filtering to measure the atmospheric transmittance t f Smooth optimization includes the following steps:

[0109] Let the guide image be I and the input image be t f , then t is:

[0110] t=a k t f +b k ;

[0111] And satisfy F(a k ,b k )Minimum:

[0112]

[0113] Among them, i represents the pixel point, w k represents a square window of size r×r with pixel k as the center (r represents the number of pixels); a k , b k It represents the constant coefficient when the central pixel is k, and eps is the regularization parameter.

[0114] In addition, if Figure 3 As shown, in this technical solution, the weights w1 and w2 are 0.5, r is 60, and the regularization coefficient eps is 10 -6 .

[0115] Step S5, based on the acquired atmospheric light intensity A and atmospheric transmittance t, estimate the defogging image J, which includes acquiring the mean value A of the atmospheric light intensity A mean Make an estimate and adjust the dynamic range based on the atmospheric light intensity A, expressed as:

[0116]

[0117] Among them, A mean =mean(mean(A)), which is the mean value of atmospheric light intensity A.

[0118] Step S6, as Figure 4 As shown, based on the histogram equalization method, the dehazed image J is adjusted and enhanced to improve the visual effect.

[0119] In addition, if Figure 5 As shown, Figure 5 (a) is the image before dehazing. Figure 5 (b) is the image processed by the defogging method of the present invention. From the image effect, the information of the building in the distant fog is restored, and the overall visual effect is good.

[0120] With the help of the above technical solution, the defogging effect is achieved by using the real-time defogging algorithm based on polarization imaging, atmospheric light estimation and dynamic range adjustment. The polarization direction image is obtained by the polarization simultaneous imaging device, and the polarization information analysis method is applied to obtain the intensity image I, polarization degree image P, polarization angle image A, linear polarization degree Q, U, 0 degree polarization direction image I(0), 90 degree polarization direction image I(90), and according to different application scenarios, the atmospheric light area is selected to estimate the light polarization degree PA and the atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞, thereby calculating the atmospheric light intensity A, using the weighted fusion method to estimate the atmospheric transmittance and using guided filtering for smoothing optimization, using the defogging model to estimate the defogging image, and adjusting the dynamic range to obtain the final clear image. The defogging algorithm of the present invention has a simple algorithm and strong real-time performance, can effectively solve the image clarity problem in foggy and hazy weather, and can be directly applied to practical projects, and can have a relatively broad application prospect in the fields of foggy monitoring, navigation, etc.

[0121] In summary, with the help of the above technical solution of the present invention, the following effects can be achieved:

[0122] 1. The real-time polarization defogging method of the present invention has a main step of linear operation, strong real-time performance, and is convenient for practical engineering applications;

[0123] 2. The atmospheric transmittance estimation method of the present invention adopts a weighted fusion method to estimate the atmospheric transmittance more accurately, thus overcoming the problem that the transmittance cannot be estimated in the absence of sky areas;

[0124] 3. The defogging image estimation method of the present invention uses the mean atmospheric light intensity to replace the original atmospheric light intensity, making the defogging image estimation more stable.

[0125] 4. Use atmospheric light intensity and histogram equalization methods to adjust the dynamic range of the dehazed image to achieve better visual effects.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. After considering the disclosure of the specification and the examples, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and the examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

[0127] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment, characterized in that: The following steps are involved: Acquire multiple polarization images of the scene in advance, and analyze and obtain polarization images in multiple directions, including intensity image I, polarization degree image P and polarization angle image A of the target; Calibrate the atmospheric light area Rect based on the application scenario and estimate the atmospheric light polarization P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , where the application scenarios include fixed scenarios and dynamic scenarios; Based on the atmospheric light polarization P A and the atmospheric light polarization angle θ A , estimate the intensity A of atmospheric polarized light p , and estimate the atmospheric light intensity A; Then estimate the atmospheric transmittance t, which includes: based on the atmospheric light intensity A and the atmospheric light intensity A at infinity ∞ The atmospheric transmittance t1 is obtained and the atmospheric transmittance t2 is calculated based on the intensity image I information, and then the atmospheric transmittance t is obtained by weighted fusion. f , and use guided filtering to calculate the atmospheric transmittance t f Perform smooth optimization to obtain the final atmospheric transmittance t; Based on the obtained atmospheric light intensity A and atmospheric transmittance t, the defogging image J is estimated, which includes obtaining the mean value A of the atmospheric light intensity A mean Make an estimate and adjust the dynamic range based on the atmospheric light intensity A, expressed as: Among them, A mean =mean(mean(A)), which is the mean value of the atmospheric light intensity A; The step of calibrating the atmospheric light area Rect based on the application scenario includes the following steps: Calibrate the current application scene as a fixed scene, manually select the atmospheric light area Rect, and estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , respectively expressed as: P A =mean(mean(P(Rect))) θ A =mean(mean(A(Rect))); A ∞ =mean(mean(I(Rect))) Among them, mean() means to find the average by row or column; The step of calibrating the atmospheric light area Rect based on the application scenario further includes the following steps: Calibrate the current application scene as a dynamic scene and automatically select the atmospheric light area, including the following steps: Normalize the intensity image I in advance to get I normal , calculate its semi-inverse image I inverse , expressed as: I inverse =max(I normal ,1-I normal ); Find the possible atmospheric region R1, and the constraint R1 is expressed as: R1=|I normal -I inverse |<ε; Among them, ε is the difference threshold; In the atmospheric region R1, the point with the maximum gray value max pos is taken as the center, and DW and DH are respectively used as the width and height of the rectangular region to obtain the atmospheric light region R2, which is expressed as: maxpos=[x,y]=Find(I(R1)==max(max(I(R1)))); R2=[x-DW / 2:x+DW / 2,y-DH / 2:y+DH / 2]; Then estimate the atmospheric light polarization degree P A 、Atmospheric light polarization angle θ A and the atmospheric light intensity A at infinity ∞ , respectively expressed as: P A =mean(mean(P(Rect))) θ A =mean(mean(A(Rect))) A ∞ =mean(mean(I(Rect)))。 2. The real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to claim 1, characterized in that: The method of pre-acquiring a plurality of polarization images of a scene comprises the following steps: A polarization simultaneous imaging device is used to collect multiple polarization images of a scene, wherein; the polarization simultaneous imaging device includes a split-beam polarization imaging device and a split-focus plane polarization imaging device, wherein; The spectroscopic polarization imaging analyzes and obtains polarization images in multiple directions, including: intensity image I, linear polarization degrees Q and U, polarization degree image P, polarization angle image A and 90 degree direction polarization image I (90); The split-focus plane polarization imaging analyzes and obtains polarization images in multiple directions, including: intensity image I, linear polarization degrees Q and U, polarization degree image P and polarization angle image A.

3. The real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to claim 2, characterized in that: The estimated atmospheric light intensity A is expressed as: Where a1 = cos 2 (θ A ), a2 = sin 2 (θ A ).

4. The real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to claim 3, characterized in that: The method of estimating the atmospheric transmittance t comprises the following steps: The atmospheric transmittance t1 is obtained in advance and is expressed as: The atmospheric transmittance t2 is obtained and expressed as: Perform weighted fusion to obtain the atmospheric transmittance t f , expressed as: t f =w1*t1+w2*t2; Among them, w1, w2 are weights; Then use guided filtering to calculate the atmospheric transmittance t f Perform smooth optimization to obtain the final atmospheric transmittance t.

5. The real-time polarization defogging method for atmospheric light estimation and dynamic range adjustment according to claim 4, characterized in that: The estimating of the dehazed image J further comprises the following steps: Based on the histogram equalization method, the dehazed image J is adjusted and enhanced.

Images