A real-time video defogging method, device and computer storage medium based on dark channel prior
By dividing the foggy day images into sub-block images and constructing transmittance pyramid sequences for fusion, combined with the edge effect filtering processing, the problems of poor real-time and time-space inequality in video defogging in the prior art are solved, and efficient and stable videos are achieved Defog removal effect.
Patent Information
- Application Number
- CN201910979097.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-10-15
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2039-10-15
AI Technical Summary
The existing defog removal method based on dark channel priors has problems such as poor real-time and inconsistent time and space, resulting in poor video defog removal and low imaging quality.
The foggy day image is divided into multiple sub-block images, the atmospheric light value is estimated, and layer-by-layer refinement and fusion is carried out by constructing a transmittance pyramid sequence, and combined with the de-edge effect filtering process, the final transmittance is obtained for de-fogging treatment.
The video defog removal with low computing complexity, high real-time and high imaging quality is achieved, which suppresses flickering jitter and color distortion problems, and improves the space-time consistency of the defog removal effect.
Smart Images

Figure CN112669223B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image and video processing, and in particular to a real-time video defogging method and device based on dark channel prior and a computer storage medium. Background Art
[0002] The visible light imaging system is affected by haze weather, and the captured outdoor images and videos have low visibility, blurred details, and distorted colors, which seriously affects the visual effects of pictures and videos and the performance of image processing systems. At present, the commonly used defogging methods are mainly divided into two categories: one is the foggy image enhancement method based on image processing, such as histogram equalization, local contrast enhancement, gamma correction, etc. This type of method does not consider the reasons for image degradation, and aims to improve the contrast of foggy images and highlight image details, but it is easy to lose certain image information. A type of foggy image restoration method based on physical models, such as defogging methods based on dark channel priors and defogging methods based on scene depth information, has less information loss and a more natural defogging effect. Compared with image defogging algorithms, video defogging algorithms have higher requirements for the real-time and spatiotemporal consistency of their algorithms. However, the existing defogging methods based on dark channel priors have the problems of poor real-time performance and spatiotemporal inconsistency, resulting in low imaging quality. Summary of the invention
[0003] In view of this, the present invention provides a real-time video defogging method, device and computer storage medium based on dark channel prior, which are used to solve the deficiencies of the prior art.
[0004] In order to achieve the above object, the present invention adopts the following technical scheme:
[0005] The embodiment of the present invention provides a real-time video defogging method based on dark channel prior, comprising:
[0006] Divide the foggy image into a plurality of sub-block images, and select pixel points that meet a second preset condition in the sub-block images that meet a first preset condition to estimate the atmospheric light value;
[0007] Constructing a transmittance pyramid sequence according to the dark channel image, and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance;
[0008] Performing edge effect removal filtering on the initial transmittance to obtain the final transmittance;
[0009] Defogging the foggy image according to the atmospheric light value and the final transmittance;
[0010] The “dividing the foggy image into a plurality of sub-block images, and selecting pixels that meet a second preset condition in the sub-block images that meet a first preset condition to estimate the atmospheric light value” specifically includes:
[0011] Divide the foggy image into multiple sub-block images of equal size;
[0012] Calculate the minimum value of the dark channel value of each sub-block image
[0013] According to the first precondition Select the corresponding sub-block image, where the threshold D min and D max according to Adaptive setting of histogram distribution;
[0014] According to the second precondition Select the pixel points in the corresponding sub-block image, where: Indicates that the G, R, and B component values of the pixel points of the selected k-th sub-block image are all less than the threshold L max ;
[0015] For all pixels that meet the conditions, the RGB components are averaged to obtain the atmospheric light value;
[0016] The constructing of a transmittance pyramid sequence according to the dark channel image and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance specifically includes:
[0017] Calculate the channel minimum value of each pixel of the foggy image to obtain the dark channel image;
[0018] Perform minimum downsampling on the dark channel image to construct a transmittance pyramid image;
[0019] The transmittance pyramid images are fused to obtain the initial transmittance image.
[0020] Furthermore, the minimum value of the dark channel value of each sub-block image is calculated Use the following expression: Where k represents the sub-block image number, (x, y) represents the coordinate point of the pixel point, and Ω represents the entire area of the sub-block image;
[0021] Indicates calculating the dark channel value of each pixel of the sub-block image, wherein the dark channel value is the minimum value among the R, G, and B values of the pixel point; Indicates that the minimum value of the dark channel values of all pixels in the sub-block image is taken as the minimum value of the dark channel value of the sub-block image;
[0022] Atmospheric light value A c =[A r A g A b ],in,
[0023] in
[0024] Mean means taking the average value.
[0025] Furthermore, the dark channel image is sampled to the minimum value to construct the transmittance pyramid image t 2↓ , t 4↓ , t 8↓ , which is calculated using the following expression:
[0026]
[0027]
[0028]
[0029] in, Represents the dark channel image I dark (x, y) performs minimum downsampling, Ω2 represents the 2×2 block domain, that is, the minimum value in each 2×2 block domain is taken as the downsampling value, ω(x, y) is the adaptive factor, That is, the value of ω(x, y) is obtained by looking up the display lookup table according to the minimum value in the current 2×2 block domain; t 4↓ It is t 2↓ The result of minimum downsampling, t 8↓ It is t 4↓ The result of minimum downsampling; t 8↓ A value in corresponds to an 8×8 region in the dark channel image, t 4↓ A value in corresponds to a 4×4 region in the dark channel image, t 2↓ A value in corresponds to a 2×2 area in the dark channel image;
[0030] The operation of fusing the transmittance pyramid image to obtain the initial transmittance image is as follows:
[0031] Calculate t 8↓ The absolute difference of the four adjacent points above, below, left and right. If the four sets of absolute differences are all less than the preset threshold Th8, the 8×8 neighborhood of the initial transmittance takes t 8↓ A value corresponding to t; otherwise, the 8×8 neighborhood is divided into four 4×4 neighborhoods; each 4×4 neighborhood corresponds to t 4↓ A point in 4↓ The absolute difference of the four adjacent points above, below, left and right. If the four groups of absolute differences are all less than the preset threshold Th4, the 4×4 neighborhood of the initial transmittance takes t 4↓ A value corresponding to t; otherwise, the 4×4 neighborhood is divided into 4 2×2 neighborhoods; each 2×2 neighborhood corresponds to t 2↓ A point in the.
[0032] Furthermore, the edge effect removal filtering process performs low-pass filtering on the initial transmittance along the horizontal direction and the vertical direction respectively.
[0033] Furthermore, the defogging process for the foggy image according to the atmospheric light value and the final transmittance adopts the following expression: Among them A c is the atmospheric light value, t s (x, y) is the final transmittance, I c (x, y) is the foggy image, J c (x, y) is the image after dehazing.
[0034] The embodiment of the present invention further provides a real-time video defogging device based on dark channel prior, comprising:
[0035] A division estimation module, used for dividing the foggy image into a plurality of sub-block images, and selecting pixel points that meet a second preset condition in the sub-block images that meet a first preset condition to estimate the atmospheric light value;
[0036] Constructing a fusion module, used for constructing a transmittance pyramid sequence according to the dark channel image, and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance;
[0037] A filtering module, used for performing edge effect removal filtering on the initial transmittance to obtain a final transmittance;
[0038] The defogging module is used to perform defogging processing on the foggy image according to the atmospheric light value and the final transmittance.
[0039] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the real-time video defogging method based on dark channel prior as described above is implemented.
[0040] An embodiment of the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the real-time video defogging method based on dark channel prior as described above is implemented.
[0041] The real-time video defogging method based on dark channel prior provided by the present invention can suppress the problems of flickering, jittering, color distortion and the like in the video during the defogging process, and has low computational complexity, good real-time performance and good temporal and spatial consistency, and high imaging quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 A schematic diagram of a process flow of a real-time video defogging method based on dark channel prior provided by an embodiment of the present invention;
[0044] Figure 2 A schematic diagram of the process of estimating atmospheric light values based on foggy images;
[0045] Figure 3 A schematic diagram of the process of obtaining the initial transmittance based on foggy images;
[0046] Figure 4 A schematic diagram of dividing an 8×8 neighborhood into four 4×4 neighborhoods;
[0047] Figure 5 A schematic structural diagram of a real-time video defogging device based on dark channel prior provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] like Figure 1 As shown, an embodiment of the present invention provides a real-time video defogging method based on dark channel prior, comprising:
[0050] S101, dividing a foggy image into a plurality of sub-block images, and selecting pixels that meet a second preset condition from among the sub-block images that meet a first preset condition to estimate the atmospheric light value. Figure 2 As shown, dividing the foggy image into a plurality of sub-block images, and selecting the pixel points that meet the second preset condition in the sub-block images that meet the first preset condition to estimate the atmospheric light value specifically includes:
[0051] S1011. Divide the foggy image into a plurality of sub-block images of equal size.
[0052] Let I c (x, y) is a foggy image with a size of M×N. The foggy image is divided into K sub-block image sequences of equal size. The size of the sub-block image is M s ×N s .
[0053] S1012, calculating the minimum value of the dark channel value of each sub-block image
[0054] As shown in expression (1), the dark channel value of each pixel of the sub-block image is first calculated, where the dark channel value is the minimum value among the R, G, and B values of the pixel. Then the minimum value of the dark channel value of the entire sub-block image is calculated, where Ω represents the entire M value of the sub-block image. s ×N s area.
[0055]
[0056] Calculate the minimum value of the dark channel value of each sub-block image Formula (1) is used: where k represents the sub-block image number, (x, y) represents the coordinate point of the pixel point, and Ω represents the entire M of the sub-block image. s ×N s area; Indicates calculating the dark channel value of each pixel of the sub-block image, wherein the dark channel value is the minimum value among the R, G, and B values of the pixel point; It means taking the minimum value of the dark channel values of all pixels in the sub-block image as the minimum value of the dark channel value of the sub-block image.
[0057] S1013: According to the first preset condition Select the corresponding sub-block image, where the threshold D min and D max according to The histogram distribution of is adaptively set.
[0058] S1014: According to the second preset condition Select the pixel points in the corresponding sub-block image, where: Indicates that the G, R, and B component values of the pixel points of the selected k-th sub-block image are all less than the threshold L max .
[0059] S1015. Take the average value of the RGB components of all pixels that meet the conditions to obtain the atmospheric light value.
[0060] Atmospheric light value A c =[A r A g A b ],in,
[0061] in
[0062] Mean means taking the average value.
[0063] As shown in formula (2), according to the first preset condition Select The corresponding sub-block images with relatively large values are selected, and the sub-block images that meet the second preset condition The R, G, and B components of all pixels are averaged to obtain the atmospheric light value A c The threshold D min and D max is based on The histogram distribution of is adaptively set.
[0064] S102 , constructing a transmittance pyramid sequence according to the dark channel image, and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance.
[0065] like Figure 3 As shown, constructing a transmittance pyramid sequence based on a dark channel image and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance specifically includes:
[0066] 1021. Calculate the channel minimum value of each pixel of the foggy image to obtain a dark channel image.
[0067] The channel minimum value of each pixel in the foggy image is calculated using formula (3):
[0068]
[0069] 1022. Perform minimum downsampling on the dark channel image to construct a transmittance pyramid image.
[0070] Perform minimum downsampling on the dark channel image to construct the transmittance pyramid image t 2↓ , t 4↓ , t 8↓ ;
[0071] where t 2↓ , t 4↓ , t 8↓ The calculation formula (4) is as follows:
[0072]
[0073] in Represents the dark channel image I dark (x, y) performs minimum downsampling, Ω2 represents the 2×2 block domain, that is, the minimum value in each 2×2 block domain is taken as the downsampling value. ω(x, y) is the adaptive factor, That is, the value of ω(x, y) is obtained by looking up the table based on the minimum value in the current 2×2 block domain. 4↓ It is t 2↓ The result of minimum downsampling, t 8↓ It is t 4↓ The result of minimum downsampling. Therefore, t 8↓ A value in corresponds to an 8×8 region in the dark channel image, t 4↓ A value in corresponds to a 4×4 region in the dark channel image, t 2↓ A value in corresponds to a 2×2 region in the dark channel image.
[0074] 1023. Fuse the transmittance pyramid images to obtain an initial transmittance image.
[0075] The specific steps of fusing the transmittance pyramid images to obtain the initial transmittance image are as follows:
[0076] Assume that the initial transmittance image is t0, as shown in formula (5), calculate t 8↓ The absolute difference between (i, j) and its four adjacent points, above, below, left and right. For boundary points, the absolute difference is calculated.
[0077]
[0078] If the absolute differences of the four groups are all less than the preset threshold, that is, Then the 8×8 neighborhood of the initial transmittance is t 8↓ A corresponding value in, that is, t0(8i-7:8i,8j-7:8j)=t 8↓ Otherwise, the 8×8 neighborhood is divided into four 4×4 neighborhoods, and each 4×4 neighborhood corresponds to t 4↓ Similarly, if Figure 4 As shown, t 4↓ Take point (2i-1, 2j-1) as an example, and calculate t 4↓ (2i-1, 2j-1) is the absolute difference between its four adjacent upper, lower, left and right points, as shown in formula (6).
[0079]
[0080] Similarly, if the absolute differences of the four groups are all less than the preset threshold, that is, Then the 4×4 neighborhood of the initial transmittance is t 4↓ A corresponding value in, that is, t0(8i-7:8i-4,8j-7:8j-4)=t 4↓ (2i-1, 2j-1). Otherwise, the 4×4 neighborhood is divided into 4 2×2 neighborhoods, and each 2×2 neighborhood corresponds to t 2↓ At this time, the perspective rate of the four 2×2 neighborhoods is t2↓ The four values corresponding to
[0081]
[0082] As described above, this is a transmittance fusion method for an 8×8 neighborhood, and this method is used to fuse the entire initial transmittance t0.
[0083] S103, performing edge effect removal filtering on the initial transmittance to obtain a final transmittance.
[0084] The edge effect removal filtering process performs low-pass filtering on the initial transmittance in the horizontal direction and the vertical direction respectively. Specifically, the initial transmittance can be optimized by using guided filtering and median filtering to achieve the purpose of smoothing the image and maintaining the edge.
[0085] S104: Defogging the foggy image according to the atmospheric light value and the final transmittance.
[0086] The defogging process for the foggy image according to the atmospheric light value and the final transmittance adopts the following expression: Among them A c is the atmospheric light value, t s (x, y) is the final transmittance, I c (x, y) is the foggy image, J c (x, y) is the image after dehazing.
[0087] like Figure 5 As shown, an embodiment of the present invention provides a real-time video defogging device based on dark channel prior, comprising:
[0088] A division estimation module, used for dividing the foggy image into a plurality of sub-block images, and selecting pixel points that meet a second preset condition in the sub-block images that meet a first preset condition to estimate the atmospheric light value;
[0089] Constructing a fusion module, used for constructing a transmittance pyramid sequence according to the dark channel image, and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance;
[0090] A filtering module, used for performing edge effect removal filtering on the initial transmittance to obtain a final transmittance;
[0091] The defogging module is used to perform defogging processing on the foggy image according to the atmospheric light value and the final transmittance.
[0092] It should be noted that: the real-time video defogging device based on dark channel prior provided in the above embodiment only uses the division of the above program modules as an example when performing defogging. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the real-time video defogging device based on dark channel prior provided in the above embodiment and the real-time video defogging method embodiment based on dark channel prior belong to the same concept. The specific implementation process is detailed in the method embodiment. The beneficial effects are the same as the method embodiment, which will not be repeated here.
[0093] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the real-time video defogging method based on dark channel prior as described above is implemented. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like.
[0094] An embodiment of the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the real-time video defogging method based on dark channel prior as described above is implemented.
[0095] In the several embodiments provided by the present invention, it should be understood that the disclosed methods and intelligent devices can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0096] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0097] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A real-time video defogging method based on dark channel prior, characterized in that: include: Divide the foggy image into a plurality of sub-block images, and select pixel points that meet a second preset condition in the sub-block images that meet a first preset condition to estimate the atmospheric light value; Constructing a transmittance pyramid sequence according to the dark channel image, and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance; Performing edge effect removal filtering on the initial transmittance to obtain the final transmittance; Defogging the foggy image according to the atmospheric light value and the final transmittance; The “dividing the foggy image into a plurality of sub-block images, and selecting pixels that meet a second preset condition in the sub-block images that meet a first preset condition to estimate the atmospheric light value” specifically includes: Divide the foggy image into multiple sub-block images of equal size; Calculate the minimum value of the dark channel value of each sub-block image According to the first precondition Select the corresponding sub-block image, where the threshold D min and D max according to Adaptive setting of histogram distribution; According to the second precondition Select the pixel points in the corresponding sub-block image, where: Indicates that the G, R, and B component values of the pixel points of the selected k-th sub-block image are all less than the threshold L max ; For all pixels that meet the conditions, the RGB components are averaged to obtain the atmospheric light value; The constructing of a transmittance pyramid sequence according to the dark channel image and fusing the transmittance pyramid sequence in a layer-by-layer refinement manner to obtain an initial transmittance specifically includes: Calculate the channel minimum value of each pixel of the foggy image to obtain the dark channel image; Perform minimum downsampling on the dark channel image to construct a transmittance pyramid image; The transmittance pyramid images are fused to obtain the initial transmittance image.
2. The method according to claim 1, characterized in that Calculate the minimum value of the dark channel value of each sub-block image Use the following expression: Where k represents the sub-block image number, (x, y) represents the coordinate point of the pixel point, and Ω represents the entire area of the sub-block image; Indicates calculating the dark channel value of each pixel of the sub-block image, wherein the dark channel value is the minimum value among the R, G, and B values of the pixel point; Indicates that the minimum value of the dark channel values of all pixels in the sub-block image is taken as the minimum value of the dark channel value of the sub-block image; Atmospheric light value A c =[A r A g A b ],in, in Mean means taking the average value.
3. The method according to claim 1, characterized in that Perform minimum downsampling on the dark channel image to construct the transmittance pyramid image t 2↓ ,t 4↓ ,t 8↓ , which is calculated using the following expression: in, Represents the dark channel image I dark (x, y) performs minimum downsampling, Ω2 represents the 2×2 block domain, that is, the minimum value in each 2×2 block domain is taken as the downsampling value, ω(x, y) is the adaptive factor, That is, the value of ω(x,y) is obtained by looking up the display lookup table according to the minimum value in the current 2×2 block domain; t 4↓ It is t 2↓ The result of minimum downsampling, t 8↓ It is t 4↓ The result of minimum downsampling; t 8↓ A value in corresponds to an 8×8 region in the dark channel image, t 4↓ A value in corresponds to a 4×4 region in the dark channel image, t 2↓ A value in corresponds to a 2×2 area in the dark channel image; The operation of fusing the transmittance pyramid image to obtain the initial transmittance image is as follows: Calculate t 8↓ The absolute difference of the four adjacent points above, below, left and right. If the four sets of absolute differences are all less than the preset threshold Th8, the 8×8 neighborhood of the initial transmittance takes t 8↓ A value corresponding to t; otherwise, the 8×8 neighborhood is divided into four 4×4 neighborhoods; each 4×4 neighborhood corresponds to t 4↓ A point in 4↓ The absolute difference of the four adjacent points above, below, left and right. If the four groups of absolute differences are all less than the preset threshold Th4, the 4×4 neighborhood of the initial transmittance takes t 4↓ A value corresponding to t; otherwise, the 4×4 neighborhood is divided into 4 2×2 neighborhoods; each 2×2 neighborhood corresponds to t 2↓ A point in the.
4. The method according to claim 1, characterized in that The edge effect removal filtering process performs low-pass filtering on the initial transmittance in the horizontal direction and the vertical direction respectively.
5. The method according to claim 1, characterized in that The defogging process for the foggy image according to the atmospheric light value and the final transmittance adopts the following expression: Among them A c is the atmospheric light value, t s (x, y) is the final transmittance, I c (x, y) is the foggy image, J c (x,y) is the image after dehazing.
6. A real-time video defogging device based on dark channel prior, characterized in that: The device is used to implement the real-time video defogging method based on dark channel prior as described in any one of claims 1 to 4, and the device comprises: A division estimation module, used for dividing the foggy image into a plurality of sub-block images, and selecting pixel points that meet a second preset condition in the sub-block images that meet a first preset condition to estimate the atmospheric light value; A fusion module is constructed to construct a transmittance pyramid sequence according to the dark channel image, and the transmittance pyramid sequence is fused in a layer-by-layer refinement manner to obtain an initial transmittance; A filtering module, used for performing edge effect removal filtering on the initial transmittance to obtain a final transmittance; The defogging module is used to perform defogging processing on the foggy image according to the atmospheric light value and the final transmittance.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the real-time video defogging method based on dark channel prior as claimed in any one of claims 1 to 4 is implemented.
8. A terminal device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the real-time video defogging method based on dark channel prior as described in any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Image dehazing method and device and mobile terminal
CN104166968A
Multi-scale transmission image fusion method for image pyramid model
CN108550130A
