A background light estimation method, an underwater image restoration method and an electronic device

Abstract

The application discloses a background light estimation method in the technical field of image processing, an underwater image restoration method and electronic equipment, and aims to solve the problem of poor accuracy caused by unified estimation of the background light of three channels of RGB in the prior art. The application determines the decay rates of three color channels through the total pixel value size of the three color channels of the original input image, screens the pixels of the three color channels according to the pixel values of the neighborhood pixels, calculates the background light estimation values of the three color channels through different methods respectively, and then restores the three channels respectively. The application can be used for underwater image processing, can realize one-to-one accurate estimation of the background light of three channels of RGB, and improves the underwater image restoration quality.

Description

TECHNICAL FIELD

[0001] The present application relates to a background light estimation method, an underwater image restoration method and an electronic device, and belongs to the technical field of image processing. BACKGROUND

[0002] The existing image processing technology has poor algorithm adaptability when estimating the background light of images with large differences in RGB (red, green and blue) three-color light attenuation degree, such as underwater images, haze images and sandstorm images. The background light of the three channels is uniformly estimated, resulting in poor accuracy of the output background light estimation. SUMMARY

[0003] The present application aims to overcome the shortcomings of the prior art and provide a background light estimation method, an underwater image restoration method and an electronic device with strong algorithm adaptability, which can accurately estimate the background light and improve the quality of underwater image restoration.

[0004] To achieve the above-mentioned purpose, the present application is realized by the following technical scheme:

[0005] In a first aspect, the present application provides a background light estimation method, comprising:

[0006] The total pixel values of the RGB three-color channels of the original input image are calculated respectively, and the first color channel, the second color channel and the third color channel are determined according to the total pixel value size; the attenuation speed of the first color channel, the second color channel and the third color channel decreases in turn;

[0007] For the first color channel, the second color channel and the third color channel, the corresponding candidate pixel group is screened according to the pixel value of the neighborhood pixels of each color channel, and the background light estimation value of the corresponding color channel is calculated according to the neighborhood pixel value of the candidate pixel group;

[0008] The background light estimation result is obtained by combining the background light B1 estimation value of the first color channel, the background light B2 estimation value of the second color channel and the background light B3 estimation value of the third color channel.

[0009] In a first aspect, further, the method for calculating the background light B1 estimation value of the first color channel comprises:

[0010] Gamma correction is performed on the pixel value of the first color channel in all pixels;

[0011] The first 1% of pixels are selected as the first candidate pixel group according to the pixel value size of the first color channel after gamma correction;

[0012] The pixel values of the remaining two color channels in the neighborhood pixels of each pixel in the first candidate pixel group are calculated by interpolation method to obtain a plurality of first estimation values.

[0013] Select the maximum first estimate from the plurality of first estimates as the background light B1 estimate of the first color channel.

[0014] In a first aspect, further, the first estimate is calculated using the following formula

[0015]

[0016] In a first aspect, further, the method for calculating the background light B2 estimate of the second color channel comprises:

[0017] Gamma-correcting the pixel values of the second color channel;

[0018] Selecting the top 1% of pixels according to the pixel values of the gamma-corrected second color channel as the second candidate pixel group;

[0019] Calculating the variance of the second color channel of the four-neighbor pixels of each pixel in the second candidate pixel group;

[0020] Selecting the pixel value of the second color channel of the pixel with the smallest variance from the second candidate pixel group as the background light B2 estimate.

[0021] In a first aspect, further, in the process of gamma-correcting the pixel values of the second color channel, the gamma value γ is calculated using the following formula:

[0022]

[0023] wherein κ is a constant, is the variance of the pixel values of the second color channel after gamma correction.

[0024] In a first aspect, further, the method for calculating the background light B3 estimate of the third color channel comprises:

[0025] Selecting the top 1% of pixels according to the pixel values of the third color channel as the third candidate pixel group;

[0026] Calculating the Q value of each pixel in the third candidate pixel group:

[0027]

[0028] wherein μ b is the mean of the pixel values of the third color channel of the eight-neighbor pixels, is the variance of the pixel values of the third color channel of the eight-neighbor pixels, and λ is a weighting factor; and the Q value is the pseudo-variance of each pixel in the third candidate pixel group.

[0029] The pixel corresponding to the minimum Q value is selected, and the pixel value of the third color channel of the pixel is taken as the background light B2 estimation value.

[0030] In a first aspect, further, the method for calculating the total pixel value of the three color channels comprises:

[0031]

[0032] In the formula, k is one of the color channels of RGB, s k is the total pixel value of the color channel k; m and n respectively represent the pixel width and the pixel height of the original input image; I nor k (i,j) represents the pixel value of the corresponding color channel k of the pixel at the pixel coordinate (i,j).

[0033] In a first aspect, further, before the total pixel value of the RGB three color channels of the original input image is calculated, the pixel value of the original input image is normalized.

[0034] In a second aspect, the application further provides an underwater image restoration method, comprising:

[0035] The background light estimation result A c of the original input image is obtained by using the background light estimation method according to any one of the first aspect.

[0036] According to the background light estimation result, a corresponding transmission map is calculated, and the expression is as follows:

[0037]

[0038] In the formula, I c (y) is the original low-quality underwater optical image collected, Ω(x) is the local area of x, t c (x) is the estimated transmittance, A c is the background light estimation result, and x is the pixel position.

[0039] The transmittance t c (x) is refined by using guided filter refinement to obtain a guided filter refined transmittance

[0040] The restored underwater image is obtained by combining the background light estimation result and the guided filter refined transmittance, and the expression is as follows:

[0041]

[0042] In the formula, t is the guided filter refined transmittance, and Ic (x) represents the original acquired image, J c (x) is the restored underwater image.

[0043] Thirdly, an electronic device includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, performs the steps of the method as described in either the first or second aspect.

[0044] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0045] This invention considers the differences in attenuation levels of the three RGB color channels when calculating the background light estimation values ​​of the first, third, and second color channels, thereby improving the adaptability of the algorithm and providing more accurate background light estimation results. Attached Figure Description

[0046] Figure 1 This is a flowchart of a background light estimation method provided in Embodiment 1 of the present invention;

[0047] Figure 2 This is a schematic diagram of the color channels used by each neighboring pixel in the first candidate pixel group when calculating the background light estimate B1 of the R channel in an embodiment of the present invention;

[0048] Figure 3 This is an original input image acquired in an embodiment of the present invention;

[0049] Figure 4 The background light estimation method provided in Embodiment 1 of this invention is used to... Figure 3 The resulting visualization of the background light estimation after processing;

[0050] Figure 5 This is another original input image acquired in an embodiment of the present invention;

[0051] Figure 6 The background light estimation method provided in Embodiment 1 of this invention is used to... Figure 5 The resulting visualization of the background light estimation after processing;

[0052] Figure 7 The underwater image restoration method provided in Embodiment 2 of this invention is used to... Figure 5 The restored image obtained after restoration;

[0053] Figure 8 This is a flowchart of an underwater image restoration method provided in Embodiment 2 of the present invention. Detailed Implementation

[0054] The technical solutions of the present application will be described in detail below with reference to the drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solutions of the present application, and are not limitations of the technical solutions of the present application. In the case of no conflict, the technical features in the embodiments and the embodiments can be combined with each other.

[0055] Embodiment one

[0056] Reference Figure 1 According to the method steps shown, the embodiment provides a background light estimation method, which can be applied to underwater image processing captured in turbid or deep water; Figure 3 The original image of an underwater image is shown, which is taken as the original input image of the embodiment of the present application, and the specific process is as follows:

[0057] Step one: calculate the total pixel value of the RGB three color channels of the original input image respectively, and determine the first color channel, the second color channel and the third color channel according to the total pixel value; the decay rate of the first color channel, the second color channel and the third color channel decreases.

[0058] First, the original input image is normalized to calculate the R, G and B three channels; for the convenience of those skilled in the art to understand, in the embodiment of the present application, the R channel represents the red channel, the G channel represents the green channel, and the B channel represents the blue channel. The total pixel value of the R, G and B three color channels of the underwater image is calculated, and the decay degree of the R, G and B three color channels is judged according to the obtained total pixel value. The calculation method used in the embodiment is as follows:

[0059]

[0060] In the formula, k is one of the color channels in RGB, s k is the total pixel value of the color channel represented by k; m and n respectively represent the pixel width and pixel height of the original input image; I nor k (i,j) represents the pixel value of the pixel at pixel coordinate (i,j) in the color channel represented by k.

[0061] It should be noted that the greater the s k value, the slower the decay of the color channel.

[0062] In a conventional underwater environment, the total pixel value of the RGB three color channels is determined according to the calculated total pixel value, and the size s B > s G > s R , it can be known that the first color channel of the underwater image is the R channel, the second color channel is the G channel, and the third color channel is the B channel.

[0063] Step two: for the first color channel, the second color channel and the third color channel, respectively, according to the pixel value of the neighborhood pixels of each color channel, screen the corresponding candidate pixel group, and calculate the background light estimate value of the corresponding color channel according to the neighborhood pixel value of the candidate pixel group.

[0064] The method for calculating the background light estimate value B1 of the R channel according to the neighborhood pixel value of the candidate pixel group comprises:

[0065] In order to compensate for the characteristics of human vision when outputting the restored underwater image using the background light estimate value B1, the R channel pixel value of all pixels of the underwater image is subjected to gamma correction.

[0066] All pixels are arranged in descending order of the R channel pixel value after gamma correction, and the top 1% of pixels are selected as the first candidate pixel group.

[0067] The neighborhood pixels of each pixel in the first candidate pixel group are calculated using an interpolation method to obtain a plurality of first estimates corresponding to each pixel in the first candidate pixel group, and the first estimate of each pixel is calculated according to Figure 2 The calculation formula of the first estimate of each pixel comprises:

[0068]

[0069] The maximum first estimate is selected from the obtained plurality of first estimates as the background light B1 estimate value. In this embodiment, the background light B1 estimate value is 52.

[0070] The method for calculating the background light B2 estimate value of the G channel according to the neighborhood pixel value of the candidate pixel group comprises:

[0071] In order to compensate for the characteristics of human vision when outputting the restored underwater image using the background light B2 estimate value, the G channel pixel value of all pixels of the underwater image is subjected to gamma correction, and the calculation formula of the gamma coefficient γ used is as follows:

[0072]

[0073] In the formula, κ is a constant, and κ is used as a coefficient for adjusting the background light B2 estimate value by adjusting the gamma coefficient γ. In this embodiment, κ takes a value of 0.1, is the variance of the pixel value of the second color channel after gamma correction, i.e. the G channel in this embodiment;

[0074] The final gamma correction formula is obtained by combining the gamma correction formula of the G channel pixel value G out = G γ

[0075] ​

[0076] The G channel pixel value after gamma correction is obtained by using the above formula, and the first 1% of pixels are selected as the second candidate pixel group according to the arrangement from large to small;

[0077] The variance of the G channel pixel value of the four neighborhood pixels of each pixel in the second candidate pixel group is calculated respectively, and the pixel corresponding to the smallest variance is selected, and the G channel pixel value of the pixel is the background light B2 estimate value; after calculation, the background light B2 estimate value in the embodiment is 185;

[0078] The method for calculating the background light estimate value B3 of the B channel according to the neighborhood pixel value of the candidate pixel group comprises:

[0079] All pixels are arranged from large to small according to the B channel pixel value, and the first 1% of pixels are selected as the third candidate pixel group;

[0080] The Q value of each pixel in the third candidate pixel group is calculated:

[0081]

[0082] In the formula, μ b is the average value of the pixel value of the third color channel of the eight-domain pixel, i.e. the B channel pixel value in the embodiment, is the variance of the pixel value of the third color channel of the eight-domain pixel, i.e. the B channel pixel value in the embodiment, and λ is a weighting factor, and the Q value is the pseudo-variance of each pixel in the third candidate pixel group;

[0083] After the Q value of each pixel in the third candidate pixel group is calculated, the pixel corresponding to the minimum Q value is selected, and the B channel pixel value of the pixel is the background light B3 estimate value; after calculation, the background light B3 estimate value in the embodiment is 239.

[0084] Step three: obtaining the background light estimate result by combining the background light B1 estimate value of the first color channel, the background light B2 estimate value of the second color channel and the background light B3 estimate value of the third color channel.

[0085] By comparison Figure 3 and Figure 4 , it can be known that Figure 4 the background light estimate result (R: 52, G: 185, B: 239) is accurately visualized and output. Figure 3

[0086] ​In this embodiment, the first color channel is determined to be the R channel, the second color channel to be the G channel, and the third color channel to be the B channel based on the assumption of a normal underwater environment. In special underwater environments, the first color channel, the third color channel, and the second color channel should be determined according to the total pixel value.

[0087] In this embodiment, to conform to visual habits, classical gamma correction is performed on the R channel pixel values ​​of all pixels in the underwater image, and the gamma coefficient used is 0.4545, i.e.

[0088] R out =R 0.4545

[0089] In the formula, R is the R-channel pixel value of the underwater image before gamma correction. out The values ​​are the R-channel pixel values ​​after classic gamma correction; however, those skilled in the art may use other gamma coefficients depending on the usage environment and desired visual effect.

[0090] The background light estimation method provided in this embodiment solves the problem of poor accuracy in background light estimation results caused by the unified estimation of background light for the three RGB channels in existing technologies. Based on the differences in the attenuation levels of the three RGB color channels underwater, this embodiment provides different algorithms for calculating the background light estimation values ​​of the first color channel (R channel), the second color channel (B channel), and the third color channel (G channel), improving the adaptability of the background light estimation method and providing more accurate background light estimation results. Applying the background light estimation method provided in this embodiment to underwater image processing can improve the processing effect of underwater images captured in turbid or deep waters. Furthermore, the background light estimation method provided in this embodiment can be widely applied to scenarios such as deep-sea fishing, underwater archaeology, diving rescue, hydrogeological exploration, underwater biota research, submarine cable laying, and intelligence gathering.

[0091] Example 2

[0092] refer to Figure 8 The method steps shown in this embodiment provide an underwater image restoration method, including:

[0093] The original input image is processed using the background light estimation method described in Example 1, such as... Figure 5 The image shown is an underwater image acquired in an embodiment of the present invention, using the background light estimation method described in Embodiment 1. Figure 5 Process and obtain Figure 6 Background light estimation result A c In the formula, c = {R, G, B} represents three color channels. In this embodiment, the background light estimation result of the underwater image is (R: 125, G: 169, B: 173).

[0094] The corresponding transmission map is calculated based on the estimated background light values ​​B1, B2, and B3 of the three color channels, and its expression is as follows:

[0095]

[0096] In the formula: I c (y) represents the original low-quality underwater optical image acquired, Ω(x) represents the local region of x, and t c (x) represents the estimated transmittance, A c This represents the background light estimation result, where x is the pixel position;

[0097] The transmittance t is refined using guided filtering. c (x) is refined to obtain the transmittance after guided filtering.

[0098] See Figure 7 The restored underwater image is obtained by combining the background light estimation results and the transmittance after guide filtering and thinning. Its expression is:

[0099]

[0100] In the formula I represents the transmittance after refinement by the guided filter. c (x) represents the acquired original input image, J c (x) is the restored underwater image.

[0101] contrast Figure 5 , Figure 6 and Figure 7 It is not difficult to see that Figure 6 Accurately visualized the output Figure 5 Background light estimation results, based on Figure 6 Background light estimation results, Figure 7 The system outputs a reconstructed underwater image with high fidelity.

[0102] Example 3

[0103] This embodiment provides a background light estimation method, which is an improvement on the scheme of Embodiment 1. Specifically, before calculating the total pixel values ​​of the RGB three color channels of the original input image, the pixel values ​​of the original input image are normalized, that is, the pixel values ​​are normalized from [0,255] to the range of [0,1]. The background light estimation method provided by this embodiment is more convenient for data processing than that of Embodiment 1.

[0104] For other steps in this embodiment, please refer to Embodiment 1, which will not be repeated here.

[0105] Example 4

[0106] The embodiment provides a background light estimation method, which can be applied to foggy and hazy image processing, is based on the difference in the penetration ability of different wavelengths of visible light in a complex gas environment and an underwater environment, and the first color channel, the second color channel and the third color channel are determined according to the total pixel value s B 、s G 、s R The size is redetermined, and the same technical solution as in the first embodiment or the third embodiment is not repeated here.

[0107] The background light estimation method provided in the embodiment is based on the difference in the attenuation degree of the three color channels of underwater RGB, provides different algorithms when calculating the background light estimation value, provides more accurate background light estimation results, can improve the hazy image processing effect, and can be widely applied to aerial photography, land resource survey, disaster prevention and reduction, crop yield estimation, traffic diversion, extraterrestrial planet imaging and the like.

[0108] Embodiment five

[0109] The embodiment provides a background light estimation method, which can be applied to sandstorm and volcanic ash image processing, is based on the difference in the penetration ability of different wavelengths of visible light in an atmosphere environment densely covered with small rock particles and an underwater environment, and the first color channel, the second color channel and the third color channel are determined according to the total pixel value s B 、s G 、s R The size is redetermined, and the same technical solution as in the first embodiment or the third embodiment is not repeated here.

[0110] The background light estimation method provided in the embodiment is based on the difference in the attenuation degree of the three color channels of underwater RGB, provides different algorithms when calculating the background light estimation value, provides more accurate background light estimation results, can improve the dust storm image processing effect, and can be widely applied to aerial photography, land resource survey, disaster prevention and reduction, crop yield estimation, traffic diversion, emergency rescue and the like.

[0111] Embodiment six

[0112] The embodiment provides an electronic device, which comprises a processor and a memory connected with the processor, and the memory stores a computer program. When the computer program is executed by the processor, the steps of the method in any one of the first to sixth embodiments are executed.

[0113] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0114] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0115] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0117] The above description is only preferred embodiments of the application. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made without departing from the technical principles of the application, and these improvements and modifications should also be considered as falling within the scope of the application.

Claims

1. A method of ambient light estimation, the method comprising: The method comprises the following steps: The total pixel values of the RGB three color channels of the original input image are calculated respectively, and the first color channel, the second color channel and the third color channel are determined according to the total pixel value size; The decay rates of the first color channel, the second color channel and the third color channel decrease in turn; For the first color channel, the second color channel and the third color channel, the corresponding candidate pixel groups are screened according to the pixel values of the neighborhood pixels of each color channel, and the background light estimates of the corresponding color channels are calculated according to the neighborhood pixel values of the candidate pixel groups; The background light estimates of the first color channel, the second color channel and the third color channel are combined to obtain the background light estimation result; The method for calculating the background light B1 estimate of the first color channel comprises the following steps: Gamma correction is performed on the pixel values of the first color channel in all pixels; The first 1% of pixels are selected as the first candidate pixel group according to the pixel value size of the first color channel after gamma correction; The pixel values of the other two color channels in the neighborhood pixels of each pixel in the first candidate pixel group are calculated by interpolation method to obtain a plurality of first estimates; The maximum first estimate is selected from the plurality of first estimates as the background light B1 estimate of the first color channel; The method for calculating the background light B2 estimate of the second color channel comprises the following steps: Gamma correction is performed on the pixel values of the second color channel; The first 1% of pixels are selected as the second candidate pixel group according to the pixel value size of the second color channel after gamma correction; The variance of the second color channel of the four neighborhood pixels of each pixel in the second candidate pixel group is calculated; The pixel value of the second color channel of the pixel with the minimum variance is selected from the second candidate pixel group as the background light B2 estimate; The method for calculating the background light B3 estimate of the third color channel comprises the following steps: The first 1% of pixels are selected as the third candidate pixel group according to the pixel value size of the third color channel; The pseudo-variance Q value of each pixel in the third candidate pixel group is calculated; The pixel corresponding to the minimum Q value is selected, and the pixel value of the third color channel of the pixel is the background light B3 estimate.

2. The background light estimation method of claim 1, wherein, The first estimate is calculated using the following formula 3. The background light estimation method of claim 1, wherein, In the process of performing gamma correction on the pixel value of the second color channel, the gamma value γ is calculated by the following formula: where k is a constant, is the variance of the pixel values of the second color channel after gamma correction.

4. The background light estimation method of claim 1, wherein, The Q value is calculated by the following formula: wherein μ b is the mean of the pixel values of the third color channel of the eight-neighborhood pixels, is the variance of the pixel values of the third color channel of the eight-neighborhood pixels, and λ is a weighting factor.

5. The ambient light estimation method of claim 1, wherein, The method for calculating the total pixel value of the three color channels: where k is one of the color channels in RGB, s k is the total pixel value of color channel k; m and n represent the pixel width and pixel height of the original input image, respectively; I nor k (i,j) represents the pixel value of the corresponding color channel k of the pixel at pixel coordinate (i,j).

6. The background light estimation method of claim 1, wherein, Before calculating the total pixel value of the RGB three color channels of the original input image, the pixel value of the original input image is normalized.

7. An underwater image restoration method characterized by comprising: The method comprises the following steps: The background light estimation method according to any one of claims 1 to 6 is used to obtain a background light estimation result A of the original input image c where c = {R, G, B} represents three color channels. The transmission map corresponding to the background light estimation result is calculated, and its expression is as follows: In the formula I c (y) is the original low-quality underwater optical image collected, Ω(x) is the local area of x, t c (x) is the estimated transmittance, A c is the background light estimation result, and x is the pixel position The transmittance t is refined using a guided filter c (x) is refined to obtain a guided filter refined transmittance The restored underwater image is obtained by combining the background light estimation result and the refined transmittance of the guided filter, and its expression is as follows: In the formula is the transmittance after guided filtering, I c (x) is the original collected image, J c (x) is the recovered underwater image.

8. An electronic device, comprising: The method comprises a processor and a memory connected to the processor, and the memory stores a computer program, which, when executed by the processor, performs the steps of the method according to any one of claims 1-7.

Images