An Image Restoration Method Based on an Underwater Non-Uniform Incident Light Model

Abstract

The present invention relates to an image restoration method based on an underwater non-uniform incident light model, belonging to the field of image restoration. It includes the following steps: Step S1, perform wavelet decomposition on the underwater image to obtain the lowest-frequency subband and each high-frequency subband; Step S2, based on the underwater non-uniform incident light model, perform scattering suppression on the lowest-frequency subband to obtain the processed lowest-frequency subband; Step S3, perform noise suppression and detail enhancement processing on each high-frequency subband to obtain the processed high-frequency subband; Step S4, perform wavelet reconstruction on the processed lowest-frequency subband and the processed high-frequency subband to obtain a preliminary restored image; and Step S5, perform dynamic range stretching and histogram matching on the preliminary restored image to obtain the restored image. Therefore, the restoration method provided by the present invention can effectively improve the clarity and contrast of the image, restore more natural color information, and has better comprehensive performance and broad application prospects.

Description

Technical Field

[0001] The present invention relates to the field of image restoration, and particularly to an image restoration method based on an underwater non-uniform incident light model. Background Art

[0002] In the exploration and investigation of marine biological resources, mineral deposits, energy, and topography, visual technology has received much attention and praise due to its ability to carry high-density information [1][2]. However, due to the absorption and scattering of light by water bodies and underwater suspended particles, etc., underwater captured images often show varying degrees of degradation such as low contrast and blurriness [1]; in addition, since the wavelength of red light is longer than that of green and blue light, it attenuates faster underwater, so that most underwater captured images appear bluish / greenish [2]. Therefore, in order to accurately obtain underwater information, it is very necessary to improve the clarity, contrast of underwater images and achieve color balance.

[0003] The single-frame underwater image clarity improvement methods reported in the existing literature are mainly divided into three categories, namely image enhancement methods based on non-physical models, image restoration methods based on physical models, and deep learning methods. Image enhancement methods based on non-physical models attempt to improve the image quality by directly increasing the pixel values of the image. For example: the multi-scale fusion method reported in literature [3]. Since these methods all start from the perspective of image enhancement, they ignore the physical characteristics of underwater light propagation, so that the enhanced images are prone to phenomena such as over-brightness, over-saturation, and color distortion. Considering the underwater physical imaging mechanism and its high similarity to the foggy weather atmospheric imaging mechanism, under the assumption of uniform underwater illumination, literature [4] reported a classic underwater imaging model in the image restoration methods based on physical models. In this way, by introducing prior knowledge to estimate the unknown parameters in the model and using the model to invert the degradation process, more realistic underwater scene information can be restored. For example: the Dark Channel Prior (DCP) method proposed in literature [5]; the Red Dark Channel Prior (RDCP) method proposed in literature [6] based on literature [5]; literature [7] reported a method based on Image Blurriness and Light Absorption (IBLA). However, when the incident light is non-uniform, the brightness of the images restored by the above methods is too dark [8]. In addition, when correcting colors, the above methods all rely on the inherent parameters of the water body [7], which makes them inapplicable to restoring images captured in different water bodies.

[0004] To better reflect the underwater imaging mechanism, Reference [9] presented a new dual-transmittance imaging model. However, the method for estimating its model parameters is too complex, and it is difficult to restore the true colors in the scene. In recent years, deep learning methods have begun to be applied to improve the quality of underwater images. Reference

[10] reported a method combining a convolutional neural network (CNN) with hybrid wavelets and directional filter banks (HWD). However, due to the common drawbacks of learning-based methods, such as the need for a large number of samples, long training time, and unclear mechanism, it is difficult to apply.

[0005] References

[0006] [1] Lin S, Zhao Y. Review on key technologies of target exploration in underwater optical images[J]. Laser & Optoelectronics Progress, 2020, 57(6): 060002.

[0007] Lin, S., & Zhao, Y. (2020). Review on key technologies of target exploration in underwater optical images. Laser & Optoelectronics Progress, 57(6), 060002.

[0008] [2] Guo J C, Li C Y, Guo C L, et al. Research progress of underwater image enhancement and restoration methods[J]. Journal of Image and Graphics, 2017, 22(3): 273 - 287.

[0009] Guo, J. C., Li, C. Y., Guo, C. L., et al. (2017). Research progress of underwater image enhancement and restoration methods. Journal of Image and Graphics, 22(3), 273 - 287.

[0010] [3] Ancuti C, Ancuti CO, Haber T, et al. Enhancing underwater images and videos by fusion[C] / / 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 16 - 21, 2012, Providence, RI. New York: IEEE, 2012.

[0011] [4] McGlamery B L. A computer model for underwater camera systems[J]. Proceedings of the SPIE, 1980, 208: 221 - 231.

[0012] [5] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341 - 2353.

[0013] [6] Galdran A, Pardo D, Picón A, et al. Automatic red - channel underwater image restoration[J]. Journal of Visual Communication and Image Representation, 2015, 26: 132 - 145.

[0014] [7] Peng Y T, Cosman P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579 - 1594.

[0015] [8] Dai C, Lin M, Wu X, et al. Single hazy image restoration using robust atmospheric scattering model[J]. Signal processing, 2020, 166: 107257.

[0016] [9] Wang G L, Tian J D, Li P Y. Image color correction based on double transmission underwater imaging model[J]. Acta Optica Sinica, 2019, 39(9): 0901002.

[0017] Wang Guolin, Tian Jiandong, Li Pengyue. Image color correction based on double transmission underwater imaging model[J]. Acta Optica Sinica, 2019, 39(9): 0901002.

[0018]

[10] Pan P, Yuan F, Cheng E. Underwater image de-scattering and enhancing using Dehazenet and HWD[J]. Journal of Marine Science and Technology, 2018, 26(4): 531-540.

[0019]

[11] Dai C G, Lin M X, Wang Z, et al. Color compensation based on bright channel and fusion for underwater image enhancement[J]. Acta Optica Sinica, 2018, 38(11): 1110003.

[0020] Dai Chenggang, Lin Mingxing, Wang Zhen, et al. Color compensation based on bright channel and fusion for underwater image enhancement[J]. Acta Optica Sinica, 2018, 38(11): 1110003.

[0021]

[12] Land E H, McCann J J. Lightness and retinex theory[J]. Journal of the Optical Society of America, 1971, 61(1): 1-11.

[0022]

[13] Liu X, Zhang H, Cheung Y M, et al. Efficient single image dehazingand denosing: an efficient multi-scale correlated wavelet approach[J]. ComputerVision and Image Understanding, 2017, 162: 23-33.

[0023]

[14] Duarte A, Codevilla F, Gaya J D O, et al. A dataset to evaluateunderwater image restoration methods[C] / / OCEANS 2016-Shanghai, April 10-13, 2016, Shanghai, China: IEEE, 2016: 1-6.

[0024]

[15] Donoho D L. De-noising by soft-thresholding. IEEE Transactions onInformation Theory, 1995, 41(3): 613-627.

[0025]

[16] Shen Y, Dang J W, Wang Y P, et al. A color underwater image clearnessalgorithm based on tetrolet transform[J]. Acta Optica Sinica, 2017, 37(9): 0910002.

[0026] Shen Yu, Dang Jianwu, Wang Yangping, et al. A color underwater image clearnessalgorithm based on tetrolet transform[J]. Acta Optica Sinica, 2017, 37(9): 0910002.

[0027]

[17] Achanta R, Shaji A, Smith K, et al. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2282.

[0028]

[18] Yang M, Liu J, Li Z. Superpixel-Based Single Nighttime Image Haze Removal[J]. IEEE Transactions on Multimedia, 2018, 20(11): 3008-3018.

[0029]

[19] Zhou W, Li D, Zhang J Q. A complementary color wavelet-based measure on color image sharpness assessment for autofocus. Acta Automatica Sinica, 2020, 46(8): 1615-1627.

[0030] Zhou Wen, Li Dan, Zhang Jianqiu. Complementary Color Wavelet Measure for Evaluating the Sharpness of Color Image Autofocus. Acta Automatica Sinica, 2020, 46(8): 1615-1627.

[0031]

[20] Yang H T, Chang Y L, Wang J, et al. A New Automatic Exposure Algorithm for Video Cameras Using Luminance Histogram[J]. Acta Optica Sinica, 2007, 27(5): 841-848.

[0032] Yang Haitao, Chang Yilin, Wang Jing, et al. An Automatic Exposure Control Method Based on Luminance Histogram[J]. Acta Optica Sinica, 2007, 27(5): 841-848.

[0033]

[21] Yang A P, Wang J B, Yang B W, et al. Joint Deep Denoising Prior for Image Blind Deblurring[J]. Acta Optica Sinica, 2018, 38(10): 1010003.

[0034] Yang Aiping, Wang Jinbin, Yang Bingwang, et al. Joint Deep Denoising Prior for Image Blind Deblurring[J]. Acta Optica Sinica, 2018, 38(10): 1010003.

[0035]

[22] Sheikh H R, Wang Z, Cormack L, et al. LIVE image quality assessment database release 2(2005)[Online], available: http: / / live.ece.utexas.edu / research / quality / , September 1, 2011.

[0036]

[23] Gonzalez R C, Richard E W. Digital image processing[M]. Ruan Q Q, Ruan Y Z, Transl. 2th edition. Beijing: Electronic Industry Press, 2007: 72 - 73.

[0037] Gonzalez, Richard Woods. Digital Image Processing[M]. Ruan Qiuqi, Ruan Yuzhi, Transl. 2nd edition. Beijing: Electronic Industry Press, 2007: 74 - 80.

[0038]

[24] Ancuti C O, Ancuti C, Vleeschouwer C D, et al. Color balance and fusion for underwater image enhancement[J]. IEEE Transactions on Image Processing, 2018, 27(1): 379 - 393.

[0039]

[25] Li CY,Guo C L,Ren W Q,etal.An underwater image enhancementbenchmark dataset and beyond[J].IEEETransactionson Image Processing,2020,29:4376-4389.

[0040]

[26] Yang M,Sowmya A.An underwater color image quality evaluationmetric[J].IEEE Transactionson Image Processing,2015,24(12):6062-6071.

[0041]

[27] Sara U,Akter M,Uddin M S.Image quality assessment through FSIM,SSIM,MSE and PSNR—a comparative study[J].Journal ofComputerandCommunications,2019,07(3):8-18. Summary of the Invention

[0042] The present invention is made to solve the above problems, and aims to provide an image restoration method based on an underwater non-uniform incident light model.

[0043] The present invention provides an image restoration method based on an underwater non-uniform incident light model, which is used to process an underwater image to obtain a restored image. It has the following features and includes the following steps: Step S1, perform wavelet decomposition on the underwater image to obtain the lowest-frequency subband and each high-frequency subband; Step S2, based on the underwater non-uniform incident light model, perform scattering suppression on the lowest-frequency subband to obtain a processed lowest-frequency subband; Step S3, perform noise suppression and detail enhancement processing on each high-frequency subband to obtain a processed high-frequency subband; Step S4, perform wavelet reconstruction on the processed lowest-frequency subband and the processed high-frequency subband to obtain a preliminary restored image; and Step S5, perform dynamic range stretching and histogram matching on the preliminary restored image to obtain the restored image.

[0044] In the image restoration method based on the underwater non-uniform incident light model provided by the present invention, it also has the following features: Among them, the underwater non-uniform incident light model is as follows:

[0045] I c (x)=(1-λ c (x))Jc (x)t c (x)+B c (1 - t c (x))+N c (x)(1)

[0046] In formula (1), I c (x) is the observed underwater image, c ∈ {r, g, b} respectively represent the red channel, green channel, and blue channel, x is the coordinate of each scene point, λ c (x) is the incident light attenuation term that varies with the scene coordinate x, (1 - λ c (x)) represents the corrected normalized incident light intensity, J c (x) is the desired clear image, t c (x) represents the transmission coefficient of each scene point, (1 - λ c (x))J c (x)t c (x) represents the directly attenuated light, B c represents the underwater ambient light in any one of the red channel, green channel, and blue channel, N c (x) represents the underwater additive noise perturbation.

[0047] In the image restoration method based on the underwater non-uniform incident light model provided by the present invention, there is also such a feature: among them, based on the underwater non-uniform incident light model, a preliminary restored image J' c (x) in which the medium scattering and noise under non-uniform incident light are both effectively suppressed is:

[0048] J' c (x) = (1 - λ c (x))J c (x)(2)

[0049] After ignoring the noise interference in each high-frequency sub-band, the expression of the lowest-frequency sub-band I c_lp (x) is as follows:

[0050] I c_lp (x) = J' c_lp (x)t c_lp (x)+B c_lp (1 - t c_lp (x))(3)

[0051] In formulas (2) and (3), I c_lp (x), J' c_lp (x) respectively represent the lowest-frequency sub-band of image I c (x) and image J' c (x), B c_lp represents the ambient light in the lowest-frequency sub-band, tc_lp (x) represents the transmission coefficient of the lowest frequency sub-band. Since the scattering light attenuation coefficients of each channel are similar, t r_lp (x) = t g_lp (x) = t b_lp (x). Therefore, the transmission coefficients of each channel are uniformly represented as t lp (x).

[0052] In the image restoration method based on the underwater non-uniform incident light model provided by the present invention, there is also such a feature, including the following steps: Among them, step S2 includes the following sub-steps: Step S2-1, using the simple linear iterative clustering algorithm to perform superpixel segmentation method and dark channel prior method on the lowest frequency sub-band to obtain the ambient light B c_lp ; Step S2-2, using the ambient light B c_lp and the lowest frequency sub-band I c_lp (x) to obtain the transmission coefficient t lp (x) of the lowest frequency sub-band. The expression is as follows:

[0053]

[0054] In formula (4), ω is the adjustment parameter for suppressing scattering, Ω(x) is the local region for performing minimum filtering, y is the pixel point in this local region. Step S2-3, substituting the ambient light B c_lp and the transmission coefficient t lp (x) into formula (3), then the processed lowest frequency sub-band can be obtained. The expression for processing the lowest frequency sub-band is as follows:

[0055]

[0056] In formula (5), t0 is a parameter with a value of 0.1.

[0057] In the image restoration method based on the underwater non-uniform incident light model provided by the present invention, there is also such a feature: Among them, step S3 includes the following sub-steps: Step S3-1, through the soft threshold method, the coefficients η τ (x) of each high-frequency sub-band after suppressing noise are obtained. The expression is as follows:

[0058] η τ (x) = sgn(x) · (|x| - τ) (6)

[0059] In formula (6), x is the coefficient of each high-frequency sub-band with noise, sgn(x) is the sign function, and τ is the threshold

[0060] Step S3-2, taking the high-frequency details of each high-frequency sub-band after suppressing noise in the horizontal, vertical, and diagonal directions as the gradients in these three directions. The gradient expression is:

[0061]

[0062] In Equation (7), Ω represents a local small window, denotes the underwater observation image I c (x) is the local gradient, denotes the image J' in which both noise and scattering are suppressed c (x) is the local gradient, t Ω denotes the transmission coefficient within the local small window, respectively denote the details in the horizontal, vertical, and diagonal directions; Step S3-3: Perform bilinear interpolation on the transmission coefficient t lp (x) estimated in the lowest-frequency subband to obtain the transmission coefficients of each high-frequency subband; Step S3-4: Divide each high-frequency subband after noise suppression by the corresponding transmission coefficient of each high-frequency subband to obtain the high-frequency subbands with enhanced details.

[0063] In the image restoration method based on the underwater non-uniform incident light model provided by the present invention, there is also such a feature: Among them, Step S5 includes the following sub-steps: Step S5-1: Use the following stretching function to perform dynamic range stretching on the preliminarily restored image to obtain the histograms of each channel after stretching. The stretching function is:

[0064]

[0065] In Equation (8), x is the pixel value of each pixel point, a max ,a min are respectively the maximum and minimum values of the pixels after removing 0.5% of the extremely bright pixels and 0.5% of the extremely dark pixels; and Step S5-2: Take the histogram of the stretched green channel as the reference histogram, and through histogram matching, match the histograms of the stretched red channel and the stretched blue channel into the reference histogram, and output the restored image.

[0066] Functions and effects of the invention

[0067] According to the image restoration method based on the underwater non-uniform incident light model involved in the present invention, the following steps are included: Step S1, perform wavelet decomposition on the underwater image to obtain the lowest-frequency sub-band and each high-frequency sub-band; Step S2, based on the underwater non-uniform incident light model, perform scattering suppression on the lowest-frequency sub-band to obtain the processed lowest-frequency sub-band; Step S3, perform noise suppression and detail enhancement processing on each high-frequency sub-band to obtain the processed high-frequency sub-band; Step S4, perform wavelet reconstruction on the processed lowest-frequency sub-band and the processed high-frequency sub-band to obtain a preliminary restored image; and Step S5, perform dynamic range stretching and histogram matching on the preliminary restored image to obtain the restored image. Since the restoration method provided by the present invention is based on the underwater non-uniform incident light model and obtains the underwater image restoration method through wavelet transform and the statistical characteristics of natural image histograms. Therefore, the image restoration method based on the underwater non-uniform incident light model provided by the present invention can effectively improve the clarity and contrast of the image and restore more natural color information, and has better comprehensive performance, is applicable to images taken in different water bodies, does not require a large number of samples for training, and has a wide range of application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a flowchart of the image restoration method based on the underwater non-uniform incident light model in the embodiment of the present invention;

[0069] Figure 2 is a flowchart of the processing of the lowest-frequency sub-band in the embodiment of the present invention;

[0070] Figure 3 is a flowchart of the processing of each high-frequency sub-band in the embodiment of the present invention;

[0071] Figure 4 is a flowchart of the processing of the preliminary restored image in the embodiment of the present invention;

[0072] Figure 5 is the histogram of the high-contrast colorless-offset image in the embodiment of the present invention;

[0073] Figure 6 is the histogram adjustment diagram of the preliminary restored image in the embodiment of the present invention;

[0074] Figure 7 is the Fourier transform frequency spectrum diagram of the reference image and its corresponding turbid underwater image in Test Example 1 of the present invention;

[0075] Figure 8 is a comparison diagram of two filtering windows in Test Example 2 of the present invention; and

[0076] Figure 9 is a result comparison diagram of the effectiveness test of different algorithms in Test Example 3. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0077] In order to make the technical means, creative features, achieved objectives and effects realized by the present invention easy to understand, the image restoration method based on the underwater non-uniform incident light model of the present invention will be specifically described below in conjunction with embodiments and the accompanying drawings.

[0078] In the following embodiments, the non-uniform incident light model in the image restoration method based on the underwater non-uniform incident light model is based on the traditional imaging model. The traditional imaging model is as follows:

[0079] In the research on underwater image imaging, it is generally considered that its model is the foggy weather atmospheric scattering model that ignores the influence of forward scattering:

[0080] I c (x) = J c (x)t c (x) + B c (1 - t c (x))(1)

[0081] In formula (1): x is the coordinate of each scene point; c represents any one of the red, green, and blue channels; I c (x) is the observed underwater image; J c (x) is the desired clear image; B c represents the underwater ambient light of any one of the red, green, and blue channels. t c (x) represents the transmission coefficient of each scene point, which is related to the scene depth, and its expression is:

[0082] t c (x) = [exp(-β c d(x))](2)

[0083] In formula (2): d(x) represents the depth of each scene point at x; β c represents the scattering coefficient of light underwater.

[0084] Formula (1) shows that underwater imaging is mainly affected by two parts:

[0085] 1) The absorption and attenuation effect of water on light, and J c (x)t c (x) represents the directly attenuated light, which describes the part of the reflected light from the underwater object surface that directly reaches the camera after being absorbed by the water body;

[0086] 2) The scattering effect of the underwater medium on light, which is represented by B c (1 - t c (x)) represents the scattered light, which describes the part of the light that reaches the camera after being scattered by underwater suspended particles and other media, and is generally considered to be the main reason for underwater image blurring.

[0087] Since the absorption and attenuation of light with different spectra during underwater propagation vary with environmental factors such as water turbidity and underwater depth, the incident light irradiating different object surfaces is non-uniform when taking underwater images, resulting in a low dynamic range and decreased contrast of the images. In addition, the degree of deviation of the incident light from white light in different water bodies is also different, which will cause different color casts in the images. When the traditional imaging model is used for underwater image restoration, it usually assumes that the incident light is uniform. Therefore, it often obtains images with insufficient contrast and inaccurate restoration of color information.

[0088] According to the Retinex theory, the light intensity S c (x) actually received by the camera can be regarded as the product of the light intensity L c (x) incident on the imaging object surface and the reflection coefficient R c (x) of the imaging object surface, that is:

[0089] S c (x) = L c (x) · R c (x) (3)

[0090] The above formula shows that if we hope to restore the real image from the image captured by the camera, we need to determine the actual incident light intensity. In the model of formula (1), since it assumes that the incident light is uniform, the light intensity values irradiating different imaging object surfaces are normalized to 1. However, the actual incident light underwater is non-uniform, that is, the incident light intensities on different object surfaces are inconsistent. Therefore, it is necessary to introduce the incident light attenuation term λ c (x) in the traditional underwater imaging model to describe the non-uniformity of the actual incident light. In addition, considering that the underwater light disturbance caused by underwater turbulence or fish swimming can be described by the additive noise term N c (x).

[0091] <Example>

[0092] Figure 1 is the flowchart of the image restoration method based on the underwater non-uniform incident light model in the embodiments of the present invention.

[0093] As Figure 1 shown, the image restoration method based on the underwater non-uniform incident light model in the embodiments of the present invention includes the following steps:

[0094] Step S1, perform wavelet decomposition on the underwater image to obtain the lowest-frequency sub-band and each high-frequency sub-band;

[0095] Step S2, based on the underwater non-uniform incident light model, perform scattering suppression on the lowest-frequency sub-band to obtain the processed lowest-frequency sub-band;

[0096] In step S3, perform noise suppression and detail enhancement on each high-frequency sub-band to obtain processed high-frequency sub-bands;

[0097] In step S4, perform wavelet reconstruction on the processed lowest-frequency sub-band and the processed high-frequency sub-bands to obtain a preliminary restored image;

[0098] In step S5, perform dynamic range stretching and histogram matching on the preliminary restored image to obtain the restored image.

[0099] In this embodiment, the underwater non-uniform incident light model is as follows:

[0100] I c (x) = (1 - λ c (x))J c (x)t c (x) + B c (1 - t c (x)) + N c (x)(1)

[0101] In formula (1), I c (x) is the observed underwater image, c ∈ {r, g, b} respectively represent the red channel, the green channel, and the blue channel, x is the coordinate of each scene point, λ c (x) is the incident light attenuation term that varies with the scene coordinate x, (1 - λ c (x)) represents the corrected normalized incident light intensity, J c (x) is the desired clear image, t c (x) represents the transmission coefficient of each scene point, (1 - λ c (x))J c (x)t c (x) represents the directly attenuated light, B c represents the underwater ambient light in any one of the red channel, the green channel, and the blue channel, N c (x) represents the underwater additive noise perturbation.

[0102] In this embodiment, since the non-uniform incident light only affects the attenuation of the direct light, that is, the first term on the right side of formula (1), the preliminary restored image J' c (x) in which the medium scattering and noise are effectively suppressed under this incident light can be defined as: J' c (x) = (1 - λ c (x))J c (x)(2),

[0103] At this time, substitute formula (2) into formula (1), and we get

[0104] I c (x) = J' c(x)t c (x)+B c (1 - t c (x))+N c (x),

[0105] Using wavelet decomposition, the influence of underwater medium scattering can be suppressed in the lowest frequency sub - band, while noise is suppressed in each high - frequency sub - band. Therefore, the expression of the lowest frequency sub - band after ignoring high - frequency noise interference is as follows:

[0106] I c_lp (x) = J' c_lp (x)t c_lp (x)+B c_lp (1 - t c_lp (x))(3)

[0107] where c ∈ {r, g, b}; I c_lp (x), J' c_lp (x) respectively represent the lowest frequency sub - bands of the image I c (x) and the image J' c (x); B c_lp represents the ambient light in the lowest frequency sub - band, and t c_lp (x) represents the transmission coefficient in the lowest frequency sub - band.

[0108] In this embodiment, since the scattering light attenuation coefficients of each channel are similar, it can be considered that t r_lp (x) = t g_lp (x) = t b_lp (x). Therefore, the transmission coefficients of each channel are uniformly represented as t lp (x).

[0109] Figure 2 is the processing flow chart of the lowest frequency sub - band in the embodiment of the present invention.

[0110] As Figure 2 shown, step S2 includes the following sub - steps:

[0111] Step S2 - 1, using the simple linear iterative clustering algorithm to perform super - pixel segmentation method and dark - channel prior method on the lowest frequency sub - band to obtain the ambient light;

[0112] In this embodiment, the specific steps of step S2 - 1 are as follows:

[0113] Perform superpixel segmentation on the lowest frequency sub-band using the Simple Linear Iterative Clustering (SLIC) algorithm, that is, segment adjacent pixels with similar features into the same region; then use local minimum filtering in the Dark Channel Prior (DCP) method to replace each irregular region after segmentation with the previous defined rectangular window. The filtered window after superpixel segmentation is as shown in Figure 6 shown.

[0114] If we let S i be the i-th window after superpixel segmentation, the corresponding dark channel map at this time can be expressed as:

[0115]

[0116] where I dark_lp (x) represents the dark channel map of the lowest frequency sub-band after superpixel segmentation.

[0117] Next, using the Dark Channel Prior method, take the top 0.1% of the brightest pixels in the dark channel map as the candidate region for ambient light, and take the average value of each channel of the pixel points corresponding to the candidate region in the original image to obtain the ambient light B c_lp .

[0118] Step S2-2, use the ambient light B c_lp and the lowest frequency sub-band I c_lp (x) to obtain the transmission coefficient t lp (x) of the lowest frequency sub-band, and the expression is as follows:

[0119]

[0120] In Equation (4), ω is the adjustment parameter for suppressing scattering, Ω(x) is the local region for minimum filtering, y is the pixel point in this local region,

[0121] where, substitute the ambient light B c_lp and the lowest frequency sub-band I c_lp (x) into the following equation

[0122]

[0123] to obtain the transmission coefficient t lp (x).

[0124] Step S2-3, substitute the ambient light B c_lp and the transmission coefficient t lp (x) into Equation (3), and we can obtain the processed lowest frequency sub-band. The expression for processing the lowest frequency sub-band is as follows:

[0125]

[0126] In Equation (5), t0 is a parameter with a value of 0.1.

[0127] Figure 3 It is the processing flow chart of each high-frequency sub-band in the embodiments of the present invention.

[0128] As Figure 3 shown, step S3 includes the following sub-steps:

[0129] Step S3-1: By using the soft threshold method, the coefficients η τ (x) of each high-frequency sub-band after noise suppression are obtained, and its expression is as follows:

[0130] η τ (x) = sgn(x)·(|x| - τ) (6)

[0131] In Equation (6), x is the coefficient of the high-frequency sub-band with noise, sgn(x) is the sign function, and τ is the threshold.

[0132] In this embodiment, since after wavelet decomposition, the image details and additive noise contained in each high-frequency sub-band are hardly affected by the scattering of the underwater medium. Therefore, directly performing a soft threshold operation on the coefficients of each high-frequency sub-band can effectively suppress noise.

[0133] Step S3-2: The high-frequency details of each high-frequency sub-band after noise suppression in the horizontal, vertical, and diagonal directions are used as the gradients in these three directions, and its gradient expression is:

[0134]

[0135] In Equation (7), Ω represents a local small window. represents the local gradient of the underwater observation image I c (x). represents the local gradient of the image J' c (x) where both noise and scattering are suppressed, and t Ω represents the transmission coefficient within the local small window. respectively represent the details in the horizontal, vertical, and diagonal directions.

[0136] Step S3-3: Perform bilinear interpolation on the transmission coefficient t lp (x) estimated in the lowest-frequency sub-band to obtain the transmission coefficients of each high-frequency sub-band.

[0137] Step S3-4: Divide each high-frequency sub-band after noise suppression by the corresponding transmission coefficient of each high-frequency sub-band to obtain a high-frequency sub-band with enhanced details.

[0138] In theory, the more levels of wavelet decomposition, the more high-frequency details available for enhancement, and the better the enhancement effect. However, on the one hand, increasing the decomposition levels will significantly increase the time-consuming for estimating the transmission coefficients of each high-frequency subband; on the other hand, after wavelet decomposition of the image, the detailed features are concentrated in level 1 and level 2. Therefore, in this embodiment, the level of wavelet decomposition is set to 2.

[0139] Step S4, perform wavelet reconstruction on the processed lowest-frequency subband and the processed high-frequency subbands to obtain a preliminary restored image J' c (x), and enter step S5.

[0140] Through steps S1 - S4, for the lowest-frequency subband after wavelet decomposition, the DCP method after superpixel segmentation is used to suppress the blur caused by medium scattering. After each high-frequency subband uses the soft threshold method to suppress noise and enhance details, and then performs wavelet reconstruction, the preliminary restored image J' c (x) in which both underwater medium scattering and noise are effectively suppressed in Equation (5) can be obtained.

[0141] Step S5, perform dynamic range stretching and histogram matching on the preliminary restored image J' c (x) to obtain the restored image.

[0142] Figure 4 is the processing flow chart of the preliminary restored image in the embodiment of the present invention.

[0143] As Figure 4 shown, step S5 includes the following sub-steps:

[0144] Step S5-1, use the following stretching function to perform dynamic stretching on the preliminary restored image to obtain the histograms of each channel after stretching,

[0145] In this embodiment, the stretching function is:

[0146]

[0147] In Equation (8), x is the pixel value of each pixel point, a max , a min are respectively the maximum value and the minimum value of the pixels after removing 0.5% of the extremely bright pixels and 0.5% of the extremely dark pixels; and

[0148] Step S5-2, take the histogram of the stretched green channel as the reference histogram, and through histogram matching, match the histograms of the stretched red channel and the stretched blue channel into the reference histogram, and output the restored image,

[0149] In this embodiment, the green channel of the underwater image loses less compared to the red channel and the blue channel. Therefore, take the histogram of the stretched green channel as the reference histogram.

[0150] Figure 5 It is the high-contrast and color-shift-free image histogram in the embodiments of the present invention.

[0151] In this embodiment, by utilizing the statistical characteristics of the natural image histogram, the reference images in the LIVE database and another approximately 200 high-contrast and color-shift-free images collected under natural light are statistically analyzed. Through random observation, it is found that in high-contrast and color-shift-free images, the histogram distribution ranges of the red, green, and blue channels are relatively wide and the histogram heights are similar. Some of the observation results are as Figure 3 shown. Figure 3 From left to right in [the figure] are the original image and the histograms of its corresponding red channel, green channel, and blue channel.

[0152] Among them, Fig. (a) is the "building2" image; Fig. (b) is the "stream" image.

[0153] Figure 6 It is the histogram adjustment diagram of the preliminary restored image in the embodiments of the present invention.

[0154] Among them, Fig. (a) is the preliminary restored image obtained in step 4 of this embodiment, and Fig. (b) is the output image after histogram adjustment of Fig. (a) using the dynamic range stretching and histogram matching method. By comparing Fig. (a) and Fig. (b), it can be seen that the dynamic ranges of each channel of the restored image obtained by stretching the dynamic range of the preliminary restored image and performing histogram matching are wider, and their histograms also have high similarity. Therefore, the color and contrast of the restored image have been greatly improved.

[0155] <Test Example 1>

[0156] Frequency-domain characteristic analysis test of underwater medium scattering and additive noise

[0157] Test method: Assume that the image blur caused by underwater medium scattering also mainly concentrates in the lower frequency range. Select the reference images in the dataset and their corresponding turbid underwater images, and verify the above assumption by observing the spectra of their Fourier transforms.

[0158] Some of the test results are as Figure 7 .

[0159] Figure 7 It is the Fourier transform spectrogram of the reference image and its corresponding turbid underwater image in Test Example 1 of the present invention.

[0160] Among them, Figure (a) and Figure (c) are reference images, Figure (b) and Figure (d) are the corresponding underwater images of Figure (a) and Figure (c) respectively, Figure (e) and Figure (g) are the Fourier spectra of Figure (a) and Figure (c) respectively, and Figure (f) and Figure (h) are the Fourier spectra of Figure (b) and Figure (d) respectively.

[0161] From Figure 7 It can be seen that the reference images usually show more high-frequency components, while the images taken underwater in turbid conditions usually show more low-frequency components. Therefore, the hypothesis proposed in Test Example 1 is reasonable, that is, after the image is wavelet-transformed, the additive noise will be mainly distributed in its high-frequency subbands, while the signal energy will be mainly concentrated in the lowest-frequency subband. This means that wavelet decomposition of the observed underwater image provides the possibility of suppressing the blur caused by medium scattering in the lowest-frequency subband, and suppressing noise and enhancing details in each high-frequency subband.

[0162] <Test Example 2>

[0163] Comparison test of two filtering windows

[0164] Test method: Compare the filtering window segmented by the superpixel segmentation method with the rectangular filtering window segmented by the dark channel prior method.

[0165] The comparison test results are as Figure 8 shown.

[0166] Figure 8 is the comparison diagram of the two filtering windows in Test Example 2 of the present invention.

[0167] Among them, Figure (a) is the rectangular filtering window; Figure (b) is the superpixel segmentation filtering window; Figure (c) is the transmission coefficient map corresponding to Figure (a); Figure (d) is the transmission coefficient map corresponding to Figure (b).

[0168] As Figure 8 shown, from Figure (a) and Figure (b), it can be seen that superpixel segmentation can better divide each small window with the same depth of field. Therefore, after minimum filtering is performed on each segmented small window, the estimated transmission coefficient map (d) can also better reflect the true depth of field information of the image than Figure (c). Observing the fish tail area in the red frame of Figure (c) and Figure (d), it can be seen that superpixel segmentation can capture more information on local transmission coefficient changes, and can better suppress the influence of underwater medium scattering.

[0169] <Test Example 3>

[0170] Effectiveness test of different algorithms

[0171] Test method: In the underwater image set and the reference underwater image set with different turbidity levels, 8 original underwater images of different types are subjected to wavelet decomposition and superpixel segmentation, and processed by the image restoration method based on the underwater non-uniform incident light model in this embodiment and the multi-scale fusion algorithm, RDCP algorithm, IBLA algorithm, dual transmittance algorithm and CNN combined with HWD algorithm, and the performance of the algorithm is analyzed by comprehensively considering subjective visual effects and objective evaluation indicators.

[0172] During the test, Haar wavelet was used for wavelet decomposition with a decomposition level of 2. The number of segmentation windows was set to 500 for superpixel segmentation. When suppressing the influence of medium scattering, the coefficient value of the lowest frequency subband was normalized to the interval [0,1] for the convenience of calculation, and then the coefficient was restored.

[0173] Test results such as Figure 9 shown.

[0174] Figure 9 This is a comparison chart of the results of the effectiveness test of different algorithms in test case three.

[0175] Among them, (a) original image; (b) multi-scale fusion algorithm; (c) RDCP algorithm; (d) IBLA algorithm; (e) dual transmittance algorithm; (f) CNN combined with HWD algorithm; (g) algorithm in the embodiment.

[0176] like Figure 9 It can be seen that by comparing the subjective visual effects of different algorithms, the multi-scale fusion algorithm can obtain clearer images with better color cast correction. However, since the algorithm ignores the relationship between underwater image blur and scene depth, it only fuses the results of white balance and contrast enhancement of the original image. Since the design of the fusion weight is not suitable for underwater images of various degradation types, local over-brightness occurs.

[0177] The RDCP algorithm has the worst performance: on the one hand, it uses the same rectangular filter window as the DCP method when estimating the transmission coefficient of the red channel, so that the accuracy of the transmission coefficient is limited, which leads to the limitation of the clarity of the restored image; on the other hand, after estimating the transmission coefficient of the red channel, it uses a fixed relationship between the scattering coefficient and the wavelength to estimate the transmission coefficients of the other two channels, so it is only suitable for images taken in specific underwater scenes.

[0178] Although the IBLA algorithm can also effectively improve the accuracy of depth of field estimation, it still adopts the method of estimating the red channel transmission coefficient first and then estimating the transmission coefficients of the other two channels. Therefore, the robustness of color correction is poor.

[0179] Although the double transmittance algorithm can better enhance the contrast and details of the image, the image color is relatively single and cannot reflect the color information of the real scene.

[0180] Although the CNN combined with the HWD algorithm can obtain high-definition images and the details are more prominent after edge enhancement, the overall color of the obtained images is grayish and the local part is darker.

[0181] The underwater non-uniform incident light model in this embodiment can better reflect the mechanism of underwater image formation. At the same time, the proposed restoration method can obtain high-quality images with better clarity, higher contrast and more natural colors for different types of underwater images. Among them, for Figure 9 Image2 in, the coral color in the image obtained by the restoration method of this embodiment is more vivid; for Figure 9 Image4 and Image7 in, there is no situation of local over-brightness / over-darkness in the small fish in the restored images by the restoration method of this embodiment; even for the fish school image taken in the deep sea like Image8, the restoration method of this embodiment has achieved better visual effects than several other algorithms.

[0182] Select the underwater color image quality evaluation (UCIQE) index to objectively evaluate the experimental results of different algorithms. The UCIQE index is a weighted index that takes into account the chromaticity, contrast and saturation of underwater images. The higher the UCIQE value of an image, the better its quality.

[0183] Figure 9 The UCIQE indexes corresponding to the images shown are shown in Table 1, where the bold and italicized values represent the optimal values of each image processed by different algorithms.

[0184] Table 1 Quantitative evaluation results of UCIQE indexes

[0185]

[0186]

[0187] As can be seen from Table 1, the average UCIQE value after being processed by the restoration method in the embodiment of the present invention is the highest; the highest UCIQE value is obtained on the images except Image2, Image4, Image6; for the images of Image2, Image4, Image6, although the UCIQE value after being processed by the restoration method in the embodiment of the present invention is the second highest, combined with Figure 9It can be seen that the IBLA algorithm processes Image2 and Image6, and the double transmittance algorithm processes Image4. Although the UCIQE index is the highest, the color cast is not well corrected. At this time, due to the too high saturation of the processed image, its UCIQE index is affected.

[0188] Considering the visual effect and objective evaluation index of the image comprehensively, it can be found that the restoration method in the embodiment of the present invention processes the underwater image to obtain an image that is visually clearer, has higher contrast, and more balanced color, and has the highest average UCIQE value.

[0189] Functions and effects of the embodiment

[0190] According to the image restoration method based on the underwater non-uniform incident light model involved in this embodiment, it includes the following steps: Step S1, perform wavelet decomposition on the underwater image to obtain the lowest frequency sub-band and each high-frequency sub-band; Step S2, based on the underwater non-uniform incident light model, perform scattering suppression on the lowest frequency sub-band to obtain the processed lowest frequency sub-band; Step S3, perform noise suppression and detail enhancement processing on each high-frequency sub-band to obtain the processed high-frequency sub-band; Step S4, perform wavelet reconstruction on the processed lowest frequency sub-band and the processed high-frequency sub-band to obtain a preliminary restored image; and Step S5, perform dynamic range stretching and histogram matching on the preliminary restored image to obtain the restored image. Because the restoration method provided in this embodiment is based on the underwater non-uniform incident light model and obtains the underwater image restoration method through wavelet transform and the histogram statistical characteristics of natural images. Therefore, the image restoration method based on the underwater non-uniform incident light model provided in this embodiment can effectively improve the clarity and contrast of the image, restore more natural color information, has better comprehensive performance, is applicable to images taken in different water bodies, does not require a large number of samples for training, and has a wide application prospect.

[0191] Furthermore, in view of the deficiencies in the description of the non-uniform incident light in the underwater imaging environment, as well as the noise interference caused by underwater turbulence and fish swimming in the classical underwater imaging model, this embodiment proposes an imaging model of underwater non-uniform incident light. Based on the proposed model, wavelet transform and the histogram statistical characteristics of natural images, a new method for underwater image restoration is further proposed. Analysis shows that by performing wavelet decomposition on the underwater image, underwater medium scattering can be suppressed in the lowest frequency and each high-frequency sub-band of the wavelet decomposition, and noise can be suppressed and details can be enhanced. The image contrast deviation and color deviation caused by underwater non-uniform incident light can be corrected through the proposed dynamic range stretching and matching that conforms to the histogram statistical characteristics of natural images. Experimental results show that the comprehensive performance of the model and the corresponding restoration method proposed in this embodiment is better than the algorithms in the prior art.

[0192] The above embodiments are preferred examples of the present invention and are not used to limit the protection scope of the present invention.

Claims

1. An image restoration method based on an underwater non-uniform incident light model, which is used to process underwater images and obtain restored images, characterized in that, It includes the following steps: Step S1: Perform wavelet decomposition on the underwater image to obtain the lowest-frequency sub-band and each high-frequency sub-band; Step S2: Based on the underwater non-uniform incident light model, perform scattering suppression on the lowest-frequency sub-band to obtain a processed lowest-frequency sub-band; Step S3: Perform noise suppression and detail enhancement processing on each high-frequency sub-band to obtain processed high-frequency sub-bands; Step S4: Perform wavelet reconstruction on the processed lowest-frequency sub-band and the processed high-frequency sub-bands to obtain a preliminary restored image; And Step S5: Perform dynamic range stretching and histogram matching on the preliminary restored image to obtain the restored image; Among them, the underwater non-uniform incident light model is as follows: I c (x) = (1 - λ c (x))J c (x)t c (x) + B c (1 - t c (x)) + N c (x)(1) In Equation (1), I c (x) is the observed underwater image, c ∈ {r, g, b} represent the red channel, the green channel, and the blue channel respectively, x is the coordinate of each scene point, and λ c (x) is the incident light attenuation term that varies with the scene coordinate x, and (1 - λ c (x)) represents the corrected normalized incident light intensity. J c (x) is the desired clear image, t c (x) represents the transmission coefficient of each scene point, and (1 - λ c (x))J c (x)t c (x) represents the directly attenuated light. B c represents the underwater ambient light of any one of the red channel, the green channel, and the blue channel, and N c (x) represents the underwater additive noise perturbation.

2. The image restoration method based on the underwater non-uniform incident light model according to claim 1, wherein: Among them, Based on the underwater non-uniform incident light model, a preliminary restored image J' is obtained, in which medium scattering and noise are effectively suppressed under non-uniform incident light c (x) is as follows: J' c (x) = (1 - λ c (x))J c (x) (2) The lowest-frequency subband I after ignoring the noise interference of each high-frequency subband c_lp (x) is expressed as follows: I c_lp (x) = J' c_lp (x)t c_lp (x) + B c_lp (1 - t c_lp (x))(3) In formulas (2) and (3), I c_lp (x), J' c_lp (x) respectively represent the lowest frequency sub-bands of the image I c (x) and the image J' c (x), B c_lp represents the ambient light in the lowest frequency sub-band, t c_lp (x) represents the transmission coefficient in the lowest frequency sub-band.

3. The image restoration method based on the underwater non-uniform incident light model according to claim 1, wherein It includes the following steps: Among them, step S2 includes the following sub-steps: Step S2-1: Use the simple linear iterative clustering algorithm to perform superpixel segmentation and dark channel prior method on the lowest frequency sub-band to obtain the ambient light B c_lp ; Step S2-2, using the ambient light B c_lp and the lowest frequency sub-band I c_lp (x) to obtain the transmission coefficient t lp (x) of the lowest frequency sub-band, and the expression is as follows: In formula (4), ω is the adjustment parameter for suppressing scattering, Ω(x) is the local region for performing minimum filtering, and y is the pixel point in the local region. Step S2-3, input the ambient light B c_lp and the transmission coefficient t lp (x) into Equation (3), then the processed lowest-frequency subband can be obtained, and the expression of the processed lowest-frequency subband is as follows: In formula (5), t0 is a parameter with a value of 0.

1.

4. The image restoration method based on the underwater non-uniform incident light model according to claim 3, It is characterized in that: Among them, The said step S3 includes the following sub-steps: Step S3-1, by means of a soft threshold method, to obtain the high-frequency subband coefficients η τ (x) after noise suppression, and its expression is as follows: η τ (x) = sgn(x)g(|x| - τ) (6) In formula (6), x is the coefficient of each high-frequency sub-band with noise, sgn(x) is the sign function, and τ is the threshold. Step S3-2: Take the high-frequency details of each high-frequency sub-band after noise suppression in the horizontal, vertical, and diagonal directions as the gradients in the three directions, and its gradient expression is: In Equation (7), Ω represents a local small window, represents the underwater observation image I c the local gradient of (x), represents the image J' in which both noise and scattering are suppressed c the local gradient of (x), t Ω represents the transmission coefficient within the local small window, respectively represent the details in the horizontal, vertical, and diagonal directions; Step S3-3, perform bilinear interpolation on the transmission coefficient t lp (x) estimated in the lowest frequency subband to obtain the transmission coefficients of each high-frequency subband; And Step S3-4: Divide each high-frequency sub-band after noise suppression by the corresponding transmission coefficient of each high-frequency sub-band to obtain a high-frequency sub-band with enhanced details.

5. The image restoration method based on the underwater non-uniform incident light model according to claim 4, It is characterized in that: Among them, The said step S5 includes the following sub-steps: Step S5-1, perform dynamic range stretching on the preliminary restored image J' c using the following stretching function (x) to obtain the histogram of each channel after stretching The stretching function is: In formula (8), x is the pixel value of each pixel point, a max , a min are respectively the maximum value and the minimum value of the pixels after removing 0.5% of the extremely bright pixels and 0.5% of the extremely dark pixels; And Step S5-2: Take the histogram of the stretched green channel as the reference histogram, and through histogram matching, match the histograms of the stretched red channel and the stretched blue channel into the reference histogram, and output the restored image.

Images