An underwater image restoration method based on multi-scale color compensation and adaptive deblurring

Abstract

The application relates to an underwater image restoration method based on multi-scale color compensation and adaptive deblurring, which comprises the following steps: inputting an original underwater image, and establishing an underwater optical imaging model of the underwater image according to optical characteristics; removing color deviation of the underwater image by adopting a multi-scale color compensation algorithm to obtain a color-corrected image; estimating global background light of the underwater image based on a secondary hierarchical search algorithm of optical prior; estimating transmittance of the underwater image based on an adaptive dark channel algorithm; and obtaining a restored underwater image through inverse transformation according to the established underwater optical imaging model, the obtained global background light, the transmittance and the color-corrected image. Compared with the prior art, the application can effectively correct color deviation of the underwater image, enhance image contrast, and cope with underwater scenes with different degradation degrees, and has strong generalization ability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater image processing, and particularly relates to an underwater image restoration method based on multi-scale color compensation and adaptive deblurring. BACKGROUND

[0002] In recent years, the role of the ocean for human development is increasingly evident, and in order to better develop and utilize the energy contained in the ocean, the underwater environment and seabed information need to be detected and analyzed. Images are important carriers of information, and underwater images play a cornerstone role in underwater target detection and underwater robots, and the visual quality of underwater images plays an extremely important role in marine military fields, marine environmental protection, marine engineering construction and other marine research. High-quality underwater images lay an important foundation for human exploration of the mysterious ocean depth. However, due to the absorption and scattering effect of light in water, the absorption and scattering effect of suspended particles on light, the underwater image often has low visibility, color deviation, low contrast, noise and image blur and other problems. These shortcomings of underwater images cause many limitations to visual perception analysis and actual underwater application. Therefore, for marine scientific research and engineering application, it is necessary to study an effective underwater image sharpening method.

[0003] At present, underwater image sharpening methods mainly include image enhancement based on non-physical model, image restoration based on physical model and data-driven method. Among them, the image enhancement method based on non-physical model can process some simple content and less serious degradation underwater images, but for harsh underwater environment, this method cannot automatically adjust according to the physical characteristics of the underwater scene, and thus the texture details of the image are easily ignored, resulting in problems such as over-saturation and color distortion in the enhanced image.

[0004] The data-driven method often needs a large amount of data and a long time to train, but most of the data are artificially synthesized, which are different from the underwater images in the real world. Therefore, the robustness and generalization ability of the data-driven method are limited.

[0005] The restoration method based on physical model estimates the transmitted light and ambient light of the underwater scene by using priori or hypothesis, and then inverses the physical model to obtain high-quality underwater images. Although this method is very effective for dewarping in most cases, it has limited effect on underwater image restoration due to the neglect of the light absorption effect of underwater medium. SUMMARY

[0006] The underwater image restoration method based on multi-scale color compensation and adaptive deblurring can effectively correct the color deviation of the underwater image, enhance the contrast of the image, and cope with underwater scenes with different degradation degrees, and has strong generalization ability.

[0007] The purpose of the present application can be achieved by the following technical solutions: an underwater image restoration method based on multi-scale color compensation and adaptive deblurring, comprising the following steps:

[0008] S1, input the original underwater image, and establish an underwater optical imaging model of the underwater image according to optical characteristics;

[0009] S2, remove the color deviation of the underwater image by using a multi-scale color compensation algorithm to obtain a color-corrected image;

[0010] S3, estimate the global background light of the underwater image based on a secondary hierarchical search algorithm of optical priori;

[0011] S4, estimate the transmittance of the underwater image based on an adaptive dark channel algorithm;

[0012] S5, according to the established underwater optical imaging model, combining the obtained global background light, transmittance and color-corrected image, the restored underwater image is obtained through inverse transformation.

[0013] Further, the underwater optical imaging model established in step S1 is:

[0014] I c (x)=J c (x)t c (x)+A c (x)(1-t c (x)),c∈{r,g,b}

[0015] Wherein, x represents a pixel, I c (x) is the observed image, J c (x) is the restored image, A c (x) is the global background light, t c (x)∈[0,1] represents the transmittance, and c is one of r, g and b red, green and blue color channels.

[0016] Further, the step S2 comprises the following steps:

[0017] S21, decompose the original underwater image into r, g and b three color channels, and calculate the total pixel mean value of each color channel, and redefine the large color channel, the middle color channel and the small color channel according to the numerical value of the total pixel mean value;

[0018] S22, based on the multi-scale color compensation algorithm, respectively for large color channel, middle color channel and small color channel compensation, get color corrected image.

[0019] Further, the formula for calculating the total pixel mean of the color channel in step S21 is:

[0020]

[0021] Where H and W represent the height and width of the input image;

[0022] According to the total pixel mean of the three color channels redefined, the maximum total pixel mean is I l , the middle is I m , and the minimum is I s .

[0023] Further, the specific process of step S22 is:

[0024] For large color channel I l , linear transmission is used to increase its dynamic range, expressed as:

[0025]

[0026] Where I l ' is the corrected large color channel, and are the maximum and minimum pixel values of the large color channel, respectively;

[0027] For the middle color channel I m , the compensation process is expressed as:

[0028]

[0029] Where I' m is the corrected middle color channel, and represent the average values of I l and I m channels;

[0030] For the most severely attenuated small color channel I s , the compensation process is expressed as:

[0031]

[0032] Where I s ' is the corrected small color channel, represents the average value of the small color channel.

[0033] Further, the step S3 specifically comprises the following steps:

[0034] S31, using a hierarchical search technique to divide the color-corrected underwater image into four rectangular regions, and selecting the region with the highest brightness;

[0035] S32, then performing a second hierarchical search on the selected high-brightness region to select the region with the highest blur degree;

[0036] S33, taking the regions obtained by the two hierarchical searches as the background light candidate region, calculating the dark channel image of the background light candidate region, sorting the pixel values in the dark channel image from large to small, and selecting the original image pixel points corresponding to the top 0.1% of the pixel points as candidate pixel points;

[0037] When the average value of the blue channel of the image is greater than the average value of the green channel, search for the pixel point with the largest blue-to-red channel ratio in the candidate pixel points as the global background light;

[0038] When the average value of the green channel of the image is greater than the average value of the blue channel, then search for the pixel with the largest green channel ratio in the candidate pixel points as the global background light.

[0039] Further, the region brightness calculation formula in the step S31 is:

[0040]

[0041] The region blur degree calculation formula in the step S32 is:

[0042]

[0043] wherein n is the total number of pixels in a region, is the percentage of bright pixels, and ε is an adjustable parameter, and S(a, b) represents a sigmoid function, and s is a constant.

[0044] Further, the specific process of the step S4 is:

[0045] First, compare the average value of the red channel with the value of the preset threshold σ, if the average value of the red channel is less than σ, then the transmittance is represented as:

[0046]

[0047] If the average value of the red channel is greater than σ, then first obtain the transmittance of the red channel through the underwater dark channel algorithm, which is represented as:

[0048]

[0049] Then the attenuation coefficient ratio of green-red channel and blue-red channel is introduced to represent the transmission of green and blue channel, which is represented as:

[0050]

[0051] Wherein, And The attenuation coefficient ratio of green-red channel and blue-red channel.

[0052] Further, the preset threshold σ is 56.

[0053] Further, the specific process of the step S5 is: substituting the obtained global background light, transmittance and color corrected image into the underwater optical imaging model to obtain the restored underwater image through inverse transformation as:

[0054]

[0055] Wherein, A c (x) is the global background light, t c (x) is the transmittance, and I c '(x) is the color corrected image.

[0056] Compared with the prior art, the present application has the following advantages:

[0057] The present application firstly establishes an underwater optical imaging model of the underwater image for the original underwater image; then on one hand, a multi-scale color compensation algorithm is adopted to obtain a color corrected image; on the other hand, a secondary hierarchical search algorithm based on optical prior is adopted to estimate the global background light of the underwater image, and an adaptive dark channel algorithm is adopted to estimate the transmittance of the underwater image; finally, according to the established underwater optical imaging model, the obtained global background light, transmittance and color corrected image are combined to obtain the restored underwater image through inverse transformation. Wherein, the multi-scale color compensation algorithm can effectively eliminate the color deviation of the underwater image; the secondary hierarchical search based on optical prior is adopted to estimate the global background light, which can effectively remove the influence of the high light target and suspended particles in the water on the estimation of the global background light; the adaptive dark channel algorithm is adopted to estimate the transmittance, which can ensure that the transmittance has better robustness and stability. Thus, the underwater image can be enhanced with high quality, and has good generalization, which can cope with the turbid and complex underwater scene in reality, and effectively improve the quality of various underwater degraded images. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 It is a method flowchart of the present application;

[0059] Figure 2 It is a schematic diagram of the application process of the embodiment;

[0060] Figure 3 The color correction results of the method of the present application and other existing methods for real underwater images are compared;

[0061] Figure 4 The color correction gray scale histograms of the method of the present application and other existing methods for real underwater images are compared;

[0062] Figure 5 The recovery effect comparison chart of the method of the present application and other existing methods;

[0063] Figure 6 The recovery effect chart of the method of the present application under different turbidity. DETAILED DESCRIPTION

[0064] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] EMBODIMENT

[0066] As shown in the figure, an underwater image restoration method based on multi-scale color compensation and adaptive deblurring includes the following steps: Figure 1

[0067] S1, input the original underwater image, and establish an underwater optical imaging model of the underwater image according to optical characteristics;

[0068] S2, use a multi-scale color compensation algorithm to remove color deviation of the underwater image to obtain a color-corrected image;

[0069] S3, estimate the global background light of the underwater image based on a secondary hierarchical search algorithm of optical priori;

[0070] S4, estimate the transmittance of the underwater image based on an adaptive dark channel algorithm;

[0071] S5, according to the established underwater optical imaging model, combined with the obtained global background light, transmittance and color-corrected image, the restored underwater image is obtained through inverse transformation.

[0072] The embodiment applies the above technical solution, as shown in the figure, mainly includes: Figure 2

[0073] Step 1, input the original underwater image, and establish an underwater optical imaging model of the underwater image according to optical characteristics:

[0074] I c (x)=J c (x)t c (x)+A c (x)(1-t c (x)),c∈{r,g,b} ​​

[0075] where x represents a pixel, I c (x) is the observed image, J c (x) is the recovered image, A c (x) is the global background light, t c (x) ∈ [0, 1] represents the transmittance, c is one of the three color channels: r, g, b.

[0076] Step 2, Color Correction: Using a multi-scale color compensation algorithm to remove the color deviation of underwater images.

[0077] First, the original underwater image is decomposed into three color channels: r, g, and b. The total pixel mean of each color channel is calculated, which is represented by the following formula:

[0078]

[0079] where H and W represent the height and width of the input image. According to the size of the total pixel mean, the three color channels are redefined: the largest total pixel mean is I l , the middle one is I m , and the smallest one is I s .

[0080] Then, the values of the three color channels are adjusted. For the large color channel I l , a simple linear transmission is used to increase its dynamic range, which is represented by:

[0081]

[0082] I l ' is the corrected large color channel, and are the maximum and minimum pixel values of the large color channel, respectively;

[0083] For the middle color channel I m , the compensation process is represented by:

[0084]

[0085] where I' m is the corrected middle color channel, and represent the average values of I l and I m channels.

[0086] For the most severely attenuated small color channel I s , the compensation process is represented by:

[0087]

[0088] where I s is the corrected small color channel, represents the average value of the small color channel.

[0089] Step 3, Estimate global background light: Estimate the global background light of underwater image based on secondary hierarchical search of optical prior.

[0090] Firstly, the color corrected underwater image is divided into four rectangular regions by using hierarchical search technique, and the region with the highest brightness is selected, where the brightness is represented as:

[0091]

[0092] where n is the total number of pixels in a sub-region, is the percentage of bright pixels, and ε is an adjustable parameter, which is set to 0.2 in this embodiment. S(a, b) represents a sigmoid function, and s is an empirical constant, which is set to s = 32 in this embodiment.

[0093] Then, a secondary hierarchical search is performed on the selected high brightness region, and the region with the highest blur degree is selected, where the blur degree is represented as:

[0094]

[0095] The regions obtained by the two hierarchical searches are taken as the candidate regions of the background light. Then, the dark channel image of the candidate region of the background light is calculated, the pixel values in the dark channel image are sorted from large to small, and the original image pixel points corresponding to the top 0.1% of the sorted pixel points are selected as candidate pixel points. When the average value of the blue channel of the image is greater than the average value of the green channel, the pixel point with the largest blue-to-red channel ratio in the candidate pixel points is searched as the global background light. When the average value of the green channel of the image is greater than the average value of the blue channel, the pixel point with the largest green channel ratio in the candidate pixel points is searched as the global background light.

[0096] Step 4, Estimate transmittance

[0097] Firstly, the average value of the red channel is compared with the threshold value σ. According to experience, the threshold value σ = 56 is set in this embodiment. If the average value of the red channel is less than σ, the transmittance is represented as:

[0098]

[0099] If the average value of the red channel is greater than σ, the transmittance of the red channel is first obtained by the underwater dark channel algorithm, which is represented as:

[0100]

[0101] Then the attenuation coefficient ratio of green-red channel and blue-red channel is introduced to represent the transmission of green and blue channel, denoted as:

[0102]

[0103] wherein, and are the attenuation coefficient ratio between green-red channel and blue-red channel, denoted as:

[0104]

[0105] Step 5, underwater image restoration

[0106] The obtained global background light A c (x) is substituted into the underwater optical imaging model, and a clear restored image is obtained by inverse transformation: c (x) and the color corrected image I c '(x) are substituted into the underwater optical imaging model, and a clear restored image is obtained by inverse transformation:

[0107]

[0108] As Figure 3 shown, the present embodiment provides the color correction results of the proposed method and other existing methods for real underwater images. From the experimental results, it can be seen that the Gray World introduces a slight red distortion, the White-patch cannot correct the blue distortion well, the Shade-of-Gray introduces a red deviation in blue-green images and green images, and the Gray-Edge color correction effect is weak. In contrast, the proposed method effectively corrects the image color distortion and is superior to other algorithms in contrast enhancement and details.

[0109] Figure 4 For Figure 3 the RGB histogram comparison results of each image, the proposed method and other existing methods show more similar peak values and trends on the three color channels, and the histogram distribution is more uniform.

[0110] Figure 5The recovery effect of the method of the present application is compared with other existing methods. The DCP method processes the three color channels of the underwater image by the same transmission mode, and the recovery effect is relatively weak. Although the UDCP method can eliminate the blur state of the original image, the synthesized image looks darker. The MSRCR algorithm is unstable and is prone to produce a darker effect. The L1 has the limitations of structural blur and unnatural color. The HLRP has a positive effect on color correction and contrast improvement, but an over-enhanced result can be seen in the first comparison image. The TIP can effectively restore the color and structure of the underwater image, but a red deviation appears in the enhanced image. In contrast, the method of the present application is superior to other existing methods in terms of structure and detail recovery, color correction, contrast enhancement, and artifact suppression.

[0111] As shown in Figure 6 The present embodiment also provides a recovery effect diagram of the method of the present application under different turbidity. First, the images of the cup under different turbidity conditions were taken in the laboratory water tank. Figure 6 The first row of images is the original image with turbidity of 2NTU-13NTU. It can be seen that the images become more and more blurred until they are almost completely invisible. The second row of images is the recovered image after processing by the method of the present application. In Figure 6 (a) and Figure 6 (b), the recovered image is relatively clear, with bright colors and high contrast. In Figure 6 (c)-(f), the captured images are quite turbid, but the color and outline of the cup can still be effectively recovered.

[0112] In summary, the present scheme adopts a multi-scale optical attenuation compensation color correction algorithm, which can eliminate the color deviation of underwater images to obtain more natural color effects. On this basis, an adaptive deblurring method for improving the visibility of underwater images is also proposed, which includes an underwater image global background light estimation algorithm based on multiple optical prior characteristics and an adaptive dark channel transmission rate estimation algorithm, which can effectively remove the influence of high-light targets and suspended particles in water on global background light estimation, and make the estimated transmission rate have better robustness and accuracy. Using the present scheme can effectively correct the color deviation of underwater images, enhance the contrast of images, and cope with underwater scenes of different degradation levels, with strong generalization ability.

Claims

1. An underwater image restoration method based on multi-scale color compensation and adaptive deblurring, characterized in that: The following steps are involved: S1. Input the original underwater image and establish an underwater optical imaging model of the underwater image based on the optical characteristics; S2, using a multi-scale color compensation algorithm to remove the color deviation of the underwater image and obtain a color-corrected image; S3, a quadratic hierarchical search algorithm based on optical priors to estimate the global background light of underwater images; S4, estimate the transmittance of underwater images based on the adaptive dark channel algorithm; S5. According to the established underwater optical imaging model, combined with the obtained global background light, transmittance and color-corrected image, a restored underwater image is obtained by inverse transformation; The underwater optical imaging model established in step S1 is: I c (x)=J c (x)t c (x)+A c (x)(1-t c (x)),c∈{r,g,b} Where x represents a pixel, I c (x) is the observed image, J c (x) is the restored image, A c (x) is the global background light, t c (x)∈[0,1] represents the transmittance, c is one of the three color channels r, g, b; Step S2 includes the following steps: S21, decomposing the original underwater image into three color channels, r, g, and b, and calculating the total pixel mean of each color channel. Based on the value of the total pixel mean, redefine the large color channel, the middle color channel, and the small color channel; S22, based on a multi-scale color compensation algorithm, performing compensation on the large color channel, the intermediate color channel, and the small color channel, respectively, to obtain a color-corrected image; The formula for calculating the total pixel mean of the color channel in step S21 is: Where H and W represent the height and width of the input image; The three color channels redefined according to the total pixel mean are I l , the middle one is I m , the smallest one is I s ; The specific process of step S22 is: For large color channels I l , linear transmission is used to increase its dynamic range, which is expressed as: Among them, I l ' is the large color channel after correction, and are the maximum pixel value and minimum pixel value of the large color channel respectively; For the intermediate color channel I m , the compensation process is expressed as: Among them, I' m is the corrected middle color channel, and Indicates I l and I m The average value of the channel; For the most severely attenuated small color channel I s , the compensation process is expressed as: Among them, I s ' is the corrected small color channel, Represents the average value of small color channels; Step S3 specifically includes the following steps: S31, using a hierarchical search technique to divide the color-corrected underwater image into four rectangular regions, and selecting the region with the highest brightness; S32, then performing a secondary hierarchical search on the selected high-brightness area, and selecting the area with the highest fuzziness; S33, using the areas obtained from the two hierarchical searches as candidate background light areas, calculating a dark channel map of the candidate background light areas, sorting the pixel values ​​in the dark channel map from large to small, and selecting the original image pixels corresponding to the top 0.1% of the pixels as candidate pixels; When the average value of the blue channel of the image is greater than the average value of the green channel, the pixel with the largest blue-red channel ratio is searched among the candidate pixels as the global background light; When the average value of the green channel of the image is greater than the average value of the blue channel, the pixel with the largest green channel ratio is searched among the candidate pixels as the global background light; The regional brightness calculation formula in step S31 is: The regional ambiguity calculation formula in step S32 is: Where n is the total number of pixels in a region, is the percentage of bright pixels, ε is an adjustable parameter, S(a,b) represents a sigmoid function, and s is a constant; The specific process of step S4 is: First, the average value of the red channel is compared with the value of the preset threshold σ. If the average value of the red channel is less than σ, the transmittance is expressed as: If the average value of the red channel is greater than σ, the transmittance of the red channel is first obtained by the underwater dark channel algorithm, which is expressed as: Then the attenuation coefficient ratio of the green-red channel to the blue-red channel is introduced to represent the transmission of the green and blue channels, which is expressed as: in, and are the ratios of the attenuation coefficients between the green and red channels and the blue and red channels, respectively.

2. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 1, characterized in that: The preset threshold σ is 56.

3. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 1, characterized in that: The specific process of step S5 is: substituting the obtained global background light, transmittance and color corrected image into the underwater optical imaging model, and obtaining the restored underwater image through inverse transformation as follows: Among them, A c (x) is the global background light, t c (x) is the transmittance, I c '(x) is the image after color correction.

Images