Underwater image restoration method based on multi-scale color compensation and adaptive deblurring

Abstract

The invention relates to an underwater image restoration method based on multi-scale color compensation and adaptive deblurring, and the method comprises the steps: inputting an original underwater image, and building an underwater optical imaging model of the underwater image according to the optical characteristics; removing color deviation of the underwater image by adopting a multi-scale color compensation algorithm to obtain an image after color correction; estimating global background light of the underwater image based on a secondary hierarchical search algorithm of optical prior; estimating the transmissivity of the underwater image based on an adaptive dark channel algorithm; and according to the established underwater optical imaging model, combining the obtained global background light, the transmissivity and the image after color correction, and obtaining a restored underwater image through inverse transformation. Compared with the prior art, the method has the advantages that the color deviation of the underwater image can be effectively corrected, the image contrast can be enhanced, the method can cope with underwater scenes with different degradation degrees, and the generalization ability is high.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image processing, and in particular to an underwater image restoration method based on multi-scale color compensation and adaptive deblurring. Background Art

[0002] In recent years, the role of the ocean in human development has become increasingly obvious. If we want to better develop and utilize the energy contained in the ocean, we must detect and analyze the underwater environment and seabed information. Images are important carriers of information. Underwater images play a cornerstone role in underwater target detection and underwater robots. The visual quality of underwater images plays an extremely important role in marine research such as marine military, marine environmental protection, and marine engineering construction. High-quality underwater images have laid an important foundation for human exploration of the mysterious depths of the ocean. However, due to the absorption and scattering effect of light in water and the absorption and scattering of light by suspended particles, underwater images often have low visibility, color deviation, low contrast, noise, and image blur. These shortcomings of underwater images have caused many limitations on visual perception analysis and actual underwater applications. Therefore, for marine scientific research and engineering applications, it is necessary to study an effective underwater image clarity method.

[0003] At present, underwater image sharpening methods mainly include image enhancement based on non-physical models, image restoration based on physical models, and data-driven methods. Among them, the image enhancement method based on non-physical models can process some underwater images with simple content and not seriously degraded. However, for harsh underwater environments, this method cannot make targeted automatic adjustments based on the physical characteristics of the underwater scene, and thus easily ignores the texture details of the image, resulting in problems such as oversaturation and color distortion in the enhanced image.

[0004] Data-driven methods often require a large amount of data and a long time to train, but most of the data is artificially synthesized and different from real-world underwater images. Therefore, the robustness and generalization ability of data-driven methods are limited.

[0005] The restoration method based on physical models uses priors or assumptions to estimate the transmitted light and ambient light of underwater scenes, and then inverts the physical model to obtain high-quality underwater images. Although this method is effective for dehazing in most cases, it ignores the light absorption effect of the underwater medium, resulting in limited restoration effect on underwater images. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an underwater image restoration method based on multi-scale color compensation and adaptive deblurring, which can effectively correct the color deviation of underwater images, enhance image contrast, and can cope with underwater scenes with different degrees of degradation and has strong generalization ability.

[0007] The object of the present invention can be achieved by the following technical solution: An underwater image restoration method based on multi-scale color compensation and adaptive deblurring, comprising the following steps:

[0008] S1, inputting the original underwater image, and establishing an underwater optical imaging model of the underwater image according to the optical characteristics;

[0009] S2, using a multi-scale color compensation algorithm to remove the color deviation of the underwater image and obtain a color-corrected image;

[0010] S3, a quadratic hierarchical search algorithm based on optical priors to estimate the global background light of underwater images;

[0011] S4, estimating the transmittance of underwater images based on the adaptive dark channel algorithm;

[0012] S5. According to the established underwater optical imaging model, the global background light, transmittance and color-corrected image are obtained, and the restored underwater image is obtained through inverse transformation.

[0013] Furthermore, the underwater optical imaging model established in step S1 is:

[0014] I c (x) = J c (x)t c (x)+A c (x)(1-t c (x)),c∈{r,g,b}

[0015] Where x represents a pixel, I c (x) is the observed image, J c (x) is the restored image, A c (x) is the global background light, t c (x)∈[0,1] represents the transmittance, and c is one of the three color channels: r, g, and b.

[0016] Furthermore, the step S2 comprises the following steps:

[0017] S21, decomposing the original underwater image into three color channels of r, g and b, and calculating the total pixel mean of each color channel, and redefining the large color channel, the middle color channel and the small color channel according to the value of the total pixel mean;

[0018] S22. Based on a multi-scale color compensation algorithm, compensation is performed on a large color channel, an intermediate color channel, and a small color channel respectively to obtain a color-corrected image.

[0019] Furthermore, the formula for calculating the total pixel mean of the color channel in step S21 is:

[0020]

[0021] Where H and W represent the height and width of the input image;

[0022] According to the three color channels redefined by the total pixel mean, the one with the largest total pixel mean is I l , the middle one is I m , the smallest one is I s .

[0023] Furthermore, the specific process of step S22 is as follows:

[0024] For large color channels I l , linear transmission is used to increase its dynamic range, expressed as:

[0025]

[0026] Among them, I l ' is the corrected large color channel, and They are the maximum pixel value and the minimum pixel value of the large color channel respectively;

[0027] For the intermediate color channel I m , the compensation process is expressed as:

[0028]

[0029] Among them, I' m is the corrected middle color channel, and Indicates I l and I m The average value of the channel;

[0030] For the most attenuated small color channel I s , the compensation process is expressed as:

[0031]

[0032] Among them, I s ' is the corrected small color channel, Represents the average of the minor color channels.

[0033] Furthermore, the step S3 specifically includes the following steps:

[0034] S31, using a hierarchical search technique to divide the color-corrected underwater image into four rectangular areas, and selecting the area with the highest brightness;

[0035] S32, then performing a secondary hierarchical search on the selected high-brightness area, and selecting the area with the highest fuzziness;

[0036] S33, taking the area obtained by the two hierarchical searches as the background light candidate area, calculating the dark channel map of the background light candidate area, sorting the pixel values ​​in the dark channel map from large to small, and selecting the original image pixel points corresponding to the top 0.1% of the pixel points as the candidate pixel points;

[0037] When the average value of the blue channel of the image is greater than the average value of the green channel, the pixel with the largest blue-red channel ratio is searched among the candidate pixels as the global background light;

[0038] When the average value of the green channel of the image is greater than the average value of the blue channel, a pixel with the largest green channel ratio is searched among the candidate pixels as the global background light.

[0039] Furthermore, the regional brightness calculation formula in step S31 is:

[0040]

[0041] The regional ambiguity calculation formula in step S32 is:

[0042]

[0043] Where n is the total number of pixels in a region, is the percentage of bright pixels, ε is an adjustable parameter, S(a,b) represents a sigmoid function, and s is a constant.

[0044] Furthermore, the specific process of step S4 is as follows:

[0045] First, the average value of the red channel is compared with the value of the preset threshold σ. If the average value of the red channel is less than σ, the transmittance is expressed as:

[0046]

[0047] If the average value of the red channel is greater than σ, the transmittance of the red channel is first obtained by the underwater dark channel algorithm, which is expressed as:

[0048]

[0049] Then the attenuation coefficient ratio of the green-red channel to the blue-red channel is introduced to represent the transmission of the green and blue channels, which is expressed as:

[0050]

[0051] in, and They are the ratios of the attenuation coefficients between the green and red channels and the blue and red channels, respectively.

[0052] Furthermore, the preset threshold σ is 56.

[0053] Furthermore, the specific process of step S5 is: substituting the obtained global background light, transmittance and color corrected image into the underwater optical imaging model, and obtaining the restored underwater image by inverse transformation:

[0054]

[0055] Among them, A c (x) is the global background light, t c (x) is the transmittance, I c '(x) is the image after color correction.

[0056] Compared with the prior art, the present invention has the following advantages:

[0057] The present invention first establishes an underwater optical imaging model of the underwater image for the original underwater image; then, on the one hand, a multi-scale color compensation algorithm is used to obtain a color-corrected image; on the other hand, a secondary hierarchical search algorithm based on optical priors is used to estimate the global background light of the underwater image, and an adaptive dark channel algorithm is used to estimate the transmittance of the underwater image; finally, according to the established underwater optical imaging model, the global background light, transmittance and color-corrected image are combined to obtain a restored underwater image through inverse transformation. Among them, the multi-scale color compensation algorithm is used to effectively eliminate the color deviation of the underwater image; the global backlight is estimated by a secondary hierarchical search based on optical priors, which can effectively remove the influence of highlight targets and suspended particles in the water on the global background light estimation; the transmittance is estimated by an adaptive dark channel algorithm, which can ensure that the transmittance has better robustness and stability. In this way, high-quality enhancement of underwater images can be achieved, and it has good generalization, can cope with turbid and complex underwater scenes in reality, and effectively improve the quality of various underwater degraded images. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0059] Figure 2 It is a schematic diagram of the application process of the embodiment;

[0060] Figure 3 The figure is a comparison diagram of the color correction results of the method of the present invention and other existing methods for real underwater images;

[0061] Figure 4 The figure is a comparison diagram of the grayscale histogram of the color correction of the method of the present invention and other existing methods for real underwater images;

[0062] Figure 5 A comparison chart of the restoration effects of the method of the present invention and other existing methods;

[0063] Figure 6 Schematic diagram of the restoration effect of the method of the present invention under different turbidities. DETAILED DESCRIPTION

[0064] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] Example

[0066] like Figure 1 As shown, an underwater image restoration method based on multi-scale color compensation and adaptive deblurring includes the following steps:

[0067] S1, inputting the original underwater image, and establishing an underwater optical imaging model of the underwater image according to the optical characteristics;

[0068] S2, using a multi-scale color compensation algorithm to remove the color deviation of the underwater image and obtain a color-corrected image;

[0069] S3, a quadratic hierarchical search algorithm based on optical priors to estimate the global background light of underwater images;

[0070] S4, estimating the transmittance of underwater images based on the adaptive dark channel algorithm;

[0071] S5. According to the established underwater optical imaging model, the global background light, transmittance and color-corrected image are obtained, and the restored underwater image is obtained through inverse transformation.

[0072] This embodiment applies the above technical solution, such as Figure 2 As shown, the main ones are:

[0073] Step 1: Input the original underwater image and establish an underwater optical imaging model to obtain the underwater image based on the optical characteristics:

[0074] I c (x) = J c (x)t c (x)+A c (x)(1-t c (x)),c∈{r,g,b}

[0075] Where: x represents a pixel, I c (x) is the observed image, J c (x) is the restored image, A c (x) is the global background light, t c (x)∈[0,1] represents the transmittance, and c is one of the three color channels: r, g, b.

[0076] Step 2: Color correction: Use a multi-scale color compensation algorithm to remove the color deviation of the underwater image.

[0077] First, the original underwater image is decomposed into three color channels: r, g, and b, and the total pixel mean of each color channel is calculated. The formula is as follows:

[0078]

[0079] Among them, H and W represent the height and width of the input image; according to the size of the total pixel mean, the three color channels are redefined: the one with the largest total pixel mean is I l , the middle one is I m , the smallest one is I s .

[0080] Then re-adjust the values ​​of the three color channels. For the large color channel I l , a simple linear transmission is used to increase its dynamic range, which is expressed as:

[0081]

[0082] I l ' is the corrected large color channel, and They are the maximum pixel value and the minimum pixel value of the large color channel respectively;

[0083] For the intermediate color channel I m , the compensation process is expressed as:

[0084]

[0085] Among them, I' m is the corrected middle color channel, and I l and I m The average value of the channel.

[0086] For the most attenuated small color channel I s , the compensation process is expressed as:

[0087]

[0088] Among them, I s ' is the corrected small color channel, Represents the average of the minor color channels.

[0089] Step 3: Estimate the global background light: Estimate the global background light of the underwater image based on a secondary hierarchical search based on optical priors.

[0090] First, the color-corrected underwater image is divided into four rectangular regions using a hierarchical search technique, and the region with the highest brightness is selected. The brightness is expressed as:

[0091]

[0092] Where n is the total number of pixels in a sub-region, is the percentage of bright pixels, ε is an adjustable parameter, which is set to 0.2 in this embodiment. S(a,b) represents a sigmoid function, s is an empirical constant, which is set to s=32 in this embodiment.

[0093] Then, a secondary hierarchical search is performed on the selected high-brightness area to select the area with the highest fuzziness. The fuzziness is expressed as:

[0094]

[0095] The area obtained by the two hierarchical searches is used as the candidate area for background light. Then, the dark channel map of the candidate area for background light is calculated, the pixel values ​​in the dark channel map are sorted from large to small, and the original image pixels corresponding to the top 0.1% of the pixels are selected as candidate pixels; when the average value of the blue channel of the image is greater than the average value of the green channel, the pixel with the largest blue-red channel ratio is searched among the candidate pixels as the global background light. When the average value of the green channel of the image is greater than the average value of the blue channel, the pixel with the largest green channel ratio is searched among the candidate pixels as the global background light.

[0096] Step 4: Estimate Transmittance

[0097] First, the average value of the red channel is compared with the value of the threshold σ. Based on experience, the threshold σ is set to 56 in this embodiment. If the average value of the red channel is less than σ, the transmittance is expressed as:

[0098]

[0099] If the average value of the red channel is greater than σ, the transmittance of the red channel is first obtained by the underwater dark channel algorithm, which is expressed as:

[0100]

[0101] Then the attenuation coefficient ratio of the green-red channel to the blue-red channel is introduced to represent the transmission of the green and blue channels, which is expressed as:

[0102]

[0103] in, and They are the ratios of the attenuation coefficients between the green and red channels and the blue and red channels, expressed as:

[0104]

[0105] Step 5: Underwater image restoration

[0106] The global background light A c (x), transmittance t c (x) and the color-corrected image I c Substitute '(x) into the underwater optical imaging model and obtain a clear restored image through inverse transformation:

[0107]

[0108] like Figure 3 As shown, this embodiment provides the color correction results of the proposed method and other existing methods for real underwater images. From the experimental results, it can be seen that Gray World introduces reddish distortion, White-patch cannot correct blue distortion well, Shade-of-Gray introduces red deviation in cyan and green images, and Gray-Edge color correction effect is weak. In contrast, the proposed method effectively corrects image color distortion and is superior to other algorithms in terms of contrast enhancement and details.

[0109] Figure 4 for Figure 3 The RGB histogram comparison results of each image in the RGB histogram show that compared with other existing methods, the method of the present invention shows more similar peaks and trends in the three color channels, and the histogram distribution is more uniform.

[0110] Figure 5The figure shows the comparison results of the restoration effects of the method of the present invention and other existing methods. The DCP method processes the three color channels of the underwater image through the same transmission method, and the restoration effect is relatively weak. Although the UDCP method can eliminate the blur state of the original image, the synthesized image looks darker. The MSRCR algorithm is unstable and easily produces a darker effect. L1 has the limitations of structural blur and unnatural colors. HLRP has a positive effect on color correction and contrast improvement, but an over-enhanced result can be seen in the first comparison image. TIP can effectively restore the color and structure of the underwater image, but a red deviation appears in the enhanced image. In comparison, the method of the present invention is superior to other existing methods in terms of structure and detail restoration, color correction, contrast enhancement and artifact suppression.

[0111] like Figure 6 As shown, this embodiment also provides the image restoration effect diagram of the method of the present invention under different turbidity conditions. First, images of cups under different turbidity conditions were taken in a laboratory water tank. Figure 6 The first row of images in the figure are original images with turbidity of 2NTU to 13NTU. It can be seen that these images become increasingly blurred until they are almost completely invisible. The second row of images are restored images after being processed using the method of the present invention. Figure 6 (a) and Figure 6 In (b), the restored image is relatively clear, with vivid colors and high contrast. Figure 6 In (c)-(f), the captured images are quite turbid, but the color and outline of the cup are still effectively restored.

[0112] In summary, this scheme adopts a multi-scale optical attenuation compensation color correction algorithm, which can eliminate the color deviation of underwater images to obtain a more natural color effect. On this basis, an adaptive deblurring method for improving the visibility of underwater images is proposed. The method includes an underwater image global background light estimation algorithm based on multiple optical prior characteristics and an adaptive dark channel transmittance estimation algorithm, which can effectively remove the influence of highlighted targets and suspended particles in the water on the global background light estimation, and make the estimated transmittance have better robustness and accuracy. This scheme can effectively correct the color deviation of underwater images, enhance image contrast, and can cope with underwater scenes with different degrees of degradation, with strong generalization ability.

Claims

1. An underwater image restoration method based on multi-scale color compensation and adaptive deblurring, characterized in that: The following steps are involved: S1, inputting the original underwater image, and establishing an underwater optical imaging model of the underwater image according to the optical characteristics; S2, using a multi-scale color compensation algorithm to remove the color deviation of the underwater image and obtain a color-corrected image; S3, a quadratic hierarchical search algorithm based on optical priors to estimate the global background light of underwater images; S4, estimating the transmittance of underwater images based on the adaptive dark channel algorithm; S5. According to the established underwater optical imaging model, the global background light, transmittance and color-corrected image are obtained, and the restored underwater image is obtained through inverse transformation.

2. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 1, characterized in that: The underwater optical imaging model established in step S1 is: I c (x)=J c (x)t c (x)+A c (x)(1-t c (x)),c∈{r,g,b} Where x represents a pixel, I c (x) is the observed image, J c (x) is the restored image, A c (x) is the global background light, t c (x)∈[0,1] represents the transmittance, and c is one of the three color channels: r, g, and b.

3. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 2, characterized in that: The step S2 comprises the following steps: S21, decomposing the original underwater image into three color channels of r, g and b, and calculating the total pixel mean of each color channel, and redefining the large color channel, the middle color channel and the small color channel according to the value of the total pixel mean; S22. Based on a multi-scale color compensation algorithm, compensation is performed on a large color channel, an intermediate color channel, and a small color channel respectively to obtain a color-corrected image.

4. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 3, characterized in that: The formula for calculating the total pixel mean of the color channel in step S21 is: Where H and W represent the height and width of the input image; According to the three color channels redefined by the total pixel mean, the one with the largest total pixel mean is I l , the middle one is I m , the smallest one is I s .

5. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 4, characterized in that: The specific process of step S22 is as follows: For large color channels I l , linear transmission is used to increase its dynamic range, expressed as: Among them, I l ' is the corrected large color channel, and They are the maximum pixel value and the minimum pixel value of the large color channel respectively; For the intermediate color channel I m , the compensation process is expressed as: Among them, I' m is the corrected middle color channel, and Indicates I l and I m The average value of the channel; For the most attenuated small color channel I s , the compensation process is expressed as: Among them, I s ' is the corrected small color channel, Represents the average of the minor color channels.

6. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 4, characterized in that: The step S3 specifically comprises the following steps: S31, using a hierarchical search technique to divide the color-corrected underwater image into four rectangular areas, and selecting the area with the highest brightness; S32, then performing a secondary hierarchical search on the selected high-brightness area, and selecting the area with the highest fuzziness; S33, taking the area obtained by the two hierarchical searches as the background light candidate area, calculating the dark channel map of the background light candidate area, sorting the pixel values ​​in the dark channel map from large to small, and selecting the original image pixel points corresponding to the top 0.1% of the pixel points as the candidate pixel points; When the average value of the blue channel of the image is greater than the average value of the green channel, the pixel with the largest blue-red channel ratio is searched among the candidate pixels as the global background light; When the average value of the green channel of the image is greater than the average value of the blue channel, a pixel with the largest green channel ratio is searched among the candidate pixels as the global background light.

7. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 6, characterized in that: The regional brightness calculation formula in step S31 is: The regional ambiguity calculation formula in step S32 is: Where n is the total number of pixels in a region, is the percentage of bright pixels, ε is an adjustable parameter, S(a,b) represents a sigmoid function, and s is a constant.

8. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 4, characterized in that: The specific process of step S4 is as follows: First, the average value of the red channel is compared with the value of the preset threshold σ. If the average value of the red channel is less than σ, the transmittance is expressed as: If the average value of the red channel is greater than σ, the transmittance of the red channel is first obtained by the underwater dark channel algorithm, which is expressed as: Then the attenuation coefficient ratio of the green-red channel to the blue-red channel is introduced to represent the transmission of the green and blue channels, which is expressed as: in, and They are the ratios of the attenuation coefficients between the green and red channels and the blue and red channels, respectively.

9. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 8, characterized in that: The preset threshold σ is 56.

10. The underwater image restoration method based on multi-scale color compensation and adaptive deblurring according to claim 8, characterized in that: The specific process of step S5 is: substituting the obtained global background light, transmittance and color corrected image into the underwater optical imaging model, and obtaining the restored underwater image through inverse transformation: Among them, A c (x) is the global background light, t c (x) is the transmittance, I c '(x) is the image after color correction.

Images