A method for underwater image enhancement and restoration

By improving the color migration algorithm and the adaptive light attenuation curve prior method, the problems of underwater image detail loss and low contrast are solved, significantly improving the restoration quality of underwater images, and achieving better color enhancement and detail retention.

CN114820336BActive Publication Date: 2024-11-19WUXI TUCHUANG INTELLIGENT TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111611055.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-27
Publication Date
2024-11-19
Estimated Expiration
2041-12-27

AI Technical Summary

Technical Problem

Existing underwater image enhancement and restoration technologies cannot effectively solve the problems of image details, low contrast, color shift and uneven brightness, resulting in low restoration quality.

Method used

The original underwater image was preprocessed using an improved color migration algorithm to estimate the transmittance of underwater background light and light rays, and image recovery was performed using an adaptive light attenuation curve prior method.

Benefits of technology

It improves the contrast and visualization of the image, effectively removes noise, reduces color shift and uneven brightness, and significantly improves the image restoration quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114820336B_ABST
    Figure CN114820336B_ABST
Patent Text Reader

Abstract

The present invention discloses an underwater image enhancement and restoration method, which can effectively improve the contrast of the enhanced image and improve the restoration quality of the underwater image. The method comprises: acquiring an original underwater image and providing an initial reference image at the same time; based on the initial reference image, using an improved color migration algorithm to preprocess a basic image in the original underwater image to obtain a preprocessed image; estimating the transmittance of underwater background light and underwater light in the preprocessed image; and restoring the estimated image using an adaptive light attenuation curve prior method to obtain a restored image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of machine vision, and in particular to an underwater image enhancement and restoration method. Background Art

[0002] In underwater operations such as offshore oil drilling, submarine tunnels, pipeline construction, underwater archaeology, biological research, and lifesaving, optical cameras are often combined with underwater technologies such as laser scanning and polarized light to collect underwater images, and then the underwater images are used to assist in analyzing the underwater operation situation. However, during the image collection process, the water body is prone to refract, scatter, and absorb light, resulting in poor quality of the final underwater image. The special physical and chemical properties of the underwater cause underwater images to exhibit unique degradation characteristics, such as: (1) The attenuation rates of red, green, and blue light are different, and red light is most severely attenuated underwater (Figure 1). The image shows a serious blue-green cast; (2) Particles suspended in the water absorb the energy of light and change the optical path. The reflected light of the object is forward scattered, making the image of the object blurred, and the scattered light of non-target objects is backward scattered, making the picture covered with a veil, which leads to low contrast, blur and fog in the image; (3) Artificial light sources are often introduced during underwater photography. In addition to being affected by absorption and scattering, these uneven light sources will also cause bright spots to appear in the center of the image; in addition, the "marine snow" effect caused by suspended particles or bubbles in the water will also affect the imaging, causing bright spots or false features to appear in the picture.

[0003] To this end, image processing technology is often used to process the collected images in order to obtain clear underwater images. Currently, commonly used image processing technologies include image enhancement and restoration technology.

[0004] Underwater image enhancement and restoration technology is mainly based on physical and mathematical methods. Currently, the commonly used underwater image enhancement and restoration method is the single image restoration method of the physical imaging model. Its basic principle is to build a physical model of underwater imaging based on the degradation process of underwater images to restore clear images. Since underwater images are similar to foggy images and both involve light reflection and scattering, dark channel prior dehazing algorithms can be used to restore underwater images. However, the selective absorption of light by water makes the dark channel value obtained directly using this type of method too small, the estimated transmittance too large, and the restored result too dark. To solve the above problems, the industry has proposed underwater image restoration algorithms, deblurring algorithms, and transmittance optimization with color distortion removal and dark channel prior. The underwater image restoration algorithm uses grayscale to remove color distortion and then uses dark channel prior for underwater enhancement; the deblurring algorithm uses dark color prior to deblur underwater images and then uses color quantization to achieve color correction of the image and restore color balance; the transmittance optimization considers that water has the highest absorption rate for red light, uses dark channel prior for the imaging model of the blue-green channel, and then uses joint triangular filtering to optimize the transmittance. Chiang et al. added parameters reflecting the relationship between wavelength, depth, and image degradation to the underwater imaging model to obtain clear images.

[0005] However, all of the above methods are based on the restoration principle of local dark channel prior. Not only is the estimation of background light easily disturbed by white objects, but it is also unable to accurately estimate the transmittance of white objects in the foreground, resulting in low restoration quality. On the other hand, the image restoration method based on non-local color prior uses the non-local characteristics of corresponding lines in the color space after the pixels with similar colors in the clear image are degraded (i.e., the RGB values ​​of the pixels have changed) to estimate the transmittance. This can effectively restore the image, but the algorithm is only applicable to atmospheric imaging models, not underwater imaging models. Summary of the invention

[0006] In order to solve the problems of image detail loss and low contrast in the original underwater image, as well as the problem of low image restoration quality caused by color cast or uneven brightness, the present invention provides an underwater image enhancement and restoration method, which can effectively improve the contrast of the enhanced image and improve the restoration quality of the underwater image.

[0007] To achieve the above object, the present invention adopts the following technical solution:

[0008] An underwater image enhancement and restoration method, characterized in that the method comprises: S1, collecting an original underwater image and providing an initial reference image;

[0009] S2. Based on the initial reference image, preprocess the basic image in the original underwater image using an improved color migration algorithm to obtain a preprocessed image;

[0010] S3, estimating the transmittance of underwater background light and underwater light in the preprocessed image;

[0011] S4. Use an adaptive light attenuation curve prior method to restore the estimated image to obtain a restored image.

[0012] Its further characteristic is that

[0013] Step S2 includes: S21, correcting the initial reference image to obtain a corrected reference image;

[0014] S22, the improved color migration algorithm is a combination of the modified reference image and the color migration algorithm, that is, the modified image is used as model input information of the color migration algorithm;

[0015] Further, step S3 includes: S31, estimating the underwater background light;

[0016] S32, calculate the color channel attenuation rate λ c′ ;

[0017] S33, based on the color channel attenuation rate λ c′ , estimating the transmittance of the underwater light, including: an initial relative transmittance estimation of the red channel;

[0018] Estimation of the attenuation coefficient Kc;

[0019] Adjust the transmission estimate of the three RGB channels.

[0020] The above method of the present invention can achieve the following beneficial effects: in this method, before restoring the original underwater image, the original underwater image is preprocessed using an initial reference image and an improved color migration algorithm, so that the color contrast and visualization effect of the underwater image are improved, thereby achieving color enhancement of the underwater image, while well preserving the detail features in the image, which is conducive to subsequent image restoration.

[0021] The transmittance of underwater background light and underwater light in the preprocessed image is estimated, and the estimated image is restored using the adaptive light attenuation curve method to obtain the restored image. Among them, the estimation of the transmittance of underwater background light and underwater light can effectively remove noise and reduce the impact of color cast or uneven brightness on the image, thereby improving the subsequent image restoration effect and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the effect diagram of various types of degraded images of underwater images taken underwater, among which, Figure 1 (a) is the effect of the original underwater image blurred due to the absorption and scattering of ambient light; Figure 1(b) is the effect of the original underwater image due to the yellowish color caused by the absorption and scattering of ambient light; Figure 1 (c) is the effect of the original underwater image with a green tint due to the absorption and scattering of ambient light; Figure 1 (d) is the effect diagram of the original underwater image with uneven image brightness caused by artificial light source;

[0023] Figure 2 This is a flowchart of image preprocessing based on the improved color migration algorithm of the present invention;

[0024] Figure 3 This is a comparison chart of the effects of image preprocessing based on the improved color migration algorithm of the present invention, where: Figure 3 (a) is a randomly selected original underwater image. Figure 3 (b) is the detail image D, Figure 3 (c) is the salient image S, Figure 3 (d) is the original reference image, Figure 3 (e) is the processing effect of common color migration algorithm. Figure 3 (f) is the corrected reference image R, Figure 3 (g) is the image processing result based on the improved color migration algorithm proposed by the present invention;

[0025] Figure 4 is a complete flow chart for realizing the adaptive light attenuation curve a priori method proposed by the present invention;

[0026] Figure 5 is a comparison diagram of the effects of the image restoration method of the present invention and the fixed light attenuation curve method, wherein Figure 5 (a1) and (b1) are randomly selected original underwater images. Figure 5 (a2) and (b2) are images restored using the fixed light attenuation curve method. Figure 5 (a3) and (b3) are images restored using the adaptive light attenuation curve prior method of the present invention. DETAILED DESCRIPTION

[0027] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0028] Figure 1 These are various types of degraded images taken underwater. Figure 1 (a) is a blurred underwater image. The water medium absorbs the direct component and scatters it. The absorption and scattering process of the ambient light is superimposed, resulting in loss of image details, reduced contrast and brightness. The uneven attenuation of different colored lights by the water medium causes the underwater image to show a strong color cast, such as Figure 1 (b) and Figure 1 (c) The underwater image is yellowish and greenish. Due to the different absorption and scattering coefficients of light of different wavelengths underwater, red light has the longest wavelength and the greatest attenuation, making the underwater image appear Figure 1 (b) The yellowish effect is shown in the figure; the blue light has a shorter wavelength and a smaller attenuation, making the underwater image appear Figure 1 (c) shows the blue effect. Figure 1 (d) The uneven brightness of underwater images caused by artificial light sources. As the underwater depth increases, the intensity of natural light decreases significantly, making it difficult to evenly illuminate the target object. Artificial light sources are often used to assist imaging in deep-water photography. However, these artificial light sources are non-uniform, causing serious uneven brightness distribution in the image.

[0029] In view of the problems of image detail loss and low contrast in the above-mentioned original underwater images, as well as the problem of low image restoration quality caused by color cast or uneven brightness, the present application provides a specific embodiment of an underwater image enhancement and restoration method, the method comprising: S1, acquiring the original underwater image and providing an initial reference image at the same time; S2, based on the initial reference image, using an improved color migration algorithm to preprocess the basic image in the original underwater image to obtain a preprocessed image; S3, estimating the transmittance of underwater background light and underwater light in the preprocessed image; S4, using an adaptive light attenuation curve prior method to restore the estimated image to obtain a restored image.

[0030] The original underwater images are collected by camera, see Figure 1 The original underwater image, i.e., the original underwater image I, is composed of a significant image S, a basic image B and a detail image D. The original underwater image I is processed by an edge-preserving smoothing method to obtain a basic image B. In this embodiment, the edge-preserving smoothing method is a bilateral filtering method. The bilateral filtering method is a nonlinear filtering method. It combines the spatial proximity of the image with the spatial information of the pixels and the grayscale similarity to achieve the purpose of edge-preserving denoising. Therefore, the basic image B obtained by this method contains the edge data of the original image, further preventing the problem of reduced image contrast caused by blurred foreground edges. The detail image D is equal to the original image I minus the basic image B. The significant image S is obtained by processing the original image I by using the Achanta algorithm.

[0031] The acquired original underwater images are first preprocessed using an improved color migration algorithm, such as Figure 2 The processing flow listed in the figure shows that the color migration algorithm only operates on the base image B, and in the color migration algorithm, a reference image is required. In common algorithms, the initial reference image R is mainly used. ini In the present invention, S21, the initial reference image is corrected to obtain a corrected reference image R, and S22, the reference image R is combined with the base image B, and a color migration algorithm is applied to obtain a better image preprocessing effect. The calculation formula of the reference image R is as follows:

[0032] R=R ini +D+S*I (1)

[0033] Figure 3 A comparison chart of the effects of image preprocessing based on the improved color migration algorithm is given in the figure. Figure 3 (a) is a randomly selected original underwater image. Figure 3 (b) is the detail image D, Figure 3 (c) is the salient image S, Figure 3 (d) is the original reference image, Figure 3 (e) is the processing effect of common color migration algorithm. Figure 3 (f) is the corrected reference image R, Figure 3 (g) is the image processing result based on the improved color migration algorithm proposed by the present invention. Figure 3 (e) with Figure 3 (g) By comparison, it can be seen that the pre-processed image obtained by processing the original underwater image with the improved color migration algorithm in the method of the present invention has better color contrast and visualization effect. This pre-processing can effectively enhance the color and well preserve the detailed features in the image, which is conducive to subsequent image restoration.

[0034] Figure 4 The complete flow chart of the adaptive light attenuation curve prior method proposed in the present invention is given in FIG. The specific steps of the adaptive light attenuation curve prior method for image restoration include: S31. According to the classic Jaffe-McGlamey simplified model of underwater imaging, the final image Ic(x) of the target in the camera is composed of the superposition of the direct component, the forward scattered component, and the backscattered component, as shown in formula (2):

[0035] I c (x) = t c (x)J c (x)+A c (1-t c (x)) (2)

[0036] Where x is the coordinate of the pixel in the image, J c(x) is the undistorted image, c = {R, G, B} is the red, green and blue channels of the image, t c (x) is the transmittance of light in channel c, which represents the absorption and reflection of light in the medium. c is the ambient light intensity, t c (x)J c (x) is the direct component of light propagating in water, A c (1-t c (x)) is the backscatter component. Since the forward scatter component has little effect on image degradation, it can be ignored.

[0037] Underwater background light refers to the uniform background light formed by backscattering. The background light B can be estimated by formula (3):

[0038] B c =I c (arg max I r (x)), c∈{r,g,b}

[0039] Among them, I r (x) is 1 c (x) Red channel image. In the present invention, we regard the pixel with the maximum value in the red channel as the background light pixel.

[0040] S32, attenuation rate λ of color channel c 'for

[0041]

[0042]

[0043] Among them, β=a+s, which represents the sum of the absorption coefficient a and the scattering coefficient s; Λ represents the wavelength of light, the wavelength of red light is 620nm, the wavelength of green light is 540nm, and the wavelength of blue light is 450nm.

[0044] S33. Initial relative transmission estimate of the red channel As formula (6)

[0045]

[0046] Attenuation coefficient K c The estimation of is as follows:

[0047]

[0048] Adjust the transmission estimation of the three RGB channels. The expression is as follows:

[0049]

[0050] where λ′c Belong to [λ c ,1],λ c =max(λ g ,λ b ), ε c is the tolerance value set to ensure a high contrast ratio.

[0051] Finally, the image is restored as follows:

[0052]

[0053] The obtained J c (x) is the restored image.

[0054] like Figure 5 (a1) and (b1) are randomly selected underwater images. Figure 5 (a2) and (b2) are images restored using the fixed light attenuation curve method. Figure 5 (a3) and (b3) are images restored by the adaptive light attenuation curve method of the present invention. Figure 5 (a3), (b3) It can be seen that the present invention can effectively remove noise and reduce the impact of color cast or uneven brightness on the image through the estimation of the transmittance of underwater background light and underwater light, thereby improving the subsequent image restoration effect and quality. The method of the present invention can more effectively remove noise and achieve better visual effects. It also effectively improves the contrast of the enhanced image and the restoration quality of the underwater image, which is conducive to the subsequent analysis of underwater operations through underwater images.

[0055] The above are only preferred embodiments of the present application, and the present invention is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present invention should be considered to be included in the protection scope of the present invention.

Claims

1. A method for underwater image enhancement and restoration, characterized in that: The method comprises: acquiring an original underwater image and providing an initial reference image; Based on the initial reference image, a color migration algorithm is used to process the original underwater image to obtain a basic image, and the basic image is preprocessed to obtain a preprocessed image; estimating the transmittance of underwater background light and underwater light in the preprocessed image; The estimated image is restored using an adaptive light attenuation curve prior method to obtain a restored image; The original underwater image is processed by using an Achanta algorithm to obtain a significant image, and the original underwater image is processed by using an edge-preserving smoothing method to obtain the basic image, wherein the detail image is equal to the original underwater image minus the basic image; Based on the initial reference image, a color migration algorithm is used to pre-process the basic image B, the steps comprising: S21, correcting the initial reference image to obtain a corrected reference image; S22, the corrected reference image is combined with the basic image B, and after being processed by a color migration algorithm, the detail image is fused to obtain a preprocessed image; In step S21, the initial reference image is corrected using the following formula: R=R ini +D+S*I; Wherein, R is the corrected reference image, R ini is the initial reference image, D is the detail image, S is the salient image, and I is the original underwater image.

2. The underwater image enhancement and restoration method according to claim 1, characterized in that: The step of estimating the transmittance of underwater background light and underwater light in the preprocessed image comprises: S31, estimating the underwater background light; S32, calculate the color channel attenuation rate λ c′ ; S33, based on the color channel attenuation rate λ c′ , estimating the transmittance of the underwater light, including: an initial relative transmittance estimation of a color channel; Estimation of the attenuation coefficient K; Adjust the transmission estimation of the three RGB channels; In step S33, the transmittance of the underwater light is estimated, including: Initial relative transmission estimate for color channels Estimation of the attenuation coefficient K, Adjust the transmission estimation of the three RGB channels. The expression is as follows: where λ′ c Belong to [λ c ,1],λ c =max(λ g ,λ b ), ε c is the tolerance value set; Represents the initial relative transmission estimate of the red channel.

3. The underwater image enhancement and restoration method according to claim 2, characterized in that: In step S31, according to the classic Jaffe-McGlamey simplified model of underwater imaging, the final image of the target in the camera is c (x) is composed of the direct component, the forward scattered component, and the back scattered component. The final image I c The calculation formula for (x) is: I c (x)=t c (x)J c (x)+A c (1-t c (x)); Where x is the coordinate of the pixel in the image, J c (x) is the undistorted image, c = {R, G, B} is the red, green and blue channels of the image, t c (x) is the transmittance of light in channel c, which represents the absorption and reflection of light in the medium. c is the ambient light intensity, t c (x)J c (x) is the direct component of light propagating in water, A c (1-t c (x)) is the backscattered component; Underwater background light refers to the uniform background light formed by backscattering. c It is estimated by the following formula B c =I c (arg max I r (x)),c∈{r,g,b}, Among them, I r (x) is 1 c (x) The red channel image, the pixel with the maximum value in the red channel is taken as the background light pixel.

4. The underwater image enhancement and restoration method according to claim 3, characterized in that: In step S32, the color channel attenuation rate λ c′ The calculation formula is:

5. The underwater image enhancement and restoration method according to claim 1 or 2, characterized in that: The calculation formula of the adaptive light attenuation curve prior method is: Among them, J c (x) is the restored image, I c (x) represents the final image formed by the superposition of the direct component, the forward scattered component, and the backscattered component. c represents background light, K represents attenuation coefficient, represents the initial relative transmission estimate of the red channel, λ c ′ represents the color channel attenuation rate, and t0 represents a constant.

Citation Information

Patent Citations

  • Underwater image restoration method based on dark channel prior and sparse representation

    CN107203977A

  • UUV near sea surface visible light image defogging method

    CN110211067A

  • Underwater image restoration method based on texture color features and optimized transmissivity

    CN110689490A

  • Image color correction method based on dual-transmittance underwater imaging model

    CN111833258A

  • Illuminant Estimation

    US20130129204A1