A multi-scale statistical histogram enhancement method for underwater images

Abstract

The present invention relates to a method for multi-scale statistical histogram enhancement of underwater images, and belongs to the technical field of underwater image processing. The method comprises: converting an underwater image from an RGB color space into a CIE-Lab color space; processing a brightness component L of the CIE-Lab color space using a CLAHE algorithm; sequentially taking values ​​from the range of histogram bin number values, dividing the grayscale of the processed brightness component L into equal intervals according to the current value of the histogram bin number, performing interval equalization histogram equalization, obtaining the brightness component L under each value of the histogram bin number, performing color space inverse conversion, and obtaining an enhanced image under each value; performing quality assessment on the enhanced image under each value, and taking the image with the best quality assessment result as the enhanced image. The method of the present invention effectively improves the contrast and brightness of underwater images, improves the blue-green bias of images, and enables underwater images to achieve a more natural enhancement effect.

Description

Technical Field

[0001] The invention relates to an underwater image multi-scale statistical histogram enhancement method, belonging to the technical field of underwater image processing. Background Art

[0002] The ocean occupies two-thirds of the earth's area. With the development of economy and population, land resources are being consumed rapidly. Today, many scientists have begun to turn their attention to the development of marine resources, such as ocean exploration, development of oil and gas, underwater fishing, underwater flora and fauna, and submarine archaeology, which all require a clear understanding of underwater conditions. Due to the special physical and chemical properties of underwater, underwater images show unique degradation characteristics. Red, green, and blue light have different attenuation rates in water, among which red light attenuates the fastest and blue-green light attenuates the slowest. In addition, suspended matter in the water absorbs the energy of light and changes the optical path. Therefore, underwater images will show a serious blue-green bias and low contrast and clarity, which will affect the progress of underwater exploration. Therefore, it is very valuable to conduct in-depth research on underwater image enhancement methods.

[0003] Some underwater image enhancement methods have been proposed in the prior art. In 1979, Jaff-McGlamery proposed a classic underwater imaging model. He proposed that the final image of the target is the sum of the direct component, the forward scattered component, and the backscattered component. Based on this theoretical model, in 2009, He et al. proposed a defogging method based on dark channel prior, using the dark channel to estimate the haze. In 2008, Joshi et al. proposed an underwater image enhancement method based on Retinex theory, which enhanced the underwater image to a certain extent. In 2001, Reinhard et al. first applied color gamut mapping to underwater image enhancement, matching the global color distribution mean and standard deviation of the image in the CIE-Lab color space. In 2011, Zhang Kai et al. equivalentized the water scattering effect to the change of ambient light, and reduced the water scattering effect and improved the image contrast by processing the multi-scale Retinex algorithm under the brightness channel of the underwater color image. The Chinese invention patent with the announcement number CN 107507145 B discloses an underwater image enhancement method based on adaptive histogram stretching in different color spaces. After the color of the G and B channels in the underwater image is equalized, the method performs adaptive histogram stretching and bilateral filtering in the RGB color space, and then converts the RGB color space into the CIE-Lab color space, and adaptively stretches the three components of L, a, and b respectively, and finally outputs the image. The above image enhancement method improves the contrast and clarity of the image to a certain extent, but the imaging effect is still not ideal. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-scale statistical histogram enhancement method for underwater images, so as to solve the problems of color degradation caused by different attenuation rates of light of different wavelengths underwater, and the problem that suspended matter in the water absorbs the energy of light and changes the light path, resulting in unsatisfactory underwater imaging and underwater color distortion.

[0005] The present invention provides a method for enhancing underwater image multi-scale statistical histogram to solve the above technical problems. The method comprises the following steps:

[0006] 1) Perform color space conversion on the underwater color image, convert the RGB color space into the CIE-Lab color space, and obtain the brightness component L and the chromaticity components a and b;

[0007] 2) performing contrast-limited adaptive histogram equalization processing on the brightness component L;

[0008] 3) taking values ​​in sequence from the range of the histogram bin number, dividing the grayscale level of the brightness component L processed in step 2) into equal intervals according to the current value of the histogram bin number, performing the equal interval histogram equalization, and obtaining a new brightness component L under the current value, until all values ​​within the range of the histogram bin number are traversed, obtaining a new brightness component L under each value of the histogram bin number, and then combining the chromaticity components a and b to obtain the CIE-Lab color space under each value of the histogram bin number;

[0009] 4) The CIE-Lab color space at each value of the histogram bin number is reversely converted to the RGB color space of the underwater image to obtain an enhanced underwater color image at each value of the histogram bin number;

[0010] 5) Performing quality assessment on the underwater color image at each value of the enhanced histogram bin number, and taking the underwater color image with the best quality assessment result as the enhanced image.

[0011] The beneficial effects of the present invention are as follows: the method of the present invention utilizes a contrast-limited adaptive histogram equalization method to process the L component of the CIE-Lab color space, improve the image contrast, and adopts equal division of the histogram bin number to evenly divide the grayscale interval, so that the image histogram is approximately evenly distributed, and finally an enhanced color image is obtained, and the enhanced underwater color image is subjected to a reference-free quality evaluation. The method of the present invention can effectively improve the contrast and brightness of underwater images, improve the blue-green bias of images, and achieve a more natural enhancement effect for underwater images. The method of the present invention has good visual effects in most application scenarios where the image details are not very high, and is mainly suitable for a variety of underwater images including underwater fishing, underwater plants and animals, and submarine archaeology.

[0012] In order to improve the accuracy of image quality assessment, in step 5), the underwater color image with each value of the enhanced histogram bin number is evaluated using the no-reference quality assessment UIQM, and the underwater color image with the largest UIQM evaluation index value is taken as the enhanced image.

[0013] In order to improve the computational efficiency of the enhancement process, in step 3), the minimum value in the histogram bin number range is used as the initial value, and the value is taken by successively increasing the bin number until the current value is equal to the maximum value in the histogram bin number range.

[0014] Furthermore, the step of converting the RGB color space to the CIE-Lab color space in step 1) includes:

[0015] Step 1-1, convert the RGB color space to the CIEXYZ color space. The conversion formula is:

[0016]

[0017] Among them, X, Y, Z represent the tristimulus values ​​of the CIEXYZ color space, and R, G, B represent the red channel, green channel, and blue channel of the RGB color space respectively;

[0018] Step 1-2: Convert CIEXYZ color space to CIE-Lab color space. The conversion formula is:

[0019] Where X n , Y n , Z n are the standard D65 lighting white point, and t is the independent variable of the f(t) function.

[0020] Furthermore, the process of performing contrast-limited adaptive histogram equalization processing on the brightness component L in step 2) is: cutting off the portion exceeding the clipping limit from the histogram of the brightness component L, and redistributing the portion exceeding the clipping limit evenly among all histogram bins.

[0021] Furthermore, the step of converting the CIE-Lab color space to the RGB color space in step 4) includes:

[0022] Step 4-1: Convert CIE-Lab color space to CIEXYZ color space. The conversion formula is:

[0023] in

[0024] Step 4-2: Convert CIEXYZ color space to RGB color space. The conversion formula is:

[0025] BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of a method for underwater image multi-scale statistical histogram enhancement in an embodiment of the method of the present invention;

[0027] Figure 2-a It is the image when the L channel of the original image is not processed by the CLAHE algorithm;

[0028] Figure 2-b It is the image processed by CLAHE algorithm on the L channel of the original image;

[0029] Figure 3-a This is the image enhancement effect diagram when the number of histogram bins is 10;

[0030] Figure 3-b This is the image enhancement effect diagram when the number of histogram bins is 50;

[0031] Figure 3-c This is the image enhancement effect diagram when the number of histogram bins is 100;

[0032] Figure 3-d This is the image enhancement effect diagram when the number of histogram bins is 150;

[0033] Figure 3-e This is the image enhancement effect diagram when the number of histogram bins is 200;

[0034] Figure 3-f This is the image enhancement effect diagram when the number of histogram bins is 250;

[0035] Figure 4 It is a UIQM evaluation index value curve corresponding to each value of the underwater image histogram bin number;

[0036] Figure 5 It is the image enhancement effect diagram corresponding to the maximum UIQM evaluation index value when UIQM is used for image quality assessment. DETAILED DESCRIPTION

[0037] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

[0038] In order to solve the problem of color degradation caused by different attenuation rates of light of different wavelengths underwater, and the problem of suspended matter in water absorbing light energy and changing the light path, resulting in unsatisfactory underwater imaging effect and underwater color distortion, the method of the present invention adopts a contrast-limited adaptive histogram equalization method CLAHE to process the L component of the CIE-Lab color space of the underwater image to enhance the image contrast; within the range of histogram bin number values, values ​​are taken in sequence, and the histogram grayscale of the L component of the CIE-Lab color space after the above processing is divided into uniform intervals according to the current value of the histogram bin number, and the interval equalization is performed to obtain enhanced images under various values ​​within the range of the histogram bin number value, so that the image histogram is approximately uniformly distributed; after obtaining the enhanced images under various values ​​of the histogram bin number, image quality evaluation is performed on each enhanced image, and the image with the best evaluation effect is used as the final enhanced image.

[0039] The color space of the underwater image obtained is generally an RGB color space. The method of the present invention needs to convert the RGB color space into a CIE-Lab color space. The conversion method is to first convert the RGB color space into a CIEXYZ color space, and then convert the CIEXYZ color space into a CIE-Lab color space. The specific steps are:

[0040] 1) Convert the RGB color space to the CIEXYZ color space. The conversion formula is:

[0041]

[0042] Among them, X, Y, Z represent the tristimulus values ​​of the CIEXYZ color space, and R, G, B represent the red channel, green channel, and blue channel of the RGB color space respectively;

[0043] 2) Convert CIEXYZ color space to CIE-Lab color space. The conversion formula is:

[0044] Where X n , Y n , Z n It is the standard D65 lighting white point, and the default values ​​are usually 95.047, 100.0, 108.883, and t is the independent variable of the f(t) function.

[0045] The principle of using CLAHE to process the L component of the CIE-Lab color space is: first, the contrast limit, that is, the threshold, is defined in CLAHE, and the part exceeding the threshold is clipped from the histogram. The clipped value of the histogram, that is, the clipping limit, depends on the normalization of the histogram and thus on the size of the area in the pixel domain; the part exceeding the clipping limit will be redistributed evenly among all histogram bins instead of being discarded.

[0046] The L component in the CIE-Lab color space is processed by CLAHE, and the processed L component is represented by L1, and the number of histogram bins of L1 is 256 by default, that is, each bin represents the statistics of a grayscale. At this time, the number of histogram bins of L1 is divided into equal parts in a uniform interval, that is, the grayscale of the histogram of the L1 component is divided into uniform intervals, and the interval is divided into equal parts for histogram equalization, so that the image histogram is approximately uniformly distributed. Specifically, the number of histogram bins of L1 is divided into equal parts in a uniform interval, which is actually to take values ​​in sequence from the range of the value of the histogram bin number, and the grayscale of the histogram of the L1 component is divided into uniform intervals according to the value of the current histogram bin number, and the enhanced image under the current value of the histogram bin number is obtained, until all values ​​within the range of the value of the histogram bin number are traversed. The value range of the histogram bin number of the luminance component L is 2 to 256. In the present invention, all values ​​within the value range of the histogram bin number are traversed, that is, 255 enhanced images are obtained. Figure 3-a~3-f , Figure 3-a~3-f The image enhancement effect diagrams are given when the number of histogram bins is 10, 50, 100, 150, 200, and 250 respectively.

[0047] After the number of histogram bins of the L component of the CIE-Lab color space is evenly divided, the processed CIE-Lab color space needs to be reversely converted to the RGB color space to obtain an enhanced image. The reverse conversion principle of this application is: first convert the CIE-Lab color space to the CIEXYZ color space, and then convert the CIEXYZ color space to the RGB color space. The specific steps are:

[0048] 1) The conversion formula from CIE-Lab color space to CIEXYZ color space is:

[0049] in

[0050] 2) CIEXYZ color space is converted to RGB color space. The specific conversion formula is:

[0051]

[0052] After obtaining the enhanced images at various values ​​of the histogram bin number, the present invention needs to perform image quality assessment on the enhanced images at various values ​​of the histogram bin number, so as to select the enhanced image with the best image quality assessment result as the final enhanced image. As for the image quality assessment method, the present invention preferably performs image assessment on the underwater image enhancement effect based on the no-reference quality assessment UIQM inspired by the characteristics of the human visual system (HVSs), and selects the enhanced image corresponding to the maximum value of the no-reference quality assessment UIQM as the final enhanced image.

[0053] The unreferenced quality assessment UIQM includes three underwater image attributes: underwater image color metric (UICM), underwater image sharpness metric (UISM), and underwater image contrast metric (UIConM). By combining the three underwater image attribute metrics of UICM, UISM, and UIConM, the underwater image quality assessment of the fusion evaluation is obtained, that is, the overall quality assessment of the underwater image is obtained. Figure 3-a~3-f , which gives the image enhancement effect diagrams when the number of histogram bins is 10, UIQM=4.44, the number of histogram bins is 50, UIQM=6.87, the number of histogram bins is 100, UIQM=7.22, the number of histogram bins is 150, UIQM=5.35, the number of histogram bins is 200, UIQM=7.19, the number of histogram bins is 250, UIQM=6.72. See also Figure 4 , Figure 4 A UIQM evaluation index value curve corresponding to different histogram bin numbers of underwater images is given. When the histogram bin number is between 150 and 200, the maximum value of the UIQM evaluation index value appears. Figure 5 This is the image enhancement effect diagram corresponding to the maximum UIQM evaluation index value.

[0054] Of course, as other implementation methods, some full reference indicators such as boundary strength, PSNR,

[0055] MSE, SSIM, etc., or some non-reference evaluation indicators such as mean, standard deviation, average gradient, information entropy, non-reference evaluation indicator UCIQM for underwater images, etc., as well as fusion evaluation of the above evaluation methods.

[0056] The following is a specific embodiment of the enhancement method of the present invention using matlab simulation software. Figure 1 As shown, the following steps are included:

[0057] Step 1: Use the imread function in Matlab to read the input color underwater image;

[0058] Step 2: Use the function rgb2lab in matlab to convert the RGB color space into the CIE-Lab color space to obtain the brightness component L and the chromaticity components a and b. The specific conversion steps are:

[0059] Step 2-1: Convert the RGB color space to the CIEXYZ color space. The following conversion formula is required for implementation:

[0060]

[0061] Step 2-2: Convert CIEXYZ color space to CIE-Lab color space. The following conversion formula is required for implementation:

[0062] Where X n , Y n , Z n It is the standard D65 lighting white point, and the default values ​​are usually 95.047, 100.0, 108.883, and t is the independent variable of the f(t) function.

[0063] Step 3: Use the adapthisteq function in MATLAB to perform contrast-limited histogram equalization (CLAHE) on the L component of the CIE-Lab color space to enhance the image contrast. After the CLAHE algorithm is processed, the contrast of the underwater image is significantly enhanced, as shown in Figure 3. Figure 2-a , 2-b shown.

[0064] Step 4, using the pair group parameter in the adapthisteq function in matlab to evenly divide the histogram bin number of the L component of the CIE-Lab color space processed in step 3), wherein the NBins parameter in the adapthisteq function is used to construct the histogram bin number of the contrast enhancement transformation. The specific implementation process of this step is: first, taking the minimum value of the histogram bin number as the initial value, dividing the histogram grayscale of the L component into uniform intervals according to the minimum value of the histogram bin number, performing interval equalization histogram equalization, and obtaining an enhanced image, so that the image histogram is approximately uniformly distributed.

[0065] Step 5: Use the lab2rgb function in matlab to combine the processed CIE-Lab brightness component L with the chromaticity components a and b to obtain the processed CIE-Lab color space, and then reversely convert the processed CIE-Lab color space to the RGB color space. The specific steps are:

[0066] Step 5-1: Convert CIE-Lab color space to CIEXYZ color space. The conversion formula for implementation is:

[0067] in

[0068] Step 5-2: Convert CIEXYZ color space to RGB color space. The conversion formula during implementation is:

[0069]

[0070] Step 6. Finally, the enhanced color image is obtained, and the image quality is evaluated by using the underwater image quality assessment UIQM. In the specific implementation, the three underwater image attribute metrics UICM, UISM, and UIConM are combined to obtain the underwater image quality assessment of the fused evaluation.

[0071] Step 7: Initialize the maximum evaluation index value Max-UIQM to 0, and determine whether the number of histogram bins is the maximum value. If not, increase the number of bins and continue to divide the image evenly. At the same time, record the image evaluation index value UIQM. If the UIQM index value is larger than Max-UIQM, replace it with Max-UIQM, otherwise it remains unchanged. Until the number of histogram bins reaches the maximum value, output the enhanced image corresponding to Max-UIQM.

[0072] Of course, as other implementation methods, the maximum bin number is used as the initial value, and the histogram grayscale of the L component is divided into uniform intervals with the maximum bin number, and the interval is equally divided to perform histogram equalization to obtain an enhanced image, and the recorded image evaluation index value UIQM is used as Max-UIQM, and then it is determined whether the bin number is the minimum value. If not, the bin number is reduced and the uniform division is continued. At the same time, the image evaluation index value UIQM is recorded. If the UIQM index value is larger than Max-UIQM, it is replaced with Max-UIQM, otherwise it remains unchanged. Until the histogram bin number reaches the minimum value, the enhanced image corresponding to Max-UIQM is output.

[0073] As another implementation, a parallel processing mechanism may also be used to achieve uniform division of the histogram bin number of the L component, inverse conversion of the color space, and image quality assessment, that is, taking multiple histogram bin numbers at a time and performing subsequent processing such as equal interval division of the grayscale level of the luminance component L after CLAHE processing according to each value, and finally selecting the enhanced image output with the largest image evaluation index value UIQM.

[0074] The method of the present invention converts underwater images from RGB (red-green-blue) color space to CIE-Lab (brightness-red / green color value-yellow / blue color value) color space; uses contrast-limited adaptive histogram equalization for the brightness component L to improve the contrast of the underwater image; divides the bin number of the contrast-enhanced transformed histogram evenly to obtain an enhanced image; and uses the underwater image quality evaluation method to evaluate the underwater image. The present invention has good visual effects in most application scenarios where the image details are not very high. It is mainly applicable to a variety of underwater images including underwater fishing, underwater flora and fauna, and submarine archaeology.

Claims

1. A method for underwater image multi-scale statistical histogram enhancement, characterized in that: The method comprises the following steps: 1) Perform color space conversion on the underwater color image, convert the RGB color space into the CIE-Lab color space, and obtain the brightness component L and the chromaticity components a and b; 2) performing contrast-limited adaptive histogram equalization processing on the brightness component L: trimming the portion exceeding the trimming limit from the histogram of the brightness component L, and redistributing the portion exceeding the trimming limit evenly among all histogram bins; 3) taking values ​​in sequence from the range of the histogram bin number, dividing the grayscale level of the brightness component L processed in step 2) into equal intervals according to the current value of the histogram bin number, performing the equal interval histogram equalization, and obtaining a new brightness component L under the current value, until all values ​​within the range of the histogram bin number are traversed, obtaining a new brightness component L under each value of the histogram bin number, and then combining the chromaticity components a and b to obtain the CIE-Lab color space under each value of the histogram bin number; 4) The CIE-Lab color space at each value of the histogram bin number is reversely converted to the RGB color space of the underwater image to obtain an enhanced underwater color image at each value of the histogram bin number; 5) Performing quality assessment on the underwater color image at each value of the enhanced histogram bin number, and taking the underwater color image with the best quality assessment result as the enhanced image.

2. The underwater image multi-scale statistical histogram enhancement method according to claim 1, characterized in that: In step 5), the underwater color image at each value of the enhanced histogram bin number is evaluated using the no-reference quality assessment UIQM, and the underwater color image with the largest UIQM evaluation index value is taken as the enhanced image.

3. The underwater image multi-scale statistical histogram enhancement method according to claim 1, characterized in that: In the step 3), the minimum value in the range of the histogram bin number is taken as the initial value, and the value is taken by successively increasing the bin number until the current value is equal to the maximum value in the range of the histogram bin number.

4. The underwater image multi-scale statistical histogram enhancement method according to any one of claims 1 to 3, characterized in that: The step of converting the RGB color space to the CIE-Lab color space in step 1) includes: Step 1-1, convert the RGB color space to the CIEXYZ color space. The conversion formula is: Among them, X, Y, Z represent the tristimulus values ​​of the CIEXYZ color space, and R, G, B represent the red channel, green channel, and blue channel of the RGB color space respectively; Step 1-2: Convert CIEXYZ color space to CIE-Lab color space. The conversion formula is: Where X n , Y n , Z n are the standard D65 lighting white point, and t is the independent variable of the f(t) function.

5. The underwater image multi-scale statistical histogram enhancement method according to claim 4, characterized in that: The step of converting the CIE-Lab color space to the RGB color space in step 4) includes: Step 4-1: Convert CIE-Lab color space to CIEXYZ color space. The conversion formula is: in Step 4-2: Convert CIEXYZ color space to RGB color space. The conversion formula is:

Images