Underwater image enhancement method based on adaptive color equalization and multi-scale fusion

Abstract

The invention relates to an underwater image enhancement method based on adaptive color equalization and multi-scale fusion, and belongs to the technical field of computer vision. The method comprises the following steps: firstly, carrying out adaptive color equalization on R, G and B channels of an original underwater image to reduce the difference among the R, G and B channels, and then carrying out color correction by using a white balance algorithm; then, constructing a gamma function for the image after color correction, performing brightness correction, and then performing Rayleigh stretching to obtain an underwater image with clear contrast; thirdly, performing sharpening based on a local variable coefficient on the image after gamma correction to obtain an underwater image with enhanced details, and stretching S and V channels of the processed underwater image to the whole interval; and finally, performing non-subsampled contourlet fusion on the underwater image after S and V stretching to obtain a final clear underwater image. According to the method, the color of the underwater image can be corrected, the contrast ratio of the underwater image is improved, and the details of the underwater image are enhanced.

Description

Technical Field

[0001] The invention relates to an underwater image enhancement method based on adaptive color balance and multi-scale fusion, and belongs to the technical field of computer vision. Background Art

[0002] my country is rich in marine resources, but the degree of development and utilization is very low. Underwater optical images can intuitively obtain seabed information, which is of great significance to the fields of seabed resource exploration, biological research, underwater engineering installation and maintenance, underwater archaeology, etc. Obtaining high-quality underwater images is helpful for the development of the ocean. However, the water molecules, plankton, organisms, etc. in the water will cause light to scatter and absorb light when it propagates in the water, resulting in the direct underwater images showing color distortion, uneven lighting, low contrast, blurred details and other problems. The underwater environment is complex, and different underwater environments have different effects on the image. The underwater image enhancement method can directly change the pixel value according to the mathematical relationship between pixels, without considering the formation and degradation process of the image. Therefore, in order to obtain high-quality underwater images, it is of great significance to design an adaptive underwater image enhancement algorithm that can cope with a variety of complex environments.

[0003] Traditional underwater image enhancement algorithms mainly make corrections based on the characteristics of underwater images. Channel compensation can reduce the gap between the three channels of R (red), G (green) and B (blue), and avoid using the white balance algorithm directly on underwater images, which leads to over-compensation of the red and blue channels and causes color distortion of the image. Commonly used channel compensation first determines the color cast of the image and then compensates for the red channel or the blue channel or the red and blue channels; the white balance algorithm corrects the color of the image. Commonly used white balance algorithms include the gray world algorithm suitable for images with rich colors and the maximum perfect reflection algorithm suitable for images with white areas. When the adaptation conditions are not met, it will cause color cast of the image; the method based on the Retinex model can improve the problem of color distortion, improve the contrast and brightness of the image, but blur the image detail information; limited contrast adaptive histogram equalization Directly stretching the RGB channels separately can improve the contrast of the image, but it may cause color distortion. The limited contrast histogram equalization method stretches the L (brightness) component of the Lab space or the V (luminance) component of the HSV space to reduce color distortion, but the contrast enhancement will be limited; the unsharp mask method can improve the details and contours of the image, but it will also amplify the noise in the image; there are many problems with underwater images, and a single method cannot obtain high-quality images. Image fusion based on pyramid decomposition can improve the utilization of image information, not only solve the color distortion problem of the image, but also improve the contrast and clarity of the image, but the pyramid decomposition obtained by upsampling and downsampling will cause some high-frequency information of the image to be lost, and the image boundary will appear blurred and distorted. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an underwater image enhancement method based on adaptive color equalization and multi-scale fusion, aiming to solve the technical problems of color distortion, uneven illumination, low contrast and blurred details existing in underwater images.

[0005] The technical scheme of the present invention is: an underwater image enhancement method based on adaptive color equalization and multi-scale fusion, firstly, adaptive color equalization is performed on the three channels R, G, and B of the original underwater image to reduce the gap between the three channels R, G, and B; then, the color of the underwater image is corrected by using a white balance algorithm for the image with reduced gap between the three channels; then, a gamma function is constructed for the color-corrected image according to its brightness mean and the illumination component in the Retinex model to correct its brightness so that its illumination is uniform; secondly, the three channels R, G, and B of the gamma-corrected image are The three channels of the gamma-corrected image R, G, and B are respectively subjected to Rayleigh stretching to obtain an underwater image with clear contrast; again, the three channels of the gamma-corrected image R, G, and B are respectively subjected to sharpening based on the local variation coefficient to obtain an underwater image with enhanced details; then, the S (saturation) and V channels of the two images with clear contrast and enhanced details are stretched to the entire interval so that S and V are in the same range when the two images are fused to avoid sudden changes in the fused image; finally, the three channels of the two images R, G, and B after S and V stretching are respectively subjected to non-subsampled contourlet fusion to obtain a final clear underwater image. The method of the present invention can correct the color of the underwater image, improve the contrast of the underwater image, and enhance the details of the underwater image.

[0006] The specific steps are:

[0007] Step 1: Adaptively balance the R, G, and B channels of the original underwater image to reduce the gap between the three channels;

[0008] Step 2: Use white balance algorithm to perform color correction on the image with reduced gap between the three channels;

[0009] Step 3: Construct a gamma function based on the brightness mean and the illumination component in the Retinex model to perform gamma correction on the brightness of the color-corrected image;

[0010] Step 4: Perform Rayleigh stretching on the three channels of the gamma-corrected image R, G, and B to obtain an underwater image with clear contrast;

[0011] Step 5: Sharpen the three channels of the gamma-corrected image R, G, and B based on the local coefficient of variation to obtain an underwater image with enhanced details;

[0012] Step 6: Stretch the S and V channels of the underwater image with clear contrast and enhanced details to the entire interval so that S and V are in the same range when the images are fused;

[0013] Step 7: Perform non-subsampled contourlet fusion on the R, G, and B channels of the underwater image after the S and V channels are stretched to obtain the final clear underwater image.

[0014] The Step 1 is specifically as follows:

[0015] Step 1.1: Calculate the weighted average of the three channels R, G, and B of the original underwater image. The calculation formula is:

[0016]

[0017] In the formula, I r_mean Indicates the size of the red channel mean, I g_mean Indicates the size of the green channel mean, I b_mean Indicates the size of the blue channel mean, w r Indicates the proportion of the red channel mean in the sum of the three channel means, w g Indicates the proportion of the green channel mean in the sum of the three channel means, w b Indicates the proportion of the blue channel mean in the sum of the three channel means, I w_mean Indicates the size of the weighted average of the three channels;

[0018] Step 1.2: Process each pixel of the R, G, and B channels of the underwater image according to the global and local characteristics of the pixel. The global characteristic of a pixel refers to the proportion of the mean of the color channel where the pixel is located in the sum of the mean values ​​of the three channels, and the local characteristic refers to the proportion of the mean of the channel in the 3*3 area where the pixel is located in the sum of the mean values ​​of the three channels;

[0019] Specifically, each pixel of the three channels R, G, and B of the underwater image is processed, and the calculation formula is:

[0020]

[0021] In the formula, c represents the color channel value r, g, b, α and β are constants, Indicates the value of the color-balanced pixel at the (x, y) position of the c channel, I c (x, y) represents the value of the pixel at the (x, y) position of the original image c channel. w_mean Indicates the size of the weighted average of the three channels, I c_mean Indicates the size of the mean value of the color channel where the pixel is located. Indicates the size of the mean value of the 3*3 area of ​​the pixel point at the (x,y) position of the c channel. Indicates the size of the average value of the 3*3 area of ​​the red channel pixel at the (x,y) position. Indicates the size of the mean value of the 3*3 area of ​​the green channel pixel at the (x,y) position. Indicates the size of the mean value of the 3*3 area of ​​the blue channel pixel at the (x,y) position.

[0022] The Step 2 is specifically as follows:

[0023] For the image with reduced three-channel gap, the white balance algorithm is selected according to the predicted white pixel ratio to correct the color of the image, which can improve the generalization ability of the algorithm, enable the image to select the appropriate white balance algorithm according to its own characteristics, so that the color of more images can be corrected and color distortion can be avoided;

[0024] Step 2.1: Convert the image from RGB space to YCbCr space. The calculation formula is:

[0025]

[0026] In the formula, Y represents the brightness of the pixel color, Cb represents the size of the pixel blue concentration offset, and Cr represents the size of the pixel red concentration offset. Indicates the size of the red channel pixel value after the image color is balanced. Indicates the size of the green channel pixel value after the image color is balanced. Indicates the size of the blue channel pixel value after the image color is balanced;

[0027] Step 2.2: Traverse the image and calculate the Z value of each pixel. The calculation formula is:

[0028]

[0029] Where z(x,y) represents the Z value of the pixel at position (x,y), Y(x,y) represents the brightness of the pixel at position (x,y), Cb(x,y) represents the blue concentration offset of the pixel at position (x,y), and Cr(x,y) represents the red concentration offset of the pixel at position (x,y);

[0030] Step 2.3: Count the number of pixels where Z>A and calculate their proportion in the total number of pixels. The calculation formula is:

[0031]

[0032] In the formula, A is a parameter, and the value range of the present invention is 0.6 to 0.7, s represents the number of pixels with Z>A, n represents the total number of pixels, and p represents the proportion of the number of pixels with Z>A in the total number of pixels;

[0033] Step 2.4: Select the appropriate white balance algorithm according to the p value. The calculation formula is:

[0034]

[0035] Among them, the gray world algorithm is used to calculate the size of the c channel value after white balance The formula is:

[0036]

[0037] In the formula, Indicates the size of the c channel value after white balance. Indicates the size of the c channel value after color balancing. Indicates the size of the red channel mean after color balancing, Indicates the size of the green channel mean after color balancing, Indicates the size of the blue channel mean after color balancing, Indicates the size of the mean value of the c channel after color balancing;

[0038] Among them, the maximum perfect reflection algorithm is used to calculate the size of the c channel value after white balance The formula is:

[0039]

[0040] In the formula, Indicates the size of the c channel value after color balancing. Indicates the maximum value of the c channel after color balancing. Indicates the size of the mean value of the c channel of the top 10% pixels in the image brightness after color balancing;

[0041] Among them, the size of the c channel value after white balance is calculated using the secondary mapping matrix mapping algorithm The formula is:

[0042]

[0043] In the formula, u c Indicates the white balance control parameter of the square of the c channel pixel after color balancing. Indicates the size of the mean of the square of the c channel pixels after color balancing, v c Indicates the white balance control parameters of the c channel pixels after color balancing. Indicates the size of the mean pixel value of channel c after color balancing. Indicates the size of the mean of the three channels after color balancing. Indicates the size of the square of the pixel value of the c channel of the pixel with Z>0.65 after color balancing. Indicates the size of the c channel pixel value of the pixel with Z>0.65 after color balancing, Y max Indicates the maximum value of the Y channel. Indicates the size of the square of the c channel pixel value after color balancing.

[0044] The Step 3 is specifically as follows:

[0045] The brightness of the color-corrected image is corrected by constructing a gamma function according to its brightness mean and the illumination component in the Retinex model. This allows the image to adaptively adjust its brightness based on the image's own brightness and the distribution characteristics of the illumination component. The illumination component takes the mean value within the neighborhood to make the image change evenly and avoid mutations.

[0046] Step 3.1: Convert the white-balanced image from RGB space to HSV space;

[0047] Step 3.2: Calculate the average brightness. The calculation formula is:

[0048]

[0049] In the formula, Indicates the average brightness of the image after white balance, F wb (x, y) represents the brightness value of the image at position (x, y) after white balance, m represents the number of image rows, and n represents the number of image columns;

[0050] Step 3.3: Extract the illumination component according to the illumination-reflection model. Use three Gaussian functions with scale factors of 15, 80, and 250 to perform convolution with the brightness after white balance, extract the illumination component at low frequency, and take the mean of the three extracted illumination components as the final extracted illumination component. The calculation formula is:

[0051]

[0052] Wherein, L1(x, y), L2(x, y), L3(x, y) represent the illumination components extracted by three Gaussian functions with scale factors of 15, 80, and 250 respectively, L(x, y) is the mean of the illumination components extracted by the three Gaussian functions, c1, c2, c3 represent the scale factors of 15, 80, and 250 respectively, and λ represents the normalization constant;

[0053] Step 3.4: Construct a gamma function based on the brightness mean and the distribution characteristics of the illumination components to correct the brightness of the white-balanced image. The construction rules are as follows:

[0054] The brightness mean is inversely proportional to the gamma value, so that the adjustment intensity can be adaptively adjusted according to the characteristics of the image itself;

[0055] The illumination component takes the mean value within the domain, so that the image changes evenly and avoids mutations;

[0056] The pixels whose illumination component is greater than the average illumination component are dimmed, and vice versa, they are brightened. The difference between the illumination component and the average illumination component is proportional to the adjustment intensity, so that it can be adaptively adjusted according to the distribution characteristics of the illumination component.

[0057] Specifically, a gamma function is constructed according to the brightness mean and the distribution characteristics of the illumination components to correct the brightness of the image after white balance. The calculation formula is:

[0058]

[0059] In the formula, r(x,y) represents the gamma value of the brightness correction of the pixel at position (x,y). Indicates the size of the average brightness of the 3*3 area of ​​pixels at the (x, y) position in the image after white balance, L mean Indicates the size of the mean value of the image illumination component after white balance, I ga (x,y) represents the brightness value at the (x,y) position after gamma correction;

[0060] Step 3.5: Convert the gamma-corrected image from HSV space back to RGB space.

[0061] The Step 4 is specifically as follows:

[0062] Rayleigh stretching is performed on the three channels of the gamma-corrected image R, G, and B to obtain an underwater image with clear contrast. The Rayleigh distribution function has only one parameter. By adjusting the Rayleigh parameter, the connection between the three channels can be deepened, the contrast can be improved, the color distortion can be reduced, and the limitation of contrast limited adaptive histogram equalization (CLAHE) on RGB three-channel stretching or V stretching or L stretching can be avoided.

[0063] Step 4.1: Construct the mapping relationship from the RGB three-channel histogram and the Rayleigh distribution histogram cumulative distribution function respectively:

[0064]

[0065] Where i is the pixel value of the c channel of the gamma-corrected image, which is an integer between [0, 255]. p(x) is the probability function of the pixel value of the c channel after gamma correction. temp(i) is the pixel value after Rayleigh stretching corresponding to the pixel value i of the c channel of the gamma-corrected image. Indicates the size of the pixel value at the (x, y) position of the c channel of the image after gamma correction, I cRL1 (x, y) represents the pixel value at the position (x, y) of the c channel after Rayleigh stretching. c represents the color channel, and its value is r, g, b, σ c represents the Rayleigh parameter of the c channel, which is determined by the mean and mean square error of each channel as follows:

[0066]

[0067] In the formula, and Respectively represent the mean and mean square error of the c channel of the gamma-corrected image, Respectively represent the mean of the red channel, green channel, and blue channel of the image after gamma correction, Respectively represent the mean square error of the red channel, green channel, and blue channel of the image after gamma correction, B1 represents the size of the Rayleigh parameter σ corresponding to the stretching of the Rayleigh histogram, B2 represents the size of the factor affecting the mean, and B3 represents the size of the factor affecting the mean square error; from the properties of the Rayleigh distribution function, the mean of the Rayleigh distribution function is 1.253σ, the mean square error is 0.655σ, and the image mean is around 0.5 to obtain a natural image, then the corresponding B1 value is about 0.4, when σ is determined, the influence factor on the mean is 1.253, and the influence factor on the mean square error is 0.655, then the corresponding B2 value is 1.253, and the B3 value is 0.655;

[0068] Step 4.2: Since data mapping is performed on discrete data, in order to reduce errors and further increase contrast, the three channels are stretched separately. The calculation formula is:

[0069]

[0070] In the formula, Indicates the size of the pixel value at the (x, y) position of the c channel after stretching, C1 indicates the size of the pixel value stretched to the maximum value of 0 pixels, and the present invention takes 0.05, and C2 indicates the size of the pixel value stretched to the minimum value of 1 pixel, and the present invention takes 0.95.

[0071] By using the Rayleigh stretching method of the present invention, the mean of the three channels of the image can be approximately 0.5, the three channels present the same distribution and cover the entire dynamic range, and the connection between the three channels is deepened, the contrast is improved and the color distortion is reduced.

[0072] The Step 5 is specifically as follows:

[0073] The three channels of the gamma-corrected image, R, G, and B, are sharpened based on the local coefficient of variation to obtain detail-enhanced underwater images to compensate for the loss of some detail information due to the increase in contrast.

[0074] Step 5.1: Calculate the local coefficient of variation and normalize it. The calculation formula is:

[0075]

[0076] Where c∈{r,g,b}, d c (x, y) represent the mean and mean square error of the pixel in the 3*3 neighborhood of the c channel (x, y) position, respectively. c (x+i, y+j) represents the size of the pixel value at the position (x+i, y+j) of the c channel, cv c (x,y),f c (x, y) represents the coefficient of variation and normalized coefficient of variation of the pixel at the position (x, y) of the c channel, max{cv r ,cv g ,cv b} is the maximum value of the coefficient of variation of the three channels, cvr, cvg, cvb represent the coefficient of variation of the red channel, green channel, and blue channel respectively;

[0077] Step 5.2: Calculate the adaptive enhancement amplitude. The calculation formula is:

[0078]

[0079] In the formula, λ c (x,y) represents the adaptive enhancement amplitude of the pixel at the position (x,y) of the c channel, λ max is the maximum gain coefficient, which is 1 by default in the present invention;

[0080] Step 5.3: Subtract the low-frequency component from the gamma-corrected image, and add the obtained high-frequency component to the gamma-corrected image according to the adaptive enhancement amplitude. The calculation formula is:

[0081]

[0082] Where c∈{r,g,b}, It represents the pixel value at the position (x, y) of the c channel of the sharpened image, and G represents Gaussian filtering.

[0083] The adaptive unsharp masking method based on local coefficient of variation in the present invention has the following advantages: the coefficient of variation is used to measure the degree of discreteness between different data sets, and the image area can be better divided by introducing the coefficient of variation; in order to avoid over-enhancement, the maximum enhancement amplitude is set to 1; in order to make the image change evenly, a piecewise continuous function is adopted; in order to avoid noise amplification, the noise area is not enhanced; when the enhancement amplitude increases with the increase of the coefficient of variation, the gradient of the enhancement amplitude also becomes larger and larger, and when the enhancement amplitude decreases with the increase of the coefficient of variation, the gradient of the enhancement amplitude also becomes smaller and smaller.

[0084] The Step 6 is specifically as follows:

[0085] Stretching the S and V channels of two images with clear contrast and enhanced details to the entire range makes S and V in the same range when the two images are fused, which can reduce problems such as sudden image changes and color distortion during fusion.

[0086] Step 6.1: Convert the Rayleigh stretched and detail sharpened underwater image from RGB space to HSV space;

[0087] Step 6.2: Stretch the S and V channels of the underwater image. The calculation formula is as follows:

[0088]

[0089] In the formula, S(x,y) represents the value of the S channel (x,y) position after stretching, S 0 (x,y) represents the size of the S channel (x,y) position before stretching. Indicates the minimum value of the S channel before stretching. Indicates the maximum value of the S channel before stretching, V(x,y) indicates the value of the V channel (x,y) after stretching, V 0 (x,y) represents the value of the V channel (x,y) position before stretching. Indicates the minimum value of the V channel before stretching. Indicates the maximum value of the V channel before stretching;

[0090] Step 6.3: Convert the underwater image after S and V channels are stretched from HSV space back to RGB space to obtain the image to be fused Figure 1 and to be integrated Figure 2 .

[0091] The Step 7 is specifically as follows:

[0092] The R, G, and B channels of the two images after S and V stretching are fused by non-subsampled contourlet to obtain the final clear underwater image. The non-subsampled contourlet decomposition process has no up- and downsampling, thus avoiding the loss of high-frequency information caused by up- and downsampling in the pyramid decomposition process, and generating artifacts and other problems.

[0093] Step 7.1: Put the Figure 1 and to be integrated Figure 2 The three RGB channels of the image are transformed by non-subsampled contourlet transform (NSCT), specifically: first, the image is decomposed into low-frequency subband and high-frequency subband using the scale-decomposed non-subsampled pyramid filter bank (NSP), and the high-frequency subband is further decomposed by the direction-decomposed non-subsampled directional filter bank (NSDFB) to obtain 8D high-frequency subband images in different directions, the low-frequency subband is decomposed by NSP and the obtained high-frequency subband is decomposed by NSDFB into 4D high-frequency subbands in different directions, and finally a set of 1D low-frequency subband images and 12D high-frequency subband images are obtained;

[0094] Step 7.2: Adopt the entropy weighting strategy of the square of local coefficients for the low-frequency sub-band coefficients. The relevant calculation formula is:

[0095]

[0096] In the formula, Indicates the size of the pixel value after the c channel low-frequency subband (x, y) position is fused. It indicates the size of the weighted coefficient of the low-frequency subband (x, y) position of the image c channel after Rayleigh. Respectively indicated as to be fused Figure 1 and to be integrated Figure 2 The size of the pixel value at the position (x, y) of the low-frequency sub-image after NSCT decomposition of the c channel, Respectively represent to be fused Figure 1 and to be integrated Figure 2 The local coefficient square entropy of the pixels in the low-frequency sub-image after NSCT decomposition of the c channel is calculated as follows:

[0097]

[0098] Step 7.3 adopts a local energy weighting strategy for the high frequency sub-band coefficients;

[0099] First, calculate the weighted energy value of each pixel neighborhood 3*3 area of ​​the high-frequency sub-image to be fused. The calculation formula is:

[0100]

[0101] In the formula, Respectively represent to be fused Figure 1 and to be integrated Figure 2 The local weighted energy value of the pixel at position (x, y) in the dth high-frequency sub-image after NSCT decomposition of the c channel, Respectively represent to be fused Figure 1 and to be integrated Figure 2 The pixel value at position (x, y) in the dth high-frequency sub-image after NSCT decomposition of the c channel is calculated as follows: w is the weighting coefficient matrix, and the calculation formula is:

[0102]

[0103] Secondly, the difference in normalized local energy is calculated. The relevant calculation formula is:

[0104]

[0105] In the formula, Indicates to be integrated Figure 1 and to be integrated Figure 2 The difference in local energy of the pixel at position (x, y) in the dth high-frequency sub-image after NSCT decomposition of the c channel;

[0106] Then, when the difference in local energy When it is less than or equal to 0.2, the calculation is as follows:

[0107]

[0108] In the formula, Indicates the size of the pixel value after fusion of the dth high-frequency subband (x, y) position of the c channel, Indicates to be integrated Figure 1 The size of the weighting coefficient of the d-th high-frequency subband (x, y) position of the c channel;

[0109] When the local energy difference When it is greater than 0.2, the calculation is as follows:

[0110]

[0111] Step 7.4: Perform inverse transform on the fused NSCT coefficients to obtain the enhanced image.

[0112] The beneficial effects of the present invention are as follows: pixels can adaptively adjust pixel values ​​according to the weighted mean of three channels and the global and local characteristics of pixels to reduce the gap between the three channels, and there is no need to select channels for compensation according to the color cast of the image; the use of a judgment-selected white balance algorithm can enable underwater images to select a suitable white balance algorithm according to the predicted white pixel ratio, thereby improving the generalization of the algorithm, allowing the colors of more underwater images to be corrected and reducing distortion; the gamma function constructed according to the brightness mean of the image and the distribution characteristics of the illumination component can adaptively adjust the image brightness, thereby exposing more details in the overly dark and overly bright areas of the image, and the illumination component takes the mean value within the neighborhood, thereby allowing the image to change evenly, Avoid mutations; during Rayleigh stretching, the Rayleigh parameters of each channel are adaptively adjusted through the mean and mean square error of each channel as well as the maximum mean and mean square error, so that the mean of each channel of the image can be approximately 0.5, and the three channels have the same distribution and cover the entire dynamic range, which deepens the connection between the three channels and reduces color distortion while improving contrast; when sharpening, the introduction of local coefficient of variation can make the image area better divided compared with the local variance; before fusion, the S and V of the image to be fused are stretched, so that the image to be fused is in the same range, avoiding mutations in the image during fusion; using non-sampled contourlet fusion can avoid problems such as high-frequency information loss and artifacts caused by up and down sampling during pyramid decomposition. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] Figure 1 is a detailed flow chart of the method of the present invention;

[0114] Figure 2 It is a simple flow chart of the method of the present invention;

[0115] Figure 3 is a white balance algorithm judgment selection diagram in an embodiment of the present invention;

[0116] Figure 4 is a comparison diagram of image color correction results of the embodiment of the present invention and the classic white balance algorithm;

[0117] Figure 5 is a Rayleigh distribution diagram in an embodiment of the present invention;

[0118] Figure 6 is a normalized histogram of the local coefficient of variation in an embodiment of the present invention;

[0119] Figure 7 is an adaptive enhancement amplitude map in an embodiment of the present invention;

[0120] Figure 8 is the NSCT decomposition diagram of the image to be fused in the embodiment of the present invention;

[0121] Fig. 9 This is a comparison chart of the effects of the embodiment of the present invention with other algorithms. DETAILED DESCRIPTION

[0122] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods.

[0123] Embodiment 1: An underwater image enhancement method based on adaptive color balancing and multi-scale fusion, specifically:

[0124] Figure 1 and Figure 2 It shows the detailed flow chart and simple flow chart of the algorithm in this patent, and the process is:

[0125] Step 1: Adaptively balance the R, G, and B channels of the original underwater image to reduce the gap between the three channels;

[0126] Step 2: Use white balance algorithm to perform color correction on the image with reduced gap between the three channels;

[0127] Step 3: Construct a gamma function based on the brightness mean and the illumination component in the Retinex model to perform gamma correction on the brightness of the color-corrected image;

[0128] Step 4: Perform Rayleigh stretching on the three channels of the gamma-corrected image R, G, and B to obtain an underwater image with clear contrast;

[0129] Step 5: Sharpen the three channels of the gamma-corrected image R, G, and B based on the local coefficient of variation to obtain an underwater image with enhanced details;

[0130] Step 6: Stretch the S and V channels of the underwater image with clear contrast and enhanced details to the entire interval so that S and V are in the same range when the images are fused;

[0131] Step 7: Perform non-subsampled contourlet fusion on the R, G, and B channels of the underwater image after the S and V channels are stretched to obtain the final clear underwater image.

[0132] Figure 3It shows the judgment and selection diagram of the white balance algorithm. First, the channel-compensated image is converted from the RGB space to the YCbCr space. Secondly, the Z value of each pixel is calculated, that is, the brightness value of each pixel minus the absolute value of the blue and red concentration offsets. Then, the pixels with Z greater than 0.65 are predicted as white pixels, and the proportion of white pixels in the image is statistically predicted. Finally, the appropriate white balance algorithm is selected according to the p value. If p is less than one thousandth, it is considered that there are no white pixels in the image, and the gray world algorithm is selected. If p is greater than three percent, it is considered that there are white areas in the image, and the maximum perfect reflection algorithm is selected. If p is between the two, it is considered that there are a small number of white pixels in the image, and the secondary mapping matrix mapping algorithm is selected. This can improve the generalization ability of the white balance algorithm, so that the image can choose the appropriate white balance algorithm according to its own characteristics, so that the colors of more images can be corrected and color distortion can be reduced.

[0133] Figure 4 It shows the processing results of the gray world algorithm, the maximum perfect reflection algorithm and the judgment and selection white balance algorithm in this patent. Image1 does not have any white pixels. The color of the image is corrected after processing by the gray world algorithm, but the maximum perfect reflection algorithm will cause the image to have color cast; image2 has a white area, the gray world algorithm will cause the white area to have color cast, the maximum perfect reflection algorithm can correct the color of the image, and there is no color cast in the white area; image3 has a small amount of white pixels, and both the gray world algorithm and the maximum perfect reflection algorithm will cause the white pixels to have color cast; and the judgment and selection white balance algorithm proposed in this patent can select the gray world algorithm for image1 without white pixels, select the maximum perfect reflection algorithm for image2 with white areas, and select the secondary mapping matrix mapping algorithm for image3 with a small amount of white pixels, so that the image can select the white balance algorithm suitable for itself according to its own characteristics, thereby correcting the color of the image.

[0134] Figure 5 It represents the Rayleigh distribution diagram. By adjusting the Rayleigh parameter value σ, the mean of each channel can be approximated to 0.5. The three-channel histogram presents the same distribution and covers the entire dynamic range. The Rayleigh parameter is adaptively adjusted according to the mean and mean square error of the three channels to deepen the connection between the three channels, thereby reducing color distortion while improving image contrast.

[0135] Figure 6It shows the normalized histogram of the local coefficient of variation of the gamma-corrected image. It can be seen that the smaller the coefficient of variation, the greater the frequency, and the larger the coefficient of variation, the smaller the frequency. Therefore, the coefficient of variation can be divided into four regions [0, 0.25], [0.25, 0.5], [0.5, 0.75], [0.75, 1], corresponding to the flat area, detail area, edge area and noise of the image. The local coefficient of variation can be used to measure the degree of discreteness between different data sets. Using the local coefficient of variation can make the region better divided than using the local mean square error.

[0136] Figure 7 It shows the function graph of adaptive enhancement amplitude, which enhances the four regions of the image to different degrees. The flat region has a small enhancement amplitude; the detail region and edge region have a large enhancement amplitude, but in order to avoid over-enhancement, the maximum enhancement amplitude is set to 1; the noise is not enhanced, thereby reducing the noise amplification problem of the image after sharpening. In order to avoid sudden changes in the image and make the image change evenly, the adaptive enhancement amplitude adopts a piecewise continuous function.

[0137] Figure 8 It shows the NSCT decomposition diagram of the image. First, the NSP tool is used to perform multi-scale decomposition of the image into 1 low-frequency sub-image and 1 high-frequency sub-image. Then, the high-frequency sub-image is decomposed into 8 high-frequency sub-images in different directions using NSDFB. Then, this process is repeated, and the low-frequency sub-image is decomposed into 1 low-frequency sub-image and 1 high-frequency sub-image using NSP. Then, the high-frequency sub-image is decomposed into 4 high-frequency sub-bands in different directions using NSDFB, and finally a set of 1 low-frequency sub-band and 12 high-frequency sub-bands of the same size are obtained. After stretching the S and V channels of the Rayleigh image and the detail image, each channel is decomposed according to the NSCT decomposition diagram, and the corresponding low-frequency and high-frequency sub-bands are fused according to the low-frequency fusion rules and high-frequency fusion rules respectively. There is no upsampling or downsampling in the NSCT decomposition process. The decomposed image is equal to the original image in size and has translation invariance, which can avoid the loss of high-frequency information during fusion and the generation of artifacts.

[0138] Fig. 9The figure shows the processing results of 8 underwater images in the dataset UIEB by this patent and other underwater image enhancement algorithms. DCP is an underwater image restoration algorithm based on priors. Due to the complexity of the underwater environment, it is impossible to obtain a fixed prior to satisfy all underwater scenes, and the color of the image cannot be well corrected; the unsupervised color correction method (UCM) method is to perform histogram normalization white light balance on the RGB three channels, and then convert to the HSV color space to perform histogram stretching on the V channel to enhance the details of the image and improve the contrast. Because the underwater image channels are not effectively compensated, obvious color cast occurs; the Ancuti method uses the gray world algorithm for white balance on all images, resulting in image color cast, such as the white area of ​​image5 is yellowish; the Fu algorithm corrects the color of the image, but the excessive contrast enhancement causes image3, image4, and image5 to have dark areas with unclear details. Compared with the original image, this patent can correct the image color, with clear contrast and significant visual structure. Compared with other algorithms, it has higher robustness.

[0139] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. An underwater image enhancement method based on adaptive color equalization and multi-scale fusion, characterized by: Step 1: Adaptively balance the R, G, and B channels of the original underwater image to reduce the gap between the three channels; Step 2: Use white balance algorithm to perform color correction on the image with reduced gap between the three channels; Step 3: Construct a gamma function based on the brightness mean and the illumination component in the Retinex model to perform gamma correction on the brightness of the color-corrected image; Step 4: Perform Rayleigh stretching on the three channels of the gamma-corrected image R, G, and B to obtain an underwater image with clear contrast; Step 5: Sharpen the three channels of the gamma-corrected image R, G, and B based on the local coefficient of variation to obtain an underwater image with enhanced details; Step 6: Stretch the S and V channels of the underwater image with clear contrast and enhanced details to the entire interval so that S and V are in the same range when the images are fused; Step 7: Perform non-subsampled contourlet fusion on the R, G, and B channels of the underwater image after the S and V channels are stretched to obtain the final clear underwater image.

2. The underwater image enhancement method based on adaptive color equalization and multi-scale fusion according to claim 1, characterized in that: The Step 1 is specifically as follows: Step 1.1: Calculate the weighted average of the three channels R, G, and B of the original underwater image. The calculation formula is: In the formula, I r_mean Indicates the size of the red channel mean, I g_mean Indicates the size of the green channel mean, I b_mean Indicates the size of the blue channel mean, w r Indicates the proportion of the red channel mean in the sum of the three channel means, w g Indicates the proportion of the green channel mean in the sum of the three channel means, w b Indicates the proportion of the blue channel mean in the sum of the three channel means, I w_mean Indicates the size of the weighted average of the three channels; Step 1.2: Process each pixel of the three channels R, G, and B of the underwater image. The calculation formula is: In the formula, c represents the color channel value r, g, b, α and β are constants, Indicates the value of the color-balanced pixel at the (x, y) position of the c channel, I c (x, y) represents the value of the pixel at the (x, y) position of the original image c channel. w_mean Indicates the size of the weighted average of the three channels, I c_mean Indicates the size of the mean value of the color channel where the pixel is located. Indicates the size of the mean value of the 3*3 area of ​​the pixel point at the (x,y) position of the c channel. Indicates the size of the average value of the 3*3 area of ​​the red channel pixel at the (x,y) position. Indicates the size of the mean value of the 3*3 area of ​​the green channel pixel at the (x,y) position. Indicates the size of the mean value of the 3*3 area of ​​the blue channel pixel at the (x,y) position.

3. The underwater image enhancement method based on adaptive color equalization and multi-scale fusion according to claim 1, characterized in that: The Step 2 is specifically as follows: Step 2.1: Convert the image from RGB space to YCbCr space. The calculation formula is: In the formula, Y represents the brightness of the pixel color, Cb represents the size of the pixel blue concentration offset, and Cr represents the size of the pixel red concentration offset. Indicates the size of the red channel pixel value after the image color is balanced. Indicates the size of the green channel pixel value after the image color is balanced. Indicates the size of the blue channel pixel value after the image color is balanced; Step 2.2: Traverse the image and calculate the Z value of each pixel. The calculation formula is: z(x,y)=Y(x,y)-|Cb(x,y)|-|Cr(x,y)| (4) Where z(x,y) represents the Z value of the pixel at position (x,y), Y(x,y) represents the brightness of the pixel at position (x,y), Cb(x,y) represents the blue concentration offset of the pixel at position (x,y), and Cr(x,y) represents the red concentration offset of the pixel at position (x,y); Step 2.3: Count the number of pixels where Z>A and calculate their proportion in the total number of pixels. The calculation formula is: In the formula, A is a parameter, s represents the number of pixels with Z>A, n represents the total number of pixels, and p represents the proportion of pixels with Z>A in the total number of pixels; Step 2.4: Select the appropriate white balance algorithm according to the p value. The calculation formula is: Among them, the gray world algorithm is used to calculate the size of the c channel value after white balance The formula is: In the formula, Indicates the size of the c channel value after white balance. Indicates the size of the c channel value after color balancing. Indicates the size of the red channel mean after color balancing, Indicates the size of the green channel mean after color balancing, Indicates the size of the blue channel mean after color balancing, Indicates the size of the mean value of the c channel after color balancing; Among them, the size of the c channel value after white balance is calculated using the maximum perfect reflection algorithm. c wb The formula is: In the formula, Indicates the size of the c channel value after color balancing. Indicates the maximum value of the c channel after color balancing. Indicates the size of the mean value of the c channel of the top 10% pixels in the image brightness after color balancing; Among them, the size of the c channel value after white balance is calculated using the secondary mapping matrix mapping algorithm The formula is: In the formula, u c Indicates the white balance control parameter of the square of the c channel pixel after color balancing. Indicates the size of the mean of the square of the c channel pixels after color balancing, v c Indicates the white balance control parameters of the c channel pixels after color balancing. Indicates the size of the mean pixel value of channel c after color balancing. Indicates the size of the mean of the three channels after color balancing. Indicates the size of the square of the pixel value of the c channel of the pixel with Z>0.65 after color balancing. Indicates the size of the c channel pixel value of the pixel with Z>0.65 after color balancing, Y max Indicates the maximum value of the Y channel. Indicates the size of the square of the c channel pixel value after color balancing.

4. The underwater image enhancement method based on adaptive color equalization and multi-scale fusion according to claim 1, characterized in that: The Step 3 is specifically as follows: Step 3.1: Convert the white-balanced image from RGB space to HSV space; Step 3.2: Calculate the average brightness. The calculation formula is: In the formula, Indicates the average brightness of the image after white balance, F wb (x, y) represents the brightness value of the image at position (x, y) after white balance, m represents the number of image rows, and n represents the number of image columns; Step 3.3: Extract the illumination component according to the illumination-reflection model. Use three Gaussian functions with scale factors of 15, 80, and 250 to perform convolution with the brightness after white balance, extract the illumination component at low frequency, and take the mean of the three extracted illumination components as the final extracted illumination component. The calculation formula is: Wherein, L1(x, y), L2(x, y), L3(x, y) represent the illumination components extracted by three Gaussian functions with scale factors of 15, 80, and 250 respectively, L(x, y) is the mean of the illumination components extracted by the three Gaussian functions, c1, c2, c3 represent the scale factors of 15, 80, and 250 respectively, and λ represents the normalization constant; Step 3.4: Construct a gamma function based on the brightness mean and the distribution characteristics of the illumination components to correct the brightness of the image after white balance. The calculation formula is: In the formula, r(x,y) represents the gamma value of the brightness correction of the pixel at position (x,y). Indicates the size of the average brightness of the 3*3 area of ​​pixels at the (x, y) position in the image after white balance, L mean Indicates the size of the mean value of the image illumination component after white balance, I ga (x,y) represents the brightness value at the (x,y) position after gamma correction; Step 3.5: Convert the gamma-corrected image from HSV space back to RGB space.

5. The underwater image enhancement method based on adaptive color equalization and multi-scale fusion according to claim 1, characterized in that: The Step 4 is specifically as follows: Step 4.1: Construct the mapping relationship from the RGB three-channel histogram and the Rayleigh distribution histogram cumulative distribution function respectively: Where i is the pixel value of the c channel of the gamma-corrected image, which is an integer between [0, 255], p(x) is the probability function of the pixel value of the c channel after gamma correction, temp(i) is the pixel value after Rayleigh stretching corresponding to the pixel value i of the c channel of the gamma-corrected image, and I c ga (x, y) represents the pixel value at the position (x, y) of the c channel of the image after gamma correction. c RL1 (x, y) represents the pixel value at the position (x, y) of the c channel after Rayleigh stretching. c represents the color channel, and its value is r, g, b, σ c represents the Rayleigh parameter of the c channel, which is determined by the mean and mean square error of each channel as follows: In the formula, and Respectively represent the mean and mean square error of the c channel of the gamma-corrected image, Respectively represent the mean of the red channel, green channel, and blue channel of the image after gamma correction, Respectively represent the mean square error of the red channel, green channel, and blue channel of the image after gamma correction, B1 represents the size of the Rayleigh parameter corresponding to the stretching of the Rayleigh histogram, B2 represents the size of the factor affecting the mean, and B3 represents the size of the factor affecting the mean square error; Step 4.2: Stretch the three channels separately, and the calculation formula is: In the formula, Indicates the size of the pixel value at the (x, y) position of the c channel after stretching. C1 indicates the size of the pixel value stretched to the maximum value of 0 pixels, and C2 indicates the size of the pixel value stretched to the minimum value of 1 pixel.

6. The underwater image enhancement method based on adaptive color equalization and multi-scale fusion according to claim 1, characterized in that: The Step 5 is specifically as follows: Step 5.1: Calculate the local coefficient of variation and normalize it. The calculation formula is: Where c∈{r,g,b}, d c (x, y) represent the mean and mean square error of the pixel in the 3*3 neighborhood of the c channel (x, y) position, respectively. c (x+i, y+j) represents the size of the pixel value at the position (x+i, y+j) of the c channel, cv c (x,y),f c (x, y) represents the coefficient of variation and normalized coefficient of variation of the pixel at the position (x, y) of the c channel, max{cv r ,cv g ,cv b } is the maximum value of the coefficient of variation of the three channels, cvr, cvg, cvb represent the coefficient of variation of the red channel, green channel, and blue channel respectively; Step 5.2: Calculate the adaptive enhancement amplitude. The calculation formula is: In the formula, λ c (x,y) represents the adaptive enhancement amplitude of the pixel at the position (x,y) of the c channel, λ max is the maximum gain coefficient; Step 5.3: Subtract the low-frequency component from the gamma-corrected image, and add the obtained high-frequency component to the gamma-corrected image according to the adaptive enhancement amplitude. The calculation formula is: Where c∈{r,g,b}, It represents the pixel value at the position (x, y) of the c channel of the sharpened image, and G represents Gaussian filtering.

7. The underwater image enhancement method based on adaptive color equalization and multi-scale fusion according to claim 1, characterized in that: The Step 6 is specifically as follows: Step 6.1: Convert the Rayleigh stretched and detail sharpened underwater image from RGB space to HSV space; Step 6.2: Stretch the S and V channels of the underwater image. The calculation formula is as follows: In the formula, S(x,y) represents the value of the S channel (x,y) position after stretching, S 0 (x,y) represents the size of the S channel (x,y) position before stretching. Indicates the minimum value of the S channel before stretching. Indicates the maximum value of the S channel before stretching, V(x,y) indicates the value of the V channel (x,y) after stretching, V 0 (x,y) represents the value of the V channel (x,y) position before stretching. Indicates the minimum value of the V channel before stretching. Indicates the maximum value of the V channel before stretching; Step 6.3: Convert the underwater image after stretching the S and V channels from the HSV space back to the RGB space to obtain the image to be fused 1 and the image to be fused 2.

8. The underwater image enhancement method based on adaptive color equalization and multi-scale fusion according to claim 1, characterized in that: The Step 7 is specifically as follows: Step 7.1: Perform NSCT transformation on the three RGB channels of the to-be-fused images 1 and 2 respectively. Specifically, first, use NSP to decompose the image into two parts: low-frequency sub-band and high-frequency sub-band. Use NSDFB to further decompose the high-frequency sub-band to obtain 8D high-frequency sub-band images in different directions. Perform NSP decomposition on the low-frequency sub-band and use NSDFB to decompose the obtained high-frequency sub-band into 4D high-frequency sub-bands in different directions. Finally, a set of 1D low-frequency sub-band images and 12D high-frequency sub-band images is obtained. Step 7.2: Adopt the entropy weighting strategy of the square of local coefficients for the low-frequency sub-band coefficients. The relevant calculation formula is: In the formula, Indicates the size of the pixel value after the c channel low-frequency subband (x, y) position is fused. It indicates the size of the weighted coefficient of the low-frequency subband (x, y) position of the image c channel after Rayleigh. They are respectively represented as the pixel values ​​of the low-frequency sub-image (x, y) after NSCT decomposition of the c channel of the image to be fused 1 and the image to be fused 2, They represent the local coefficient square entropy of pixels in the low-frequency sub-images of the c channels of the fused images 1 and 2 after NSCT decomposition. The calculation formula is as follows: Step 7.3 adopts a local energy weighting strategy for the high frequency sub-band coefficients; First, calculate the weighted energy value of each pixel neighborhood 3*3 area of ​​the high-frequency sub-image to be fused. The calculation formula is: In the formula, They respectively represent the local weighted energy value of the pixel at position (x, y) in the dth high-frequency sub-image after NSCT decomposition of the c channel of the image to be fused 1 and the image to be fused 2, They represent the pixel value at the (x, y) position in the dth high-frequency sub-image after NSCT decomposition of the c channel of the image to be fused 1 and the image to be fused 2, respectively. w is the weighting coefficient matrix, and the calculation formula is: Secondly, the difference in normalized local energy is calculated. The relevant calculation formula is: In the formula, It represents the difference in local energy between the pixel at position (x, y) in the dth high-frequency sub-image of the c channel of the image to be fused 1 and the image to be fused 2 after NSCT decomposition; Then, when the difference in local energy When it is less than or equal to 0.2, the calculation is as follows: In the formula, Indicates the size of the pixel value after fusion of the dth high-frequency subband (x, y) position of the c channel, Indicates the size of the weighted coefficient of the position of the d-th high-frequency sub-band (x, y) of the c-channel of the image 1 to be fused; When the local energy difference When it is greater than 0.2, the calculation is as follows: Step 7.4: Perform inverse transform on the fused NSCT coefficients to obtain the enhanced image.

Images