Underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion

Abstract

The invention relates to an underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion, and the method comprises the steps: calculating the gray average value of each color channel, and selecting a reference channel; obtaining a corrected first channel and a corrected second channel; decomposing the V channel into an original base layer and three scale detail layers; obtaining an enhanced detail layer; and generating a final enhancement result. According to the method, a gain factor is designed to compensate information loss of other channels, and color loss of a color balance image is reduced by considering attenuation characteristics of different color channels, so that a gray world hypothesis is met; transmission estimation can be improved, so that better detail enhancement and color preservation can be obtained; transmission can be estimated without calculating a dark channel, so that halo artifacts and calculation complexity can be reduced; a detail pyramid is embedded in multi-scale fusion to compensate for detail loss of an underexposure or overexposure area of the underwater image; the contrast and detail recovery are realized on different scales.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, and in particular to an underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion. Background Art

[0002] Due to the complex imaging mechanism of underwater scenes, underwater images are prone to quality degradation. Unfortunately, these degradation issues are not conducive to visual perception analysis and practical underwater applications, such as marine ecological research and aquatic robot inspection, etc. Therefore, enhancing underwater quality has a positive impact on underwater vision applications.

[0003] Current underwater image enhancement methods can be roughly divided into three categories: methods based on multiple images, methods based on single images, and methods based on deep learning. Among them, methods based on multiple images usually use specialized instruments or polarization filters to capture multiple orthogonal images and apply the complementary information between them to reconstruct high-quality underwater images. However, due to the rapid changes in scene structure and lighting conditions, it is very difficult to capture orthogonal polarization images underwater in the same underwater scene. In addition, these complex and expensive imaging devices are not popular in practical applications.

[0004] Single-image based methods usually improve underwater image quality in two ways: one is to derive two inputs from the degraded image using white balance and detail sharpening, and then fuse them by designing multiple weights to obtain an enhanced image; the other is to design a hand-crafted statistical prior based on some haze-free images and use the prior to estimate the haze parameters of the physical imaging model to produce an enhanced result. However, both approaches produce more or less unwanted enhancement problems in extreme environments, such as non-uniform exposure, residual haze, color distortion, etc., for two reasons: first, it is a challenging problem to construct and generalize strong image priors for complex underwater imaging environments; second, using the same enhancement coefficient on the original image to enhance details ignores the local detail exposure differences in complex lighting scenes.

[0005] Deep learning-based methods estimate the unknown parameters of the physical model or directly generate enhanced images based on the powerful learning ability of neural networks. Generally speaking, the training of deep networks requires a large number of real degraded images and clear images, but these pairs of images are difficult to capture in dynamic underwater scenes.

[0006] Therefore, how to design a more effective method to enhance the quality of underwater images has become a technical problem that needs to be solved urgently. Summary of the invention

[0007] In order to solve the limitations of existing underwater image enhancement methods, such as failure to fully consider the underwater wavelength-correlated attenuation characteristics, high complexity of scene transmittance estimation and single local features, which leads to incomplete detail enhancement and unnatural color appearance of the enhancement results, the purpose of the present invention is to provide an underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion, which can reduce the color loss of color balanced images, compensate for the detail loss of underexposed or overexposed areas of underwater images, and effectively prevent over-enhancement or under-enhancement of results.

[0008] To achieve the above object, the present invention adopts the following technical solution: an underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion, the method comprising the following steps in order:

[0009] (1) Calculate the grayscale average of each color channel and select the color channel with the largest grayscale average as the reference channel for color compensation

[0010] (2) Use the stretch function to increase the reference channel Dynamic range, forming the reference channel Corrected version of And use Compensate the first channel to be compensated The second channel to be compensated Get the corrected first channel and the corrected second channel

[0011] (3) Merge into a color-corrected image, convert the color-corrected image from RGB image space to HSV color space, and use a Gaussian filter to decompose the V channel into the original base layer containing the haze effect and three scale detail layers, the three scale detail layers comprising a first scale detail layer, a second scale detail layer, and a third scale detail layer;

[0012] (4) Estimation of global backscattered light and the scene transmittance on the V channel Using the scene transmittance on the V channel Remove the original base layer containing the haze effect The haze effect is clear. Using the scene transmittance on the V channel As an independent stretching factor, the texture details on each scale detail layer are enhanced to obtain the enhanced detail layer

[0013] (5) Introducing weighted fusion function to fuse and enhance detail layer Will enhance the detail layer and clear base layer Merge into a clear image J of the reconstructed brightness channel V V (x,y), the reconstructed clear image J of the brightness channel V V (x, y) is then fused with the original H and S channels of the underwater image to form an enhanced image in the HSV space, and the enhanced image in the HSV space is converted into RGB color space expression to generate the final enhanced result.

[0014] Step (1) specifically refers to: for the underwater image I c , calculate the grayscale average of each color channel

[0015]

[0016] Where H and W represent the rows and columns of the input underwater image, r, g, b represent the red, green, and blue channels of the input underwater image in the RGB space, respectively. c (i, j) is the pixel value of each color channel at the pixel coordinate (i, j); c is the color channel of the underwater image;

[0017] Select the color channel with the largest grayscale average as the reference channel for color compensation

[0018]

[0019] Among them, max() is the maximum value function, Represents the grayscale average of the red, green, and blue channels respectively.

[0020] Step (2) specifically refers to: adding reference channels by stretching the function Dynamic range, forming the reference channel Corrected version of

[0021]

[0022] in, and are the maximum and minimum values ​​of the input underwater images respectively;

[0023] use Compensate the first channel to be compensated The second channel to be compensated Get the corrected first channel and the first channel after correction

[0024]

[0025] in, for The average grayscale value of for The average grayscale value of Represents the grayscale average of the reference channel; Represents the grayscale average of the red, green, and blue channels respectively.

[0026] Step (3) specifically refers to: first, using the Gaussian convolution mask G and the brightness channel image I in the HSV color space v The convolution operation between them generates the original base layer containing the haze effect

[0027]

[0028] In the formula, the Gaussian convolution mask G is expressed as:

[0029]

[0030] Where σ is the standard deviation of the distribution and (x,y) is the pixel coordinate;

[0031] Using the brightness channel image I v and the original base layer containing the haze effect The difference between the first scale detail layer of the Gaussian pyramid is obtained.

[0032]

[0033] Then the brightness channel image I v Downsample to obtain the brightness channel image I of the second scale v,2 , and then calculate the second scale detail layer according to the following formula

[0034]

[0035] In the formula, is the second scale base layer;

[0036] Finally, the brightness channel image I of the second scale is v,2 Downsample to obtain the brightness channel image I of the third scale v,3 , the third scale detail layer is calculated according to the following formula

[0037]

[0038] In the formula, It is the third scale base layer.

[0039] Step (4) specifically refers to: the original base layer including the haze effect is expressed as follows:

[0040]

[0041] In the formula, It is a clear base layer. refers to the global backscattered light; represents the scene transmittance on the V channel, By taking the original transmittance of the underwater scene t c Convert from RGB space to HSV space and extract the V channel component; when hour, choose The pixel coordinates of the top 0.1% of the smallest pixels are used to estimate the global backscattered light:

[0042]

[0043] in, means The pixel value of the largest pixel in the top 99.9% of the pixels, (x*, y*) is The pixel coordinates of the smallest pixels in the top 0.1% of pixels, means The pixel value at the (x*,y*) coordinate;

[0044] For the scale detail layer, it is assumed that the scene transmittance in the local block area of ​​the underwater image is consistent, and the underwater physical imaging model shows the inverse relationship between the gradient of the underwater image and the scene transmittance:

[0045]

[0046] in, represents the gradient operator;

[0047] Using the original transmittance t of the underwater scene c Zoom in underwater image I c Gradient To enhance the clear image c gradient

[0048] The operation of formula (1) is transformed to the gradient domain on the detail layer, and the gradient enhancement of the detail layer is expressed as follows:

[0049]

[0050] in, refers to the original gradient of the i-th scale detail layer, ω is a non-negative parameter that limits the gradient enhancement strength of different detail layers. With the help of Poisson's equation, the enhanced detail layer of each scale is obtained according to the following modified gradient:

[0051]

[0052] In the formula, Restore refers to the gradient restoration operation.

[0053] Step (5) specifically refers to: according to the enhanced detail layer, the first, second and third scale detail layers are weighted summed using the following formula to obtain the enhanced detail layer:

[0054]

[0055] Among them, sgn is a symbolic function; w1, w2, w3 are three composite parameters of different levels; and The enhanced detail layers represent the first, second, and third scale detail layers respectively;

[0056] Based on the clear base layer and Enhanced Detail Layer The reconstructed clear image J of the brightness channel V is derived by the following formula: V (x,y):

[0057]

[0058] Finally, by adding J V The (x, y) and H and S channels of the underwater image are fused to form an enhanced image in the HSV space, and then the enhanced image in the HSV space is converted into RGB color space expression to generate the final enhanced result.

[0059] In step (4), it is known that the underwater physical imaging model consists of two parts: the irradiance D of the reflecting object c and backscattered light irradiance A c , and the calculation formulas for the two are as follows:

[0060] D c (x,y)=J c (x,y)t c (x,y),c∈{r,g,b} (2)

[0061]

[0062] Where (x, y) is the pixel coordinate, J c For clear images, is the global air light constant, tc is the original transmittance of the underwater scene. According to equations (2) and (3), the underwater physical imaging model is expressed as:

[0063]

[0064] Among them, I c is an underwater image; when in the underwater image I c After color correction, the global backscattered light of the RGB channels is almost the same and has a pixel value close to 1. Therefore, for the color-corrected image, equation (3) is rewritten as:

[0065] A c (x,y)=1-t c (x,y) (5)

[0066] According to the underwater physical imaging model, the backscattered light irradiance A c Each pixel value of is higher than 0, but cannot be higher than I c The minimum value of the component, under this premise, define an I c The minimum channel, which consists of the minimum pixel value of each pixel between the r, g, and b channels, that is:

[0067] I m (x,y)=min{I r (x,y),I g (x,y),I b (x,y)}

[0068] Among them, I m (x, y) refers to the minimum channel composed of the minimum values ​​of the r, g, and b channels of the underwater image, min{I r (x,y),I g (x,y),I b (x,y)} represents the selection of the minimum value among the r, g, and b channels; I r (x,y),I g (x,y),I b (x, y) represents the pixel value of the r, g, and b channels at the pixel coordinate (i, j) respectively; since the underwater image has a similar backscattered light in a small local area, a median filter is used to make each pixel have the same backscattered light:

[0069]

[0070] Among them, s v is the window size of the median filter; M(x,y) refers to the minimum channel for which the backscattering is approximately uniform;

[0071] The residual structure and texture information in M(x,y) are further filtered out using the following equation:

[0072]

[0073] Where A is the rough backscattered light;

[0074] A scaling factor is designed to compensate for the local backscattered illumination, which is given by the following equation:

[0075]

[0076] Among them, MeanA and MinA are the average and minimum values ​​of the rough backscattered light A respectively; Thres is the threshold, which determines whether the backscattered light needs to be compensated, Thres = MaxA; then according to the Lambert-Beer empirical law, the degradation of light intensity is related to the properties of the material, and light propagates with exponential dependence; at the same time, considering the opposite iterative process between the exponential function and the power function, using A = A ratio to compensate for the backscattered light; finally, with the help of the improved backscattered light, the original transmittance of the underwater scene is estimated according to the following formula, namely:

[0077] t c (x,y)=1-A c (x,y)

[0078] In the formula, A c (x, y) refers to the compensated backscattered light;

[0079] The original transmittance of the underwater scene t c Transformation to V channel spatial expression formation And through the estimated scene transmittance on the V channel and global backscatter Combined with the original base layer containing the haze effect Remove the original base layer containing the haze effect The haze effect is clear.

[0080]

[0081] It can be seen from the above technical solutions that the beneficial effects of the present invention are as follows: first, the present invention proposes a segmented color balancing method, which uses the maximum mean to explore the reference channel. With the help of this channel, a gain factor is designed to compensate for the information loss of other channels according to the degree of its own information loss; the present invention reduces the color loss of the color balanced image by considering the attenuation characteristics of different color channels, thereby satisfying the gray world hypothesis; second, the present invention proposes a pixel-based transmission estimation method, which helps to improve the transmission estimation based on the mapping relationship between the transmission and the adjusted backscattered light of the underwater image to obtain better detail enhancement and color preservation; the present invention can estimate the transmission without calculating the dark channel, which can reduce halo artifacts and computational complexity; third, the present invention proposes a multi-scale enhancement fusion strategy to improve the contrast based on the estimated transmission, wherein a detail pyramid is embedded in the multi-scale fusion to compensate for the detail loss in the underexposed or overexposed areas of the underwater image; the contrast and detail recovery of the present invention are achieved at different scales, which can effectively prevent the results from being over-enhanced or under-enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 , 2 All of them are flow charts of the methods of the present invention;

[0083] Figure 3 This is a visual comparison result diagram. DETAILED DESCRIPTION

[0084] like Figure 1 , Figure 2 As shown, an underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion includes the following steps in sequence:

[0085] (1) Calculate the grayscale average of each color channel and select the color channel with the largest grayscale average as the reference channel for color compensation

[0086] (2) Use the stretch function to increase the reference channel Dynamic range, forming the reference channel Corrected version of And use Compensate the first channel to be compensated The second channel to be compensated Get the corrected first channel and the corrected second channel

[0087] (3) Merge into a color-corrected image, convert the color-corrected image from RGB image space to HSV color space, and use a Gaussian filter to decompose the V channel into the original base layer containing the haze effect and three scale detail layers, the three scale detail layers comprising a first scale detail layer, a second scale detail layer, and a third scale detail layer;

[0088] (4) Estimation of global backscattered light and the scene transmittance on the V channel Using the scene transmittance on the V channel Remove the original base layer containing the haze effect The haze effect is clear. Enhances contrast and clarity of base layer; utilizes scene transmittance on V channel As an independent stretching factor, the texture details on each scale detail layer are enhanced to obtain the enhanced detail layer Thereby improving the overall image outline and key target information;

[0089] (5) Introducing weighted fusion function to fuse and enhance detail layer Will enhance the detail layer and clear base layer Merge into a clear image J of the reconstructed brightness channel V V (x,y), the reconstructed clear image J of the brightness channel V V (x, y) is then fused with the original H and S channels of the underwater image to form an enhanced image in the HSV space, and the enhanced image in the HSV space is converted into RGB color space expression to generate the final enhanced result.

[0090] like Figure 1 , Figure 2 As shown, step (1) specifically refers to: for the underwater image I c , calculate the grayscale average of each color channel

[0091]

[0092] Where H and W represent the rows and columns of the input underwater image, r, g, b represent the red, green, and blue channels of the input underwater image in the RGB space, respectively. c (i, j) is the pixel value of each color channel at the pixel coordinate (i, j); c is the color channel of the underwater image;

[0093] Select the color channel with the largest grayscale average as the reference channel for color compensation

[0094]

[0095] Among them, max() is the maximum value function, Represents the grayscale average of the red, green, and blue channels respectively.

[0096] like Figure 1 , Figure 2 As shown, step (2) specifically refers to: increasing the reference channel by stretching the function Dynamic range, forming the reference channel Corrected version of

[0097]

[0098] in, and are the maximum and minimum values ​​of the input underwater images respectively;

[0099] use Compensate the first channel to be compensated The second channel to be compensated Get the corrected first channel and the first channel after correction

[0100]

[0101] in, for The average grayscale value of for The average grayscale value of Represents the grayscale average of the reference channel; Represents the grayscale average of the red, green, and blue channels respectively.

[0102] like Figure 1 , Figure 2 As shown, step (3) specifically refers to: first, using the Gaussian convolution mask G and the brightness channel image I in the HSV color space v The convolution operation between them generates the original base layer containing the haze effect

[0103]

[0104] In the formula, the Gaussian convolution mask G is expressed as:

[0105]

[0106] Where σ is the standard deviation of the distribution and (x,y) is the pixel coordinate;

[0107] Using the brightness channel image I vand the original base layer containing the haze effect The difference between the first scale detail layer of the Gaussian pyramid is obtained.

[0108]

[0109] Then the brightness channel image I v Downsample to obtain the brightness channel image I of the second scale v,2 , and then calculate the second scale detail layer according to the following formula

[0110]

[0111]

[0112] In the formula, is the second scale base layer;

[0113] Finally, the brightness channel image I of the second scale is v,2 Downsample to obtain the brightness channel image I of the third scale v,3 , the third scale detail layer is calculated according to the following formula

[0114]

[0115] In the formula, It is the third scale base layer.

[0116] To address the problem that the whole image enhancement process may lead to under-enhancement or over-enhancement of complex underwater scenes, different operations are applied to multiple residual images to enhance contrast and details respectively.

[0117] like Figure 1 , Figure 2 As shown, step (4) specifically refers to: the original base layer including the haze effect is represented by the following formula:

[0118]

[0119] In the formula, It is a clear base layer. refers to the global backscattered light; represents the scene transmittance on the V channel, By converting the original transmittance of the underwater scene t c Convert from RGB space to HSV space and extract the V channel component; when hour, choose The pixel coordinates of the top 0.1% of the smallest pixels are used to estimate the global backscattered light:

[0120]

[0121] in, means The pixel value of the largest pixel in the top 99.9% of the pixels, (x*, y*) is The pixel coordinates of the smallest pixels in the top 0.1% of pixels, means The pixel value at the (x*,y*) coordinate;

[0122] For the scale detail layer, it is assumed that the scene transmittance in the local block area of ​​the underwater image is consistent, and the underwater physical imaging model shows the inverse relationship between the gradient of the underwater image and the scene transmittance:

[0123]

[0124] in, represents the gradient operator;

[0125] Using the original transmittance t of the underwater scene c Zoom in underwater image I c Gradient To enhance the clear image c gradient

[0126] The operation of formula (1) is transformed to the gradient domain on the detail layer, and the gradient enhancement of the detail layer is expressed as follows:

[0127]

[0128] in, refers to the original gradient of the i-th scale detail layer, ω is a non-negative parameter that limits the gradient enhancement strength of different detail layers. With the help of Poisson's equation, the enhanced detail layer of each scale is obtained according to the following modified gradient:

[0129]

[0130] In the formula, Restore refers to the gradient restoration operation.

[0131] like Figure 1 , Figure 2 As shown, step (5) specifically refers to: according to the enhanced detail layer, the first, second, and third scale detail layers are weighted summed using the following formula to obtain the enhanced detail layer:

[0132]

[0133] Among them, sgn is a symbolic function; w1, w2, w3 are three composite parameters of different levels; and The enhanced detail layers represent the first, second, and third scale detail layers respectively;

[0134] Based on the clear base layer and Enhanced Detail Layer The reconstructed clear image J of the brightness channel V is derived by the following formula: V (x,y):

[0135]

[0136] Finally, by adding J V The (x, y) and H and S channels of the underwater image are fused to form an enhanced image in the HSV space, and then the enhanced image in the HSV space is converted into RGB color space expression to generate the final enhanced result.

[0137] In step (4), it is known that the underwater physical imaging model consists of two parts: the irradiance D of the reflecting object c and backscattered light irradiance A c , and the calculation formulas for the two are as follows:

[0138] D c (x,y)=J c (x,y)t c (x,y),c∈{r,g,b} (2)

[0139]

[0140] Where (x, y) is the pixel coordinate, J c For clear images, is the global air light constant, t c is the original transmittance of the underwater scene. According to equations (2) and (3), the underwater physical imaging model is expressed as:

[0141]

[0142] Among them, I c is an underwater image; when in the underwater image I c After color correction, the global backscattered light of the RGB channels is almost the same and has a pixel value close to 1. Therefore, for the color-corrected image, equation (3) is rewritten as:

[0143] A c (x,y)=1-t c (x,y) (5)

[0144] According to the underwater physical imaging model, the backscattered light irradiance A c Each pixel value of is higher than 0, but cannot be higher than I c The minimum value of the component, under this premise, define an I c The minimum channel, which consists of the minimum pixel value of each pixel between the r, g, and b channels, that is:

[0145] I m (x,y)=min{I r (x,y),I g (x,y),I b (x,y)}

[0146] Among them, I m (x, y) refers to the minimum channel composed of the minimum values ​​of the r, g, and b channels of the underwater image, min{I r (x,y),I g (x,y),I b (x,y)} represents the selection of the minimum value among the r, g, and b channels; I r (x,y),I g (x,y),I b (x, y) represents the pixel value of the r, g, and b channels at the pixel coordinate (i, j) respectively; since the underwater image has a similar backscattered light in a small local area, a median filter is used to make each pixel have the same backscattered light:

[0147]

[0148] Among them, s v is the window size of the median filter; M(x,y) refers to the minimum channel for which the backscattering is approximately uniform;

[0149] The residual structure and texture information in M(x,y) are further filtered out using the following equation:

[0150]

[0151] Where A is the rough backscattered light;

[0152] In underwater scenes, the incident light is significantly attenuated by the water, resulting in the restoration result being usually low in brightness. To further address this issue, an average of the rough backscattered light is used to adaptively compensate itself.

[0153] A scaling factor is designed to compensate for the local backscattered illumination, which is given by the following equation:

[0154]

[0155] Among them, MeanA and MinA are the average and minimum values ​​of the rough backscattered light A respectively; Thres is the threshold, which determines whether the backscattered light needs to be compensated, Thres = MaxA; then according to the Lambert-Beer empirical law, the degradation of light intensity is related to the properties of the material, and light propagates with exponential dependence; at the same time, considering the opposite iterative process between the exponential function and the power function, using A = A ratio to compensate for the backscattered light; finally, with the help of the improved backscattered light, the original transmittance of the underwater scene is estimated according to the following formula, namely:

[0156] t c (x,y)=1-A c (x,y)

[0157] In the formula, A c (x, y) refers to the compensated backscattered light;

[0158] The original transmittance of the underwater scene t c Transformation to V channel spatial expression formation And through the estimated scene transmittance on the V channel and global backscatter Combined with the original base layer containing the haze effect Remove the original base layer containing the haze effect The haze effect is clear.

[0159]

[0160] exist Figure 3 In the figure, the original image of the underwater image input by the embodiment of the present invention is shown in Figure (a), the underwater image after processing by the prior art HFM is shown in Figure (b), the underwater image after processing by the prior art PCDE is shown in Figure (c), the underwater image after processing by the prior art WWPF is shown in Figure (d), the underwater image after processing by the prior art LANet is shown in Figure (e), the underwater image after processing by the prior art PUIE-Net is shown in Figure (f), the underwater image after processing by the prior art NUDCP is shown in Figure (g), the underwater image after processing by the prior art DAACC is shown in Figure (h), the underwater image after processing by the prior art DATVR is shown in Figure (i), and the underwater image processed by the method proposed in the present invention is shown in Figure (j).

[0161] In summary, the present invention calculates the grayscale average value of each color channel and selects a reference channel; and uses it to correct and compensate the other two channels to form a color-corrected image; decomposes the V channel into an original base layer and three scale detail layers, and obtains a clear base layer and an enhanced detail layer by estimating the global backscattered light and the scene transmittance, and generates the final enhancement result. The present invention proposes a segmented color balancing method, which uses the maximum mean to explore the reference channel. With the help of this channel, a gain factor is designed to compensate for the information loss of other channels according to the degree of its own information loss; the present invention reduces the color loss of the color balanced image by considering the attenuation characteristics of different color channels, thereby satisfying the gray world hypothesis; the present invention proposes a pixel-based transmission estimation method, which is based on the mapping relationship between the transmission and the adjusted backscattered light of the underwater image, which helps to improve the transmission estimation to obtain better detail enhancement and color preservation; the present invention can estimate the transmission without calculating the dark channel, which can reduce halo artifacts and computational complexity; the present invention proposes a multi-scale enhancement fusion strategy to improve the contrast based on the estimated transmission, wherein a detail pyramid is embedded in the multi-scale fusion to compensate for the detail loss in the underexposed or overexposed areas of the underwater image; the contrast and detail recovery of the present invention are achieved at different scales, which can effectively prevent the results from being over-enhanced or under-enhanced.

Claims

1. An underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion, characterized by: The method comprises the following steps in order: (1) Calculate the grayscale average of each color channel and select the color channel with the largest grayscale average as the reference channel for color compensation (2) Use the stretch function to increase the reference channel Dynamic range, forming the reference channel Corrected version of And use Compensate the first channel to be compensated The second channel to be compensated Get the corrected first channel and the corrected second channel (3) Merge into a color-corrected image, convert the color-corrected image from RGB image space to HSV color space, and use a Gaussian filter to decompose the V channel into the original base layer containing the haze effect and three scale detail layers, the three scale detail layers comprising a first scale detail layer, a second scale detail layer, and a third scale detail layer; (4) Estimation of global backscattered light and the scene transmittance on the V channel Using the scene transmittance on the V channel Remove the original base layer containing the haze effect The haze effect is clear. Using the scene transmittance on the V channel As an independent stretching factor, the texture details on each scale detail layer are enhanced to obtain the enhanced detail layer (5) Introducing weighted fusion function to fuse and enhance detail layer Will enhance the detail layer and clear base layer Merge into a clear image J of the reconstructed brightness channel V V (x,y), the reconstructed clear image J of the brightness channel V V (x, y) is then fused with the original H and S channels of the underwater image to form an enhanced image in the HSV space, and the enhanced image in the HSV space is converted into RGB color space expression to generate the final enhanced result.

2. The underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion according to claim 1, characterized in that: Step (1) specifically refers to: for the underwater image I c , calculate the grayscale average of each color channel Where H and W represent the rows and columns of the input underwater image, r, g, b represent the red, green, and blue channels of the input underwater image in the RGB space, respectively. c (i, j) is the pixel value of each color channel at the pixel coordinate (i, j; c is the color channel of the underwater image; Select the color channel with the largest grayscale average as the reference channel for color compensation Among them, max() is the maximum value function, Represents the grayscale average of the red, green, and blue channels respectively.

3. The underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion according to claim 1, characterized in that: Step (2) specifically refers to: adding reference channels by stretching the function Dynamic range, forming the reference channel Corrected version of in, and are the maximum and minimum values ​​of the input underwater images respectively; use Compensate the first channel to be compensated The second channel to be compensated Get the corrected first channel and the first channel after correction in, for The average grayscale value of for The average grayscale value of Represents the grayscale average of the reference channel; Represents the grayscale average of the red, green, and blue channels respectively.

4. The underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion according to claim 1, characterized in that: Step (3) specifically refers to: first, using the Gaussian convolution mask G and the brightness channel image I in the HSV color space v The convolution operation between them generates the original base layer containing the haze effect In the formula, the Gaussian convolution mask G is expressed as: Where σ is the standard deviation of the distribution and (x,y) is the pixel coordinate; Using the brightness channel image I v and the original base layer containing the haze effect The difference between the first scale detail layer of the Gaussian pyramid is obtained. Then the brightness channel image I v Downsample to obtain the brightness channel image I of the second scale v,2 , and then calculate the second scale detail layer according to the following formula In the formula, is the second scale base layer; Finally, the brightness channel image I of the second scale is v,2 Downsample to obtain the brightness channel image I of the third scale v,3 , the third scale detail layer is calculated according to the following formula In the formula, It is the third scale base layer.

5. The underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion according to claim 1, characterized in that: Step (4) specifically refers to: the original base layer including the haze effect is expressed as follows: In the formula, It is a clear base layer. refers to the global backscattered light; represents the scene transmittance on the V channel, By taking the original transmittance of the underwater scene t c Convert from RGB space to HSV space and extract the V channel component; when hour, choose The pixel coordinates of the top 0.1% of the smallest pixels are used to estimate the global backscattered light: in, means The pixel value of the largest pixel in the top 99.9% of the pixels, (x*, y*) is The pixel coordinates of the smallest pixels in the top 0.1% of pixels, means The pixel value at the (x*,y*) coordinate; For the scale detail layer, it is assumed that the scene transmittance is consistent in the local block area of ​​the underwater image. The underwater physical imaging model shows the inverse relationship between the gradient of the underwater image and the scene transmittance: in, represents the gradient operator; Using the original transmittance t of the underwater scene c Zoom in underwater image I c Gradient To enhance the clear image c gradient The operation of formula (1) is transformed to the gradient domain on the detail layer, and the gradient enhancement of the detail layer is expressed as follows: in, refers to the original gradient of the i-th scale detail layer, ω is a non-negative parameter that limits the gradient enhancement strength of different detail layers. With the help of Poisson's equation, the enhanced detail layer of each scale is obtained according to the following modified gradient: In the formula, Restore refers to the gradient restoration operation.

6. The underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion according to claim 1, characterized in that: Step (5) specifically refers to: according to the enhanced detail layer, the first, second and third scale detail layers are weighted summed using the following formula to obtain the enhanced detail layer: Among them, sgn is a symbolic function; w1, w2, w3 are three composite parameters of different levels; and The enhanced detail layers represent the first, second, and third scale detail layers respectively; Based on the clear base layer and Enhanced Detail Layer The reconstructed clear image J of the brightness channel V is derived by the following formula: V (x,y): Finally, by adding J V The (x, y) and H and S channels of the underwater image are fused to form an enhanced image in the HSV space, and then the enhanced image in the HSV space is converted into RGB color space expression to generate the final enhanced result.

7. The underwater image enhancement method based on segmented color balance and multi-scale enhancement fusion according to claim 5, characterized in that: In step (4), it is known that the underwater physical imaging model consists of two parts: the irradiance D of the reflecting object c and backscattered light irradiance A c , the calculation formulas of the two are as follows: D c (x,y)=J c (x,y)t c (x,y),c∈{r,g,b}(2) Where (x, y) is the pixel coordinate, J c For clear images, is the global air light constant, t c is the original transmittance of the underwater scene. According to equations (2) and (3), the underwater physical imaging model is expressed as: Among them, I c is an underwater image; when in the underwater image I c After color correction, the global backscattered light of the RGB channels is almost the same and has a pixel value close to 1. Therefore, for the color-corrected image, equation (3) is rewritten as: TO c (x,y)=1-t c (x,y)(5) According to the underwater physical imaging model, the backscattered light irradiance A c Each pixel value of is higher than 0, but cannot be higher than I c The minimum value of the component, under this premise, define an I c The minimum channel, which consists of the minimum pixel value of each pixel between the r, g, and b channels, that is: I m (x,y)=min{I r (x,y),I g (x,y),I b (x,y)} Among them, I m (x, y) refers to the minimum channel composed of the minimum values ​​of the r, g, and b channels of the underwater image, min{I r (x,y),I g (x,y),I b (x,y)} represents the selection of the minimum value among the r, g, and b channels; I r (x,y),I g (x,y),I b (x, y) represents the pixel value of the r, g, and b channels at the pixel coordinate (i, j) respectively; since the underwater image has a similar backscattered light in a small local area, a median filter is used to make each pixel have the same backscattered light: Among them, s v is the window size of the median filter; M(x,y) refers to the minimum channel for which the backscattering is approximately uniform; The residual structure and texture information in M(x,y) are further filtered out using the following equation: Where A is the rough backscattered light; A scaling factor is designed to compensate for the local backscattered illumination, which is given by the following equation: Among them, MeanA and MinA are the average and minimum values ​​of the rough backscattered light A respectively; Thres is the threshold, which determines whether the backscattered light needs to be compensated, Thres = MaxA; then according to the Lambert-Beer empirical law, the degradation of light intensity is related to the properties of the material, and light propagates with exponential dependence; at the same time, considering the opposite iterative process between the exponential function and the power function, using A = A ratio to compensate for the backscattered light; finally, with the help of the improved backscattered light, the original transmittance of the underwater scene is estimated according to the following formula, namely: t c (x,y)=1-A c (x,y) In the formula, A c (x, y) refers to the compensated backscattered light; The original transmittance of the underwater scene t c Transformation to V channel spatial expression formation And through the estimated scene transmittance on the V channel and global backscatter Combined with the original base layer containing the haze effect Remove the original base layer containing the haze effect The haze effect is clear.

Images