Underwater image enhancement method, system and equipment combined with attenuation channel correction

Abstract

The invention relates to an underwater image enhancement method, system and device combined with attenuation channel correction, and the method comprises the following steps: carrying out the color correction of an image through employing an attenuation channel adaptive correction white balance technology, and recording the image as a first enhancement image I1; de-noising the first enhanced image I1 by using a fast non-local mean algorithm based on a local block, and recording the de-noised first enhanced image I1 as a second enhanced image I2; improving the contrast ratio of the first enhanced image I1 by adopting a self-adaptive stretching method based on the local characteristics of a histogram, and recording the first enhanced image I1 as a third enhanced image I3; and obtaining a final enhanced image by using a bilateral weight fusion strategy in combination with the complementary advantages of the second enhanced image I2 and the third enhanced image I3. The system and the equipment are used for executing the method. According to the method, qualitative and quantitative experiment results on different underwater image reference data sets show that the contrast of the image can be uniformly improved and details and textures of the image can be enhanced while the color of the image is corrected.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image processing, and in particular to an underwater image enhancement method, system and device combined with attenuation channel correction. Background Art

[0002] In recent years, underwater imaging has played a vital role in underwater vehicles, deep-sea exploration, and underwater target identification. Underwater images are an important information carrier for obtaining marine resource information, but due to the complex and diverse underwater imaging environment, coupled with wavelength-related light absorption and scattering, underwater images have color deviation, low contrast, and blurred details.

[0003] Specifically, when light propagates in water, it will experience different degrees of attenuation. Generally speaking, blue light and green light propagate relatively well in water, while red light and other wavelengths of light are more easily absorbed. Therefore, the light reflected by underwater objects contains relatively more blue-green components, resulting in the unique blue-green appearance of underwater images. At the same time, particles and suspended matter in aquatic media can cause light scattering, causing light to diverge in multiple directions, ultimately causing the contrast of underwater images to decrease and damaging the overall quality of the image.

[0004] In the existing technology, the improvement of underwater image quality mainly starts from four directions: hardware-based, restoration-based, deep learning-based and enhancement-based. Due to the instability of the aquatic environment and the high cost of equipment, the improvement direction based on hardware is difficult and costly. The main limitation of the restoration-based method is that it is difficult to adapt to the complex underwater environment and it is heavily dependent on prior knowledge and parameter selection. The deep learning-based method has the problems of limited generalization ability, complex parameter setting and high computational complexity.

[0005] Compared with the above three methods, the enhancement-based improvement has the advantages of being easy to implement and highly efficient. However, it is difficult for the enhancement-based method to comprehensively solve various problems in underwater images, which leads to over-enhancement or under-enhancement in the enhanced images.

[0006] For example, the Chinese patent application document with application number "202310657636.2" discloses "a multi-scale fusion underwater image enhancement method combined with adaptive gamma correction", which performs color correction and contrast enhancement on the image through white balance and adaptive gamma correction respectively, and then uses the fusion algorithm to obtain the final enhanced image. Although the above method can perform color correction and contrast enhancement on the original image, it does not take into account the noise in the original image, so the final enhanced image does not have prominent details due to the presence of noise.

[0007] In summary, the existing underwater image enhancement methods do not fully consider the image noise problem, resulting in mediocre enhancement effects. Summary of the invention

[0008] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide an underwater image enhancement method, system and device combined with attenuation channel correction. The present invention corrects the color of the original image through white balance technology to solve the color cast problem, adopts a fast non-local mean algorithm based on local blocks to denoise the color-corrected image, and adopts an adaptive stretching method based on local features of histogram to enhance contrast. Finally, a bilateral weight fusion strategy is introduced to combine the complementary advantages of denoised images and contrast-enhanced images, effectively enhance underwater images, and highlight the details and edge textures of the image.

[0009] The underwater image enhancement method combined with attenuation channel correction described in the present invention comprises the following steps:

[0010] S01, obtaining an original image, performing white balance processing on the original image to perform color correction, and obtaining an image after color correction as a first enhanced image I1;

[0011] S02, processing the first enhanced image I1 by a non-local means denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, and enhancing the contrast of the first enhanced image I1 by an adaptive stretching method to obtain a contrast-enhanced image recorded as a third enhanced image I3;

[0012] S03, calculating and obtaining the saliency weight map, contrast weight map and gradient weight map of the second enhanced image I2 and performing weight normalization processing to obtain a first normalized weight map; calculating and obtaining the saliency weight map, noise weight map and gradient weight map of the third enhanced image I3 and performing weight normalization processing to obtain a second normalized weight map;

[0013] S04, decomposing the second enhanced image I2 and the third enhanced image I3 into Laplacian pyramid structures, decomposing the first normalized weight map and the second normalized weight map into Gaussian pyramid structures, and performing bilateral weight fusion on the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image.

[0014] Preferably, step S01 includes the following steps:

[0015] S011. Calculate the index parameters of the RGB channel of the original image according to the following formula, wherein the index parameters include information entropy H, interval length DR, and average difference ADD:

[0016]

[0017] ADD c =(|n2-n1|+|n3-n2|+…+|ni+1 -n i |) / L,

[0018] Where c = R or G or B, H c represents the information entropy of the c channel, represents the probability of the i-th pixel level of the c channel;

[0019] DR c Indicates the length of the interval where the number of pixels in the c channel is greater than or equal to μ, n i Indicates the number of pixels at the i-th pixel level, min(n i ≥μ) and max(n i ≥μ) respectively means when n i ≥μ, the minimum and maximum values ​​of i, and L represents the total number of pixel levels;

[0020] ADD c represents the average difference of the c channel;

[0021] S012, assigning weights to the index parameters, and constructing compensation parameters cp for each channel in the original image:

[0022] cp c =δ H H c +δ DR DR c +δ ADD (1 / ADD c ),

[0023] Among them, cp c represents the compensation parameter of the c channel, δ H , δ DR and δ ADD They represent the weight coefficients of information entropy H, interval length DR and average difference ADD respectively;

[0024] S013, sort the RGB channels according to the value of the compensation parameter cp, and record them in descending order as follows: the first channel C first , Second channel C second and the third channel C third , according to the compensation parameters cp of each channel c Assign values ​​to each channel, and construct compensation factors and compensation formulas based on the sorting and assignment results:

[0025]

[0026] Among them, cf1 and cf2 represent the first compensation factor and the second compensation factor respectively, C s ′ econdand C t ′ hird They represent the second channel and the third channel after compensation respectively;

[0027] S014. Process the compensated image using a gray world algorithm to obtain the first enhanced image I1 after color correction.

[0028] Preferably, in step S02, the first enhanced image I1 is processed by a non-local means denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, which specifically includes the following steps:

[0029] S021a, reading the first enhanced image I1, defining the length of a local block as p and the width as q;

[0030] S022a, initializing an image of the same size as the first enhanced image I1 as a blank image;

[0031] S023a. Calculate the similarity weight in the local block according to the following formula:

[0032]

[0033] Wherein, w(i,j,m,n) represents the similarity weight between the target pixel (i,j) and the similar pixel (m,n), I(i,j) represents the pixel value of the target pixel (i,j) in the local block, I(m,n) represents the pixel value of the similar pixel (m,n) in the local block, and h is the weight parameter that controls the influence of the similarity weight.

[0034] S024a, calculate the weighted average of similar pixels in the local block according to the similarity weight, and obtain the denoising result FNLM(i,j):

[0035]

[0036] S025a, reading the three channels R, G, and B in parallel, and calculating the denoising result FNLM(i, j) on each channel, storing the denoising result FNLM(i, j) of each channel in the blank image, and obtaining the second enhanced image I2.

[0037] Preferably, in step S02, the contrast of the first enhanced image I1 is enhanced by an adaptive stretching method, and the image after contrast enhancement is obtained is recorded as a third enhanced image I3, which specifically includes the following steps:

[0038] S021b, reading the RGB channel histogram of the first enhanced image I1, and calculating the exposure value and exposure threshold of the obtained RGB channel histogram:

[0039]

[0040] ET=(1-EX)(MAX-MIN)+MIN,

[0041] Wherein, EX represents the exposure value, ET represents the exposure threshold, and ET is used as the segmentation point to segment the RGB channel histogram into two sub-histograms: a first sub-histogram and a second sub-histogram. The distribution range of the first sub-histogram is in the low-exposure area of ​​[MIN, ET], and the distribution range of the second sub-histogram is in the over-exposure area of ​​[ET, MAX]. i represents the number of pixels at the i-th pixel level, and μ represents the pixel number threshold;

[0042] S022b, using the median of the RGB channel histogram as the clipping threshold to clip the first sub-histogram and the second sub-histogram, clipping the part where the total number of pixels is greater than the clipping threshold, leaving the part where the total number of pixels is less than or equal to the clipping threshold unchanged, redistributing the total number of pixels after clipping to the entire interval of the sub-histogram, retaining the image information, and the calculation formula of the clipping process is as follows:

[0043]

[0044] Among them, Hist(i) represents the cropped subhistogram, Q represents the total number of cropped pixels, M represents the median of the subhistogram, and l represents the interval length of the subhistogram;

[0045] S023b. According to the interval length ratio of the sub-histogram before and after stretching, a stretching factor is constructed, and the sub-histogram after stretching is obtained according to the rule of proportional stretching:

[0046]

[0047] Among them, θ low and θ high Respectively represent the stretching factors of the first subhistogram and the second subhistogram, Hist′ low (i) and Hist′ high (i) represent the stretched sub-histograms, and the corresponding stretching ranges are [0, ET] and [ET, L] respectively;

[0048] S024b, combining the two sub-histograms after the stretching process to obtain an image after contrast enhancement, which is the third enhanced image I3.

[0049] Preferably, step S03 includes the following steps:

[0050] S031. Calculate the significance weights of the second enhanced image I2 and the third enhanced image I3 according to the following formula:

[0051] WS,k (i,j)=(L k (i,j)-ML k ) 2 +(A k (i,j)-MA k ) 2 +(B k (i,j)-MB k ) 2 ,

[0052] Among them, (i, j) represents the target position, W S,k represents the saliency weight of the kth input image, L k , A k and B k They represent the brightness channel, red and green channel, and yellow and blue channel in the CIELAB color space, respectively. k 、MA k and MB k They are the average values ​​corresponding to the brightness channel, red and green channels, and yellow and blue channels respectively;

[0053] The gradient weights of the second enhanced image I2 and the third enhanced image I3 are calculated according to the following formula:

[0054]

[0055] Among them, (i, j) represents the target position, W G,k Represents the gradient weight of the image, G x and G y Represent the gradient of the image in the horizontal and vertical directions respectively;

[0056] The noise weight of the third enhanced image I3 is calculated according to the following formula:

[0057]

[0058] Among them, (i, j) represents the target position, W N is the noise weight, W N (i, j) represents the local noise level at position (i, j), W H and W W hW are the height and width of the local window respectively. H represents the size of the half window, I(is,jt) represents the pixel value of the pixel (is,jt) in the local window, μ ij represents the mean of the local window;

[0059] S032. Normalize each enhancement map by dividing the sum of the weights of each enhancement map by the sum of the weights of all enhancement maps. The calculation formula is as follows:

[0060]

[0062] Among them, W nor,k represents the normalized weight map of the kth input image, W k represents the weighted cumulative sum of the k-th input image, represents the regularization term.

[0063] Preferably, in step S04, bilateral weight fusion is performed on the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image, specifically:

[0064]

[0065] Among them, I enhanced (x,y) represents the enhanced image, represents the qth layer weight map of the kth input image after Gaussian pyramid decomposition, represents the qth layer image of the kth input image after Laplacian pyramid decomposition, and N represents the number of decomposition layers.

[0066] Preferably, the number of decomposed layers N=3.

[0067] An underwater image enhancement system combined with attenuation channel correction of the present invention comprises:

[0068] A color correction module is used to obtain an original image, perform white balance processing on the original image to perform color correction, and obtain an image after color correction as a first enhanced image I1;

[0069] an enhancement module, which is used to process the first enhanced image I1 by a non-local mean denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, and to enhance the contrast of the first enhanced image I1 by an adaptive stretching method to obtain a contrast-enhanced image recorded as a third enhanced image I3;

[0070] A normalization processing module, which is used to calculate and obtain the saliency weight map, contrast weight map and gradient weight map of the second enhanced image I2 and perform weight normalization processing to obtain a first normalized weight map; calculate and obtain the saliency weight map, noise weight map and gradient weight map of the third enhanced image I3 and perform weight normalization processing to obtain a second normalized weight map;

[0071] A fusion module is used to decompose the second enhanced image I2 and the third enhanced image I3 into Laplacian pyramid structures, decompose the first normalized weight map and the second normalized weight map into Gaussian pyramid structures, and perform bilateral weight fusion of the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image.

[0072] A computer device of the present invention comprises a processor and a memory connected by signals, wherein the memory stores at least one instruction or at least one program, and when the at least one instruction or the at least one program is loaded by the processor, the underwater image enhancement method combined with attenuation channel correction as described above is executed.

[0073] A computer-readable storage medium of the present invention stores at least one instruction or at least one program, and when the at least one instruction or the at least one program is loaded by a processor, the underwater image enhancement method combined with attenuation channel correction as described above is executed.

[0074] The underwater image enhancement method, system and device combined with attenuation channel correction described in the present invention have the advantages that, starting from the four problems of color distortion, low contrast, increased noise and blurred details existing in underwater images, the present invention first uses the white balance algorithm of attenuation channel adaptive correction to balance the color of the original image, and then first performs a non-local mean denoising algorithm based on local blocks on the color-corrected image to remove local noise, and then performs an adaptive stretching method based on the local characteristics of the histogram to improve the contrast. Finally, a bilateral weight fusion strategy is used to combine the complementary advantages of the denoised image and the contrast-enhanced image to effectively enhance the underwater image, while highlighting the details and edge textures in the image.

[0075] Unlike the methods based on restoration and deep learning, the method of the present invention does not require the establishment of complex physical models and the processing of large amounts of data. At the same time, compared with the existing underwater image enhancement methods, the present invention fully considers the noise of the original image, adopts the non-local mean denoising based on local blocks to effectively remove the noise of the original image, and calculates the gradient weight, contrast weight and significance weight for the subsequent normalization processing and bilateral fusion in combination with the characteristics of the denoised image, and adaptively stretches the first enhanced image in the other direction in parallel, and calculates the noise weight for the third enhanced image obtained by stretching, so that the image noise problem is fully considered in the subsequent bilateral fusion process, so that the underwater image enhancement effect of the present invention is far superior to the existing underwater image enhancement algorithm, especially capable of highlighting the details and edge texture of the image. A large number of experimental results conducted on multiple benchmark underwater image data sets have proved the efficiency and robustness of the proposed method. In addition, the method of the present invention also shows a significant enhancement effect on low-light and foggy images. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a flowchart of the steps of the underwater image enhancement method described in this embodiment;

[0077] Figure 2 is the original image processed by this embodiment;

[0078] Figure 3 is an enhanced diagram after being processed by each process method of this embodiment, wherein: Figure 3-1 represents the first enhanced image I1 after adaptive correction of the attenuation channel; Figure 3-2 represents the second enhanced image I2 after being processed by the non-local mean denoising algorithm; Figure 3-3 represents the adaptive stretching third enhancement image I3 based on the local characteristics of the histogram, Figure 3-4 Represents the enhanced image of the bilateral weighted fusion algorithm;

[0079] Figure 4 is the second enhanced graph I2 and its weight graphs, where Figure 4-1 represents the second enhanced image I2, Figure 4-2 represents the significance weight map of I2, Figure 4-3 represents the contrast weight map of I2, Figure 4-4 represents the gradient weight map of I2, Figure 4-5 is the first normalized weight map of I2;

[0080] Figure 5 is the third enhanced graph I3 and its weight graphs, where Figure 5-1 represents the third enhanced graph I3, Figure 5-2 represents the significance weight map of I3, Figure 5-3 represents the noise weight map of I3, Figure 5-4 represents the gradient weight map of I3, Figure 5-5 is the second normalized weight map of I3;

[0081] Figure 6 The color and contrast enhancement comparison diagram of the method in this embodiment and other typical underwater degraded images is shown in FIG. Figure 6-1 represents the original underwater image, Figure 6-2 This is the result graph of HLRP processing. Figure 6-3 The result graph of PCDE processing is shown in Figure 2. Figure 6-4 It represents the result graph processed by FUnIE-GAN. Figure 6-5 This is the result graph processed by UIEC^2-Net. Figure 6-6 The result graph of CBAF processing is shown in Figure 2. Figure 6-7 It represents the result graph of MMLE processing. Figure 6-8 A diagram showing the result of processing using the method of this embodiment;

[0082] Figure 7 The following is a comparison of the method in this embodiment and other typical underwater degraded image detail enhancement diagrams, where: Figure 7-1 represents the original underwater image, Figure 7-2 This is the result graph of HLRP processing. Figure 7-3 The result graph of PCDE processing is shown in Figure 2. Figure 7-4 It represents the result graph processed by FUnIE-GAN. Figure 7-5 This is the result graph processed by UIEC^2-Net. Figure 7-6 The result graph of CBAF processing is shown in Figure 2. Figure 7-7 It represents the result graph of MMLE processing. Figure 7-8 A diagram showing the result of processing using the method of this embodiment;

[0083] Figure 8 It is a schematic diagram of the structure of the computer device described in this embodiment.

[0084] Description of reference numerals: 101 - processor, 102 - memory. DETAILED DESCRIPTION

[0085] like Figure 1 As shown, the underwater image enhancement method combined with attenuation channel correction described in the present invention comprises the following steps:

[0086] S01. Obtain the original image, and use the white balance technology of attenuation channel adaptive correction to perform color correction on the original underwater image to solve the color cast problem. The specific steps are as follows:

[0087] S011. Calculate the index parameters of the RGB channel of the original image according to the following formula, wherein the index parameters include information entropy H, interval length DR, and average difference ADD:

[0088]

[0089] ADD c =(|n2-n1|+|n3-n2|+…+|n i+1 -n i |) / L,

[0090] Where c = R or G or B, H c represents the information entropy of the c channel, represents the probability of the i-th pixel level of the c channel;

[0091] DR c Indicates the length of the interval where the number of pixels in the c channel is greater than or equal to μ, n i Indicates the number of pixels at the i-th pixel level, min(n i ≥μ) and max(n i ≥μ) respectively means when n i ≥μ, the minimum and maximum values ​​of i, and L represents the total number of pixel levels;

[0092] ADD c represents the average difference of the c channel;

[0093] S012, assigning weights to the index parameters, and constructing compensation parameters cp for each channel in the original image:

[0094] cp c =δ H H c +δ DR DR c +δ ADD (1 / ADD c ),

[0095] Among them, cp c represents the compensation parameter of the c channel; δ H , δ DR and δ ADD They represent the weight coefficients of information entropy H, interval length DR and average difference ADD respectively; for example, the weight coefficients of the three indicators are assigned as δ H =0.0908,δ DR =0.0009,δ ADD =0.9083.

[0096] S013, sort the RGB channels according to the value of the compensation parameter cp, and record them in descending order as follows: the first channel C first , Second channel C second and the third channel C third , according to the compensation parameters cp of each channel c Assign values ​​to each channel, for example, cp R >cp G >cp B , then let C R =C first ,cp R =cp first , and so on.

[0097] Construct compensation factors and compensation formulas based on the sorting and assignment results:

[0098]

[0099] Among them, cf1 and cf2 represent the first compensation factor and the second compensation factor respectively, C s ′ econd and C t ′ hird They represent the second channel and the third channel after compensation respectively;

[0100] S014, the compensated image is processed by a gray world algorithm, such as Figure 3-1As shown, the first enhanced image I1 after color correction is obtained.

[0101] S02, processing the first enhanced image I1 by a non-local means denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, and enhancing the contrast of the first enhanced image I1 by an adaptive stretching method to obtain a contrast-enhanced image recorded as a third enhanced image I3;

[0102] In step S02, the first enhanced image I1 is processed using a local block-based non-local mean denoising algorithm to eliminate local noise in the image and highlight the details of the image, and the denoised image is obtained as a second enhanced image I2, which specifically includes the following steps:

[0103] S021a, reading the first enhanced image I1, defining the length of a local block as p and the width as q;

[0104] S022a, initializing an image of the same size as the first enhanced image I1 as a blank image;

[0105] S023a. Calculate the similarity weight in the local block according to the following formula:

[0106]

[0107] Wherein, w(i,j,m,n) represents the similarity weight between the target pixel (i,j) and the similar pixel (m,n), I(i,j) represents the pixel value of the target pixel (i,j) in the local block, I(m,n) represents the pixel value of the similar pixel (m,n) in the local block, and h is the weight parameter that controls the influence of the similarity weight.

[0108] S024a, calculate the weighted average of similar pixels in the local block according to the similarity weight, and obtain the denoising result FNLM(i,j):

[0109]

[0110] S025a, use the parfor function to read the three channels R, G, and B in parallel, and calculate the denoising result FNLM(i, j) on each channel, and store the denoising result FNLM(i, j) of each channel in the blank image, such as Figure 3-2 As shown, the second enhanced image I2 is obtained.

[0111] In step S02, the contrast of the first enhanced image I1 is enhanced by using an adaptive stretching method based on local characteristics of the histogram, and the image after contrast enhancement is obtained is recorded as a third enhanced image I3, which specifically includes the following steps:

[0112] S021b, reading the RGB channel histogram of the first enhanced image I1, and calculating the exposure value and exposure threshold of the obtained RGB channel histogram:

[0113]

[0114] ET=(1-EX)(MAX-MIN)+MIN,

[0115] Wherein, EX represents the exposure value, ET represents the exposure threshold, and ET is used as the segmentation point to segment the RGB channel histogram into two sub-histograms: a first sub-histogram and a second sub-histogram. The distribution range of the first sub-histogram is in the low-exposure area of ​​[MIN, ET], and the distribution range of the second sub-histogram is in the over-exposure area of ​​[ET, MAX]. i represents the number of pixels at the i-th pixel level, and μ represents the pixel number threshold;

[0116] S022b. Use the median of the RGB channel histogram as a clipping threshold to clip the first sub-histogram and the second sub-histogram. The clipping rule is as follows:

[0117] The part with a total number of pixels greater than the clipping threshold is clipped to remove the overly bright or dark areas in the image;

[0118] The part whose total number of pixels is less than or equal to the cropping threshold remains unchanged;

[0119] The total number of pixels after clipping is redistributed to the entire interval of the sub-histogram to retain the image information. The calculation formula of the clipping process is as follows:

[0120]

[0121] Among them, Hist(i) represents the cropped subhistogram, Q represents the total number of cropped pixels, M represents the median of the subhistogram, and l represents the interval length of the subhistogram;

[0122] S023b. According to the interval length ratio of the sub-histogram before and after stretching, a stretching factor is constructed, and the sub-histogram after stretching is obtained according to the rule of proportional stretching:

[0123]

[0124] Among them, θ low and θ high Respectively represent the stretching factors of the first subhistogram and the second subhistogram, Hist′ low (i) and Hist′ high (i) represent the stretched sub-histograms, and the corresponding stretching ranges are [0, ET] and [ET, L] respectively;

[0125] S024b, using a direct superposition method, the two sub-histograms after stretching are combined to obtain a contrast-enhanced image, such as Figure 3-3 As shown, it is the third enhanced image I3. S03, calculate and obtain the saliency weight map, contrast weight map and gradient weight map of the second enhanced image I2 and perform weight normalization processing to obtain a first normalized weight map; calculate and obtain the saliency weight map, noise weight map and gradient weight map of the third enhanced image I3 and perform weight normalization processing to obtain a second normalized weight map;

[0126] Step S03 includes the following steps:

[0127] S031. Calculate the significance weights of the second enhanced image I2 and the third enhanced image I3 according to the following formula:

[0128] W S,k (i,j)=(L k (i,j)-ML k ) 2 +(A k (i,j)-MA k ) 2 +(B k (i,j)-MB k ) 2 ,

[0129] Among them, (i, j) represents the target position, W S,k represents the saliency weight of the kth input image, L k , A k and B k They represent the brightness channel, red and green channel, and yellow and blue channel in the CIELAB color space, respectively. k 、MA k and MB k They are the average values ​​corresponding to the brightness channel, red and green channels, and yellow and blue channels respectively;

[0130] The gradient weights of the second enhanced image I2 and the third enhanced image I3 are calculated according to the following formula:

[0131]

[0132] Among them, (i, j) represents the target position, W G,k Represents the gradient weight of the image, G x and G y Represent the gradient of the image in the horizontal and vertical directions respectively;

[0133] The noise weight of the third enhanced image I3 is calculated according to the following formula:

[0134]

[0135] Among them, (i, j) represents the target position, W N is the noise weight, W N (i, j) represents the local noise level at position (i, j), W H and W W hW are the height and width of the local window respectively. H represents the size of the half window, I(is,jt) represents the pixel value of the pixel (is,jt) in the local window, μ ij represents the mean of the local window;

[0136] The contrast weight of the second enhanced image I2 is calculated using a weight based on global contrast. Exemplarily, the weight calculation method based on global contrast is as follows:

[0137]

[0138] Among them, W C (i, j) is the contrast weight, (i, j) represents the target position, I(i, j) represents the pixel value of the target position, represents the global average gray value of the image, It represents the maximum absolute value of the difference between all pixels in the whole image and the global average grayscale value, which is used for normalization calculation.

[0139] S032, dividing the sum of the weights of each enhancement map by the sum of the weights of all enhancement maps to normalize each enhancement map, the first normalized weight map and the second normalized weight map are respectively as follows: Figure 4-5 As shown in Figure 5-5, the calculation formula is as follows:

[0140]

[0141] Among them, W nor,k represents the normalized weight map of the kth input image, W k represents the weighted cumulative sum of the k-th input image, represents a regularization term that aims to ensure that each input contributes to the output.

[0142] S04, decomposing the second enhanced image I2 and the third enhanced image I3 into Laplacian pyramid structures, decomposing the first normalized weight map and the second normalized weight map into Gaussian pyramid structures, and performing bilateral weight fusion on the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image.

[0143] In step S04, bilateral weight fusion is performed on the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image, specifically:

[0144]

[0145] Among them, I enhanced (x,y) represents the enhanced image, represents the qth layer weight map of the kth input image after Gaussian pyramid decomposition, represents the qth layer image of the kth input image after Laplacian pyramid decomposition, N represents the number of decomposition layers, in a specific embodiment, N=3, that is, the number of decomposition layers of Gaussian pyramid and Laplacian pyramid are both 3, and the final enhanced underwater image effect diagram is as follows Figure 3-4 shown.

[0146] Experimental example

[0147] In order to verify the high efficiency of the present invention for underwater image enhancement, three groups of typical underwater degraded images are selected from the underwater image benchmark datasets of different scenes for experiments, and the following algorithms in the prior art are selected for comparative experiments:

[0148] HLRP (Underwater Image Enhancement With Hyper-Laplacian ReflectancePriors),

[0149] PCDE (Underwater Image Enhancement via Piecewise Color Correction andDual Prior Optimized Contrast Enhancement),

[0150] FUnIE-GAN (Fast Underwater Image Enhancement for Improved VisualPerception),

[0151] UIEC^2-Net(UIEC^2-Net:CNN-based underwater image enhancement using two color space),

[0152] CBAF (Color Balance and Fusion for Underwater Image Enhancement), MMLE (Underwater Image Enhancement via Minimal Color Loss and Locally AdaptiveContrast Enhancement).

[0153] like Figure 6 As shown, this embodiment provides experimental results of processing typical underwater degraded images with other algorithms. Figure 6-1 This is a typical degraded original image under water. Figure 6-2 As can be seen from Figure 6-3, the HLRP and PCDE algorithms fail to solve the problem of image color cast, and some areas have over-enhancement problems. Figure 6-4 It can be seen that the FUnIE-GAN algorithm introduces unnecessary red artifacts into the processing results of the blue image; Figure 6-5 It can be seen that although the UIEC^2-Net algorithm corrects some color cast problems, it does not enhance the contrast of the image. Figure 6-6 It can be seen that while the CBAF algorithm enhances the image contrast, the image details are lost. Figure 6-7 It can be seen that the MMLE algorithm has local blue artifacts and the image contrast is not obvious. Figure 6-8 It can be seen that the algorithm proposed in this embodiment can eliminate the color cast of the image while enhancing the contrast and brightness of the image.

[0154] At the same time, from Figure 7 The results show that the algorithm proposed in the present invention highlights the local details of the image and obtains an enhanced image with better visual effect.

[0155] In order to avoid the deviation caused by subjective qualitative evaluation, this embodiment uses five representative image quality evaluation indicators for quantitative analysis. They are AG, IE, PCQI, UIQM and UCIQE. Among them, AG reflects the richness of image information and the slight changes in image details. IE can quantitatively describe the richness of image color. The higher the AG or IE score, the better the visual quality of the image. PCQI describes the contrast and structural information of the image. The higher its value, the better the contrast of the image. UIQM and UCIQE are comprehensive indicators that describe multiple aspects of image quality. The higher the UIQM or UCIQE score, the higher the overall quality of the enhanced image. Table 1 is Figure 6 and Figure 7 The average results of AG, IE, PCQI, UIQM and UCIQE for all images, where bold indicates the maximum value of the corresponding indicator.

[0156] Table 1

[0157]

[0158] The data in Table 1 show that the method proposed in this embodiment is superior to other algorithms in all five indicators. From the qualitative and quantitative comprehensive results, it can be seen that the method proposed in this embodiment has fully solved the four problems of color distortion, low contrast, noise enhancement and blurred details in underwater images. The enhanced image does not have the problem of color cast, and the image has a good visual effect and prominent local details.

[0159] The present invention starts from the four problems of color distortion, low contrast, increased noise and blurred details in underwater images. First, the white balance algorithm of attenuation channel adaptive correction is used to balance the color of the original image. Next, the color-corrected image is first processed by the non-local mean denoising algorithm based on local blocks to remove local noise, and then the adaptive stretching method based on the local characteristics of the histogram is processed to improve the contrast. Finally, the bilateral weight fusion strategy is used to combine the complementary advantages of the denoised image and the contrast-enhanced image, effectively enhance the underwater image, and highlight the details and edge textures in the image.

[0160] Unlike the methods based on restoration and deep learning, the method of the present invention does not require the establishment of complex physical models and the processing of large amounts of data. At the same time, compared with the existing underwater image enhancement methods, the present invention fully considers the noise of the original image, adopts the non-local mean denoising based on local blocks to effectively remove the noise of the original image, and calculates the gradient weight, contrast weight and significance weight for the subsequent normalization processing and bilateral fusion in combination with the characteristics of the denoised image, and adaptively stretches the first enhanced image in the other direction in parallel, and calculates the noise weight for the third enhanced image obtained by stretching, so that the image noise problem is fully considered in the subsequent bilateral fusion process, so that the underwater image enhancement effect of the present invention is far superior to the existing underwater image enhancement algorithm, especially capable of highlighting the details and edge texture of the image. A large number of experimental results conducted on multiple benchmark underwater image data sets have proved the efficiency and robustness of the proposed method. In addition, the method of the present invention also shows a significant enhancement effect on low-light and foggy images.

[0161] This embodiment also provides an underwater image enhancement system combined with attenuation channel correction, including:

[0162] A color correction module is used to obtain an original image, perform white balance processing on the original image to perform color correction, and obtain an image after color correction as a first enhanced image I1;

[0163] an enhancement module, which is used to process the first enhanced image I1 by a non-local mean denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, and to enhance the contrast of the first enhanced image I1 by an adaptive stretching method to obtain a contrast-enhanced image recorded as a third enhanced image I3;

[0164] A normalization processing module, which is used to calculate and obtain the saliency weight map, contrast weight map and gradient weight map of the second enhanced image I2 and perform weight normalization processing to obtain a first normalized weight map; calculate and obtain the saliency weight map, noise weight map and gradient weight map of the third enhanced image I3 and perform weight normalization processing to obtain a second normalized weight map;

[0165] A fusion module is used to decompose the second enhanced image I2 and the third enhanced image I3 into Laplacian pyramid structures, decompose the first normalized weight map and the second normalized weight map into Gaussian pyramid structures, and perform bilateral weight fusion of the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image.

[0166] The underwater image enhancement system of this embodiment and the aforementioned underwater image enhancement method belong to the same inventive concept, which can be understood with reference to the above description and will not be described in detail here.

[0167] like Figure 8 As shown, this embodiment also provides a computer device, including a processor 101 and a memory 102 connected by bus signals, wherein the memory 102 stores at least one instruction or at least one program, and the at least one instruction or at least one program is executed when loaded by the processor 101. The memory 102 can be used to store software programs and modules, and the processor 101 executes various functional applications by running the software programs and modules stored in the memory 102. The memory 102 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory 102 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 102 can also include a memory controller to provide the processor 101 with access to the memory 102.

[0168] The method embodiments provided in the embodiments of the present invention may be executed in a computer terminal, a server or a similar computing device, that is, the above-mentioned computer device may include a computer terminal, a server or a similar computing device. The internal structure of the computer device may include but is not limited to: a processor, a network interface and a memory. Among them, the processor, network interface and memory in the computer device may be connected via a bus or other means.

[0169] Among them, the processor 101 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device. The network interface may optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.). The memory 102 (Memory) is a memory device in the computer device for storing programs and data. It can be understood that the memory 102 here can be a high-speed RAM storage device, or a non-volatile storage device (non-volatile memory), such as at least one disk storage device; optionally, it can also be at least one storage device located away from the aforementioned processor 101. The memory 102 provides a storage space, which stores the operating system of the electronic device, which may include but is not limited to: Windows system (an operating system), Linux (an operating system), Android (Android, a mobile operating system) system, IOS (a mobile operating system) system, etc., and the present invention is not limited to this; and, in the storage space, one or more instructions suitable for being loaded and executed by the processor 101 are also stored, and these instructions can be one or more computer programs (including program codes). In the embodiment of the present specification, the processor 101 loads and executes one or more instructions stored in the memory 102 to implement the underwater image enhancement method described in the above method embodiment.

[0170] The embodiment of the present invention further provides a computer-readable storage medium, on which at least one instruction or at least one program is stored, and when the at least one instruction or the at least one program is loaded by the processor 101, the underwater image enhancement method described above is executed. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0171] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0172] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the device or element referred to must have a specific direction or be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the scope of protection of the present invention.

[0173] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. An underwater image enhancement method combined with attenuation channel correction, characterized in that: The following steps are involved: S01, obtaining an original image, performing white balance processing on the original image to perform color correction, and obtaining an image after color correction as a first enhanced image I1; S02, processing the first enhanced image I1 by a non-local means denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, and enhancing the contrast of the first enhanced image I1 by an adaptive stretching method to obtain a contrast-enhanced image recorded as a third enhanced image I3; S03, calculating and obtaining a saliency weight map, a contrast weight map and a gradient weight map about the second enhanced image I2 and performing weight normalization processing to obtain a first normalized weight map; Calculating and obtaining a saliency weight map, a noise weight map, and a gradient weight map about the third enhanced image I3 and performing weight normalization processing to obtain a second normalized weight map; S04, decomposing the second enhanced image I2 and the third enhanced image I3 into Laplacian pyramid structures, decomposing the first normalized weight map and the second normalized weight map into Gaussian pyramid structures, and performing bilateral weight fusion on the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image.

2. The underwater image enhancement method combined with attenuation channel correction according to claim 1, characterized in that: Step S01 includes the following steps: S011. Calculate the index parameters of the RGB channel of the original image according to the following formula, wherein the index parameters include information entropy H, interval length DR, and average difference ADD: ADD c =(|n2-n1|+|n3-n2|+…+|n i+1 -n i |) / L, Where c = R or G or B, H c represents the information entropy of the c channel, represents the probability of the i-th pixel level of the c channel; DR c Indicates the length of the interval where the number of pixels in the c channel is greater than or equal to μ, n i Indicates the number of pixels at the i-th pixel level, min(n i ≥μ) and max(n i ≥μ) respectively means when n i ≥μ, the minimum and maximum values ​​of i, and L represents the total number of pixel levels; ADD c represents the average difference of the c channel; S012, assigning weights to the index parameters, and constructing compensation parameters cp for each channel in the original image: c.p. c =d H H c +d DR DR c +d ADD (1 / ADD c ) Among them, cp c represents the compensation parameter of the c channel, δ H , δ DR and δ ADD They represent the weight coefficients of information entropy H, interval length DR and average difference ADD respectively; S013, sort the RGB channels according to the value of the compensation parameter cp, and record them in descending order as follows: the first channel C first , Second channel C second and the third channel C third , according to the compensation parameters cp of each channel c Assign values ​​to each channel, and construct compensation factors and compensation formulas based on the sorting and assignment results: Where cf1 and cf2 represent the first compensation factor and the second compensation factor respectively, C′ second and C′ third They represent the second channel and the third channel after compensation respectively; S014. Process the compensated image using a gray world algorithm to obtain the first enhanced image I1 after color correction.

3. The underwater image enhancement method combined with attenuation channel correction according to claim 1, characterized in that: In step S02, the first enhanced image I1 is processed by a non-local means denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, which specifically includes the following steps: S021 a. Read the first enhanced image I1, and define the length of a local block as p and the width as q; S022a, initializing an image of the same size as the first enhanced image I1 as a blank image; S023a. Calculate the similarity weight in the local block according to the following formula: Wherein, w(i,j,m,n) represents the similarity weight between the target pixel (i,j) and the similar pixel (m,n), I(i,j) represents the pixel value of the target pixel (i,j) in the local block, I(m,n) represents the pixel value of the similar pixel (m,n) in the local block, and h is the weight parameter that controls the influence of the similarity weight. S024a, calculate the weighted average of similar pixels in the local block according to the similarity weight, and obtain the denoising result FNLM(i,j): S025a, reading the three channels R, G, and B in parallel, and calculating the denoising result FNLM(i, j) on each channel, storing the denoising result FNLM(i, j) of each channel in the blank image, and obtaining the second enhanced image I2.

4. The underwater image enhancement method combined with attenuation channel correction according to claim 3, characterized in that: In step S02, the contrast of the first enhanced image I1 is enhanced by an adaptive stretching method to obtain an image after contrast enhancement, which is recorded as a third enhanced image I3. The steps specifically include the following steps: S021 b. Read the RGB channel histogram of the first enhanced image I1, and calculate the exposure value and exposure threshold of the obtained RGB channel histogram: ET=(1-EX)(MAX-MIN)+MIN, Wherein, EX represents the exposure value, ET represents the exposure threshold, and ET is used as the segmentation point to segment the RGB channel histogram into two sub-histograms: a first sub-histogram and a second sub-histogram. The distribution range of the first sub-histogram is in the low-exposure area of ​​[MIN, ET], and the distribution range of the second sub-histogram is in the over-exposure area of ​​[ET, MAX]. i represents the number of pixels at the i-th pixel level, and μ represents the pixel number threshold; S022b, using the median of the RGB channel histogram as the clipping threshold to clip the first sub-histogram and the second sub-histogram, clipping the part where the total number of pixels is greater than the clipping threshold, leaving the part where the total number of pixels is less than or equal to the clipping threshold unchanged, redistributing the total number of pixels after clipping to the entire interval of the sub-histogram, retaining the image information, and the calculation formula of the clipping process is as follows: Among them, Hist(i) represents the cropped subhistogram, Q represents the total number of cropped pixels, M represents the median of the subhistogram, and l represents the interval length of the subhistogram; S023b. According to the interval length ratio of the sub-histogram before and after stretching, a stretching factor is constructed, and the sub-histogram after stretching is obtained according to the rule of proportional stretching: Among them, θ low and θ high Respectively represent the stretching factors of the first subhistogram and the second subhistogram, Hist′ low (i) and Hist′ high (i) represent the stretched sub-histograms, and the corresponding stretching ranges are [0, ET] and [ET, L] respectively; S024b, combining the two sub-histograms after the stretching process to obtain an image after contrast enhancement, which is the third enhanced image I3.

5. The underwater image enhancement method combined with attenuation channel correction according to claim 4, characterized in that: Step S03 includes the following steps: S031. Calculate the significance weights of the second enhanced image I2 and the third enhanced image I3 according to the following formula: W S,k (i,j)=(L k (i,j)-ML k ) 2 +(A k (i,j)-MA k ) 2 +(B k (i,j)-MB k ) 2 , Among them, (i, j) represents the target position, W S,k represents the saliency weight of the kth input image, L k , A k and B k They represent the brightness channel, red and green channel, and yellow and blue channel in the CIELAB color space, respectively. k 、MA k and MB k They are the average values ​​corresponding to the brightness channel, red and green channels, and yellow and blue channels respectively; The gradient weights of the second enhanced image I2 and the third enhanced image I3 are calculated according to the following formula: Among them, (i, j) represents the target position, W G,k Represents the gradient weight of the image, G x and G y Represent the gradient of the image in the horizontal and vertical directions respectively; The noise weight of the third enhanced image I3 is calculated according to the following formula: Among them, (i, j) represents the target position, W N is the noise weight, W N (i, j) represents the local noise level at position (i, j), W H and W W hW are the height and width of the local window respectively. H represents the size of the half window, I(is,jt) represents the pixel value of the pixel (is,jt) in the local window, μ ij represents the mean of the local window; S032. Normalize each enhancement map by dividing the sum of the weights of each enhancement map by the sum of the weights of all enhancement maps. The calculation formula is as follows: Among them, W nor,k represents the normalized weight map of the kth input image, W k represents the weighted cumulative sum of the k-th input image, represents the regularization term.

6. The underwater image enhancement method combined with attenuation channel correction according to claim 5, characterized in that: In step S04, bilateral weight fusion is performed on the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image, specifically: Among them, I enhanced (x,y) represents the enhanced image, represents the qth layer weight map of the kth input image after Gaussian pyramid decomposition, represents the qth layer image of the kth input image after Laplacian pyramid decomposition, and N represents the number of decomposition layers.

7. The underwater image enhancement method combined with attenuation channel correction according to claim 6, characterized in that: The number of decomposition levels N=3.

8. An underwater image enhancement system combined with attenuation channel correction, characterized in that: include: A color correction module is used to obtain an original image, perform white balance processing on the original image to perform color correction, and obtain an image after color correction as a first enhanced image I1; an enhancement module, which is used to process the first enhanced image I1 by a non-local mean denoising algorithm to obtain a denoised image recorded as a second enhanced image I2, and to enhance the contrast of the first enhanced image I1 by an adaptive stretching method to obtain a contrast-enhanced image recorded as a third enhanced image I3; A normalization processing module, which is used to calculate a saliency weight map, a contrast weight map and a gradient weight map about the second enhanced image I2 and perform weight normalization processing to obtain a first normalized weight map; Calculating and obtaining a saliency weight map, a noise weight map, and a gradient weight map about the third enhanced image I3 and performing weight normalization processing to obtain a second normalized weight map; A fusion module is used to decompose the second enhanced image I2 and the third enhanced image I3 into Laplacian pyramid structures, decompose the first normalized weight map and the second normalized weight map into Gaussian pyramid structures, and perform bilateral weight fusion of the decomposed Laplacian pyramid structure and the Gaussian pyramid structure to obtain an enhanced image.

9. A computer device comprising a processor and a memory connected in a signal connection, characterized in that: The memory stores at least one instruction or at least one program, and when the at least one instruction or the at least one program is loaded by the processor, the underwater image enhancement method combined with attenuation channel correction as described in any one of claims 1-7 is executed.

10. A computer-readable storage medium having at least one instruction or at least one program stored thereon, characterized in that: When the at least one instruction or the at least one program is loaded by the processor, the underwater image enhancement method combined with attenuation channel correction as described in any one of claims 1 to 7 is executed.

Images