Underwater image restoration method and system based on fusion model and color balance
By fusing models and color balancing methods, and utilizing background light values and dark channel priors, the problem of insufficient adaptability in underwater image color compensation is solved, achieving efficient enhancement and color fidelity restoration of underwater images.
Patent Information
- Application Number
- CN202411179040.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-22
AI Technical Summary
Existing underwater image enhancement methods suffer from insufficient color compensation adaptability and inadequate parameter estimation stability, making it difficult to effectively restore the details and color authenticity of underwater images.
We employ a fusion model and color balance-based approach, transforming the underwater image restoration problem into a foggy image restoration problem by leveraging color constancy theory and dark channel priors. We utilize background light values and color balance maps for adaptive compensation and construct a fusion model for image enhancement.
It achieves adaptive color compensation, improves image contrast and color fidelity, reduces restoration complexity, and preserves grayscale transition information and natural color transitions.
Smart Images

Figure CN119006341A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an underwater image restoration method and system based on a fusion model and color balance, and belongs to the technical field of underwater image processing. Background Art
[0002] In underwater detection tasks, providing robots with the location and environmental information of underwater targets can provide an important basis for robot decision-making. Compared with sonar images, visual signals have obvious advantages: rich details and not easily disturbed by reflection noise; the propagation speed of light is faster than that of sound, and the real-time detection is better; human eye recognition is more intuitive and easy to understand. Therefore, as an indispensable preprocessing link for many underwater detection tasks, the effectiveness of underwater image information recovery methods has become an important factor affecting the quality of task completion. However, visual images obtained in underwater environments usually suffer from degradation phenomena such as blurred details, excessive noise, and color distortion due to the absorption attenuation of light when propagating in water, the scattering attenuation caused by the presence of substances in water, and the uneven brightness of the scene caused by the introduction of artificial light sources. In response to these problems, current underwater image enhancement methods are mainly solved from the perspectives of physical model restoration, image information enhancement, data-driven learning or generation.
[0003] The method based on physical model restoration starts with analyzing the causes of degradation, establishing a mathematical model to simulate the degradation process, and estimating the parameters in the model, so as to invert the clear image before degradation. Existing studies have proposed a variety of prior information to more accurately estimate parameters or establish imaging models with stronger scene adaptability in order to obtain better restoration effects. However, there are still problems such as low reliability of prior information and insufficient adaptability of color compensation methods. The method of parameter estimation by building a deep learning network also has the problem of insufficient stability of estimation results.
[0004] Therefore, in order to better conform to the underwater imaging degradation process, adaptively perform color compensation, and be simple and effective when restoring underwater images, there is an urgent need for an underwater image restoration method and system based on a fusion model and color balance. Summary of the invention
[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an underwater image restoration method and system based on a fusion model and color balance. An underwater image degradation model is proposed. By integrating color constancy theory and dark channel prior, the underwater image restoration problem is transformed into a restoration problem of foggy images with color deviation, which provides a new idea for realizing underwater image enhancement.
[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions: In a first aspect, the present invention provides an underwater image restoration method based on a fusion model and color balance, which is characterized by comprising: Obtaining degraded original images underwater; Decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; The dark channel of the color balance map is estimated, and the obtained background light value, color balance map and dark channel are input into a pre-built fusion model to obtain a restored clear image output by the fusion model.
[0007] Further, pre-built fusion models include:
[0008] In the formula, is the restored clear image, (x, y) corresponds to the pixel coordinates, is a color balance map, represents the dark channel of the color balance map, B is the background light intensity.
[0009] Furthermore, the iterative step includes: Estimate the background light value of each channel of the three-channel image; According to the estimated background light value, the current color cast type is defined, and the background light mean value calculation factor, mean value tolerance and adjustment step size corresponding to the current iteration are determined from the defined color cast type; Determine the background light mean according to the mean calculation factor, and obtain an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted according to the background light mean and the mean tolerance; Determine whether the background light values of each channel converge to an unbiased interval; if not, perform a grayscale linear conversion according to the adjustment step size to obtain an estimated color balance map; The above process is repeated until the background light values of the three channels converge to the unbiased interval, and the final color balance image after adaptive color compensation of each channel is obtained.
[0010] Furthermore, the background light value of each channel of the three-channel image is estimated, including: Apply Gaussian low-pass filtering to each channel of the obtained three-channel image to obtain a smoothed image; Apply the Sobel operator on the smoothed image to calculate the gradient map; After binarizing the calculated gradient map, the flat area where the background light is located is separated; In the separated flat area, the gray value difference of the three channels in the flat area is analyzed to estimate the background light value.
[0011] Furthermore, according to the estimated background light value, the current color cast type is defined, and the background light mean value calculation factor, mean value tolerance and adjustment step size corresponding to the current iteration are determined from the defined color cast type, including: According to the background light value, calculating the range of background light values between channels; The color cast type of the image is determined according to the range, wherein: When the range is less than the threshold, it is a non-color cast type, and the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.1, and the adjustment step is 0.02; When the range is not less than the threshold and the background light value of one channel is greater than the current mean, it is a single-channel color cast type, and the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.2, and the adjustment step is 0.04; When the range is not less than the threshold and the background light values of the two channels are greater than the current mean, it is a dual-channel color cast type, and the maximum and minimum values of the background light estimation value are used as mean calculation factors, the mean tolerance is 0.2, and the adjustment step is 0.04.
[0012] Further, the background light mean is determined according to the mean calculation factor, and according to the background light mean and the mean tolerance, an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted is obtained, including: Calculate the background light mean value according to the obtained background light mean value calculation factor and the defined color cast type; According to the background light mean and mean tolerance, an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted is obtained.
[0013] Further, a grayscale linear conversion is performed according to the adjustment step size to obtain an estimated color balance map, including: The grayscale linear conversion formula is as follows: ; in, is the adjustment step size, where: When the background light value estimated by the color cast channel is higher than the background light mean, the gray value is reduced by subtraction to correct the excessive gray value of the channel pixel; if there is a gray value lower than 0, the gray value [0, 2] is converted to 1 / 2 of the original value to avoid underflow. The formula is as follows: ; When the background light value estimated by the color cast channel is lower than the background light mean value, the gray value is increased by adding to correct the low gray value of the channel pixel; if the gray value is higher than 1, the gray value is increased. , 1] is converted to [ , 1], to avoid overflow, the formula is as follows: .
[0014] In a second aspect, the present invention provides an underwater image restoration system based on a fusion model and color balance, characterized in that it includes: Acquisition module: used to obtain the original image with degraded quality underwater; Processing module: used for decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; Restoration module: used to estimate the dark channel of the color balance map, input the obtained background light value, color balance map and dark channel into the pre-built fusion model, and obtain the restored clear image output by the fusion model.
[0015] In a third aspect, the present invention provides a computer device, including a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.
[0016] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the method described in the first aspect are implemented.
[0017] Compared with the prior art, the present invention has the following beneficial effects: The underwater image restoration method based on fusion model and color balance provided by the present invention proposes an underwater image degradation model. By integrating color constancy theory and dark channel prior, the underwater image restoration problem is transformed into the restoration problem of foggy images with color deviation, which provides a new idea for underwater image enhancement. The constructed model does not need to estimate absorption attenuation and depth of field. Through the background light and dark channel information of the image itself, it can effectively enhance the image contrast and reduce the complexity of restoration. The underwater image restoration method based on fusion model and color balance provided by the present invention adopts a gradient-based background light value estimation method, which avoids the influence of the local highlight part in the foreground on the background light value estimation deviation, and obtains color cast information through the difference of background light values in different channels. It provides a basis for color cast compensation while estimating parameters; The underwater image restoration method based on fusion model and color balance provided by the present invention adjusts the grayscale distribution of each channel based on the difference in background light value to achieve adaptive color compensation. The grayscale conversion process does not require parameter adjustment, retains grayscale transition information, and has no information overflow. The restored result has high color fidelity, natural color transition, and further improved contrast. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is a schematic diagram of the overall process of the underwater image restoration method based on the fusion model and color balance provided in Example 1.
[0019] Figure 2 A comparison chart of the restoration effects of the underwater image restoration method based on the fusion model and color balance provided in Example 1 and other algorithms on blurred images of real underwater scenes; Figure 3 A comparison chart of the restoration effects of the underwater image restoration method based on the fusion model and color balance provided in Example 1 and other methods on the color cast images of real underwater scenes; Figure 4 A comparison chart of the restoration effects of the underwater image restoration method based on the fusion model and color balance provided in Example 1 and other methods on underwater complex scene images; Figure 5 It is a flowchart of the underwater image restoration method based on fusion model and color balance provided in Example 1. DETAILED DESCRIPTION
[0020] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.
[0021] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. Embodiment 1
[0022] Figure 5 This is a flow chart of an underwater image restoration method based on a fusion model and color balance in Embodiment 1 of the present invention. The underwater image restoration method based on a fusion model and color balance provided in this embodiment can be applied to a terminal and can be executed by an underwater image restoration system based on a fusion model and color balance. The system can be implemented by software and / or hardware. The system can be integrated in a terminal, for example: any smart phone, tablet computer or computer device with communication function. Figure 5 The method of this implementation specifically includes the following steps: Obtaining degraded original images underwater; Decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; The dark channel of the color balance map is estimated, and the obtained background light value, color balance map and dark channel are input into a pre-built fusion model to obtain a restored clear image output by the fusion model.
[0023] Further, pre-built fusion models include:
[0024] In the formula, is the restored clear image, (x, y) corresponds to the pixel coordinates, is a color balance map, represents the dark channel of the color balance map, B is the background light intensity.
[0025] Furthermore, the iterative step includes: Estimate the background light value of each channel of the three-channel image; According to the estimated background light value, the current color cast type is defined, and the background light mean value calculation factor, mean value tolerance and adjustment step size corresponding to the current iteration are determined from the defined color cast type; Determine the background light mean according to the mean calculation factor, and obtain an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted according to the background light mean and the mean tolerance; Determine whether the background light values of each channel converge to an unbiased interval; if not, perform a grayscale linear conversion according to the adjustment step size to obtain an estimated color balance map; The above process is repeated until the background light values of the three channels converge to the unbiased interval, and the final color balance image after adaptive color compensation of each channel is obtained.
[0026] Furthermore, the background light value of each channel of the three-channel image is estimated, including: Apply Gaussian low-pass filtering to each channel of the obtained three-channel image to obtain a smoothed image; Apply the Sobel operator on the smoothed image to calculate the gradient map; After binarizing the calculated gradient map, the flat area where the background light is located is separated; In the separated flat area, the gray value difference of the three channels in the flat area is analyzed to estimate the background light value.
[0027] Furthermore, according to the estimated background light value, the current color cast type is defined, and the background light mean value calculation factor, mean value tolerance and adjustment step size corresponding to the current iteration are determined from the defined color cast type, including: According to the background light value, calculating the range of background light values between channels; The color cast type of the image is determined according to the range, wherein: When the range is less than the threshold, it is a non-color cast type, and the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.1, and the adjustment step is 0.02; When the range is not less than the threshold and the background light value of one channel is greater than the current mean, it is a single-channel color cast type, and the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.2, and the adjustment step is 0.04; When the range is not less than the threshold and the background light values of the two channels are greater than the current mean, it is a dual-channel color cast type, and the maximum and minimum values of the background light estimation value are used as mean calculation factors, the mean tolerance is 0.2, and the adjustment step is 0.04.
[0028] Further, the background light mean is determined according to the mean calculation factor, and according to the background light mean and the mean tolerance, an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted is obtained, including: Calculate the background light mean value according to the obtained background light mean value calculation factor and the defined color cast type; According to the background light mean and mean tolerance, an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted is obtained.
[0029] Further, a grayscale linear conversion is performed according to the adjustment step size to obtain an estimated color balance map, including: The grayscale linear conversion formula is as follows: ; in, is the adjustment step size, where: When the background light value estimated by the color cast channel is higher than the background light mean, the gray value is reduced by subtraction to correct the excessively high gray value of the channel pixel; if the gray value is lower than 0, the gray value [0, 2 ] is converted to 1 / 2 of the original value to avoid underflow. The formula is as follows: ; When the background light value estimated by the color cast channel is lower than the background light mean value, the gray value is increased by adding to correct the low gray value of the channel pixel; if the gray value is higher than 1, the gray value is increased. , 1] is converted to [ , 1], to avoid overflow, the formula is as follows: .
[0030] like Figure 1 As shown, the underwater image restoration method based on the fusion model and color balance provided in this embodiment specifically includes: Step 01: Obtain an underwater degraded original image, and substitute the underwater degraded original image into a fusion model; Step 02: Decompose the original underwater degraded image into three-channel images of R, G, and B; obtain a smoothed image through Gaussian low-pass filtering, obtain a gradient image using the Sobel operator, separate the flat area where the background light is located after binarization, and estimate the grayscale value with differences between the three channels as the background light value; Step 03: According to the estimated background light value, determine the background light mean calculation factor, mean tolerance and adjustment step size corresponding to this round from the defined color cast type; Step 04: Calculate the background light mean, and obtain an unbiased interval for determining whether the grayscale value of each channel needs to be adjusted based on the mean tolerance; Step 05: Determine whether the background light values of each channel converge to the unbiased interval; if not, perform grayscale linear conversion according to the adjustment step size to obtain an estimated color balance map, the color performance of which is partially improved; because grayscale conversion will cause spatial gradient changes, repeat the process of estimating background light and estimating color balance map until the background light values of the three channels converge to the unbiased interval, and obtain the final color balance map after color adaptive compensation of each channel; Step 06: According to the final color balance map, estimate its dark channel, and substitute it and the estimated background light value into the fusion model to obtain a restored clear image.
[0031] The method for constructing the fusion model in step 01 comprises the following steps: Step 011: The fusion model is based on the underwater imaging physical model, taking into account the scattering and absorption attenuation problems caused by light propagating in the water medium, and can be expressed as: ; Among them, (x, y) corresponds to the pixel coordinates, To capture underwater optical images, is the direct attenuation component, is the forward scattering component, is the backscattering component; Step 012: The direct attenuation component can be expressed by Lambert-Beer law as follows: ; in, is the reflected light intensity of the real underwater scene, that is, the clear image to be restored. d is the scene depth, a is the total attenuation coefficient, including the spectrally selective absorption attenuation coefficient a o and the scattering attenuation coefficient a s ; Step 013: The forward scattering component is the remaining light intensity after the small-angle scattering loss of the reflected light during propagation, so its size can be obtained by convolving the direct component and the point spread function: ; in, is a Gaussian point spread function; Step 014: The backscattered component is the light intensity of the ambient light entering the optical sensor after being scattered, which can be expressed as: ; in, B is the background light intensity; Step 015: Combining the above components, the underwater optical image light intensity is obtained as follows: ; When imaging underwater, the distance between the camera and the scene is relatively close, and the underwater imaging model can be simplified by ignoring the forward scattering component: ; Therefore, restoring the real image , the attenuation coefficient needs to be estimated a , Depth of Field d and background light B . Let the transmittance and the scattered transmittance be: ; ; Then the solution to the restoration problem is converted into solving the transmittance and background light problem: ; Step 016: According to the description in the color constancy theory, the natural received image can be expressed as the product of the ambient light intensity and the reflection component; the direct attenuation term in the underwater imaging model Similarly, it can be described as: ; in, L is the ambient light intensity. The ambient light intensity underwater can be expressed as: ; Then the solution to the restoration problem in step 015 can be further transformed into: ; After simplification, we get: ; Thus, it is no longer necessary to estimate absorption attenuation and depth of field parameters; Step 017: Underwater object images often show fog-like characteristics. Therefore, after color cast correction, the scattering transmittance of the underwater image can be calculated based on the dark channel prior theory. The formula is as follows: ; in, This is the color balance diagram after color cast correction. Represents the dark channel of the color balance map; the degraded image in step 016 model after color cast correction Replace with color balance map , and then the transmittance Substitute it and get the fusion model: ; By substituting the estimated background light value, color balance map, and dark channel map, the clear image before degradation can be inverted.
[0032] In the present application, the background light estimation method in step 02 includes the following steps: Step 021: Smooth the underwater degraded original image by Gaussian low-pass filtering, and the filter is defined as follows: ; Where σ is the standard deviation of the normal distribution; Step 022: Use the Sobel operator to obtain a gradient map of the smoothed image, as shown in the following formula: ; ; ; in, , are the horizontal and vertical gradient values of the pixel points, respectively. is the gradient value of the pixel, * is the convolution operation; Step 023: Set a threshold for the gradient map of each channel. Pixels below the gradient threshold belong to the background light area. After separating the background light area through binarization, start taking values from high grayscale values, avoid the same grayscale values in the background light areas of the three channels, take the first 0.1% high grayscale values with differences, and use them as the background light estimation value of the channel.
[0033] As an implementation manner, the step of determining the background light mean value calculation factor, mean value tolerance and adjustment step length in step 03 includes: Step 031: According to the background light value, calculate the range of the background light value between channels, the formula is as follows: ; in, Represents the estimated background light value by channel; Step 032: Determine the color cast type of the image according to the extreme difference; if the color cast is not serious, the image has less absorption attenuation, is more sensitive to grayscale adjustment, and has a low adjustment step value; if the color cast is serious, a larger adjustment step value is used to make the correction effect more obvious; there are three situations: When the range is less than the threshold, it is a color cast-free type, and the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.1, and the adjustment step is 0.02; When the range is not less than the threshold and the background light value of one channel is greater than the current mean, it is a single-channel color cast type, and the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.2, and the adjustment step is 0.04; When the range is not less than the threshold and the background light values of the two channels are greater than the current mean, it is a dual-channel color cast type, and the maximum and minimum values of the background light estimation value are used as mean calculation factors, the mean tolerance is 0.2, and the adjustment step is 0.04.
[0034] The determination of the unbiased interval in step 04 includes: Step 041: Calculate the background light mean value according to the color cast type, using the following formula: ; in, n Indicates the number of calculated factors; Step 042: According to the background light mean and mean tolerance, determine the unbiased interval for judging whether the grayscale value of each channel needs to be adjusted: ; in, is the mean tolerance described in step 032.
[0035] Preferably, the process of grayscale linear conversion and estimating the color balance map in step 05 includes: Step 051: If the estimated background light value of a channel does not converge to the unbiased interval, grayscale conversion is performed, and the formula is as follows: ; in, is the adjustment step size described in step 032; there are two cases here: When the background light value estimated by the color cast channel is higher than the background light mean value, the gray value is reduced by subtraction to correct the excessively high gray value of the channel pixel; if the gray value is lower than 0, the gray value [0, 2 ] is converted to 1 / 2 of the original value to avoid underflow. The formula is as follows: ; When the background light value estimated by the color cast channel is lower than the background light mean value, the gray value is increased by adding to correct the low gray value of the channel pixel; if the gray value is higher than 1, the gray value is increased. , 1] is converted to [ , 1], to avoid overflow, the formula is simplified as follows: Step 052: Due to the complexity of underwater imaging scenes, the degree of color cast of each channel of the underwater degraded image is usually inconsistent. Therefore, the channels corresponding to the grayscale conversion of different color cast types and the number of adjustment rounds are also different. The process of estimating the color balance map can adaptively perform color compensation according to the severity of color cast of each channel of the image. The iterative process includes: Step 0521: Input the color balance map estimated in the previous round, decompose it into three channels of red, green and blue, and estimate the background light value of each channel respectively; Step 0522: Determine the color cast type, and obtain the corresponding background light mean calculation factor, mean tolerance, and adjustment step size; Step 0523: For channels whose background light values are not within the unbiased interval, grayscale conversion is performed; Step 0524: After the color cast channel adjustment is completed, the color balance map estimated in this round is obtained; Step 0525: Repeat the above process until all three channels are adjusted to converge to the unbiased interval, that is, they are judged as the type without color cast; Step 0526: Merge the compensated three channels to obtain the final color balance map as output.
[0036] Preferably, the content of estimating the dark channel in step 06 includes: The dark channel prior is proposed for the problem of defogging natural images: in the vast majority of fog-free images, at least one of the three channels in the non-sky area has a very low pixel value, even close to 0, while foggy images do not have this characteristic. This prior is applied to the underwater environment, and the minimum grayscale value of the three-channel pixel is obtained for the color balance map. The formula is as follows: .
[0037] like Figure 2 As shown, Figure 2(a) represents an underwater degraded image, (b) represents the result image processed by ULAP (A Rapid Scene Depth Estimation Model Based on Underwater Light Attenuation Prior for Underwater Image Restoration), (c) represents the result image processed by IBLA (Underwater Image Restoration Based on Image Blurriness and Light Absorption), (d) represents the result image processed by UWCC (Underwater Image Enhancement based on Deep Learning and Image Formation Model), (e) represents the result image processed by UT (U-Shape Transformer for Underwater Image Enhancement), and (f) represents the result image processed by the present invention. It can be seen that the present invention provides a comparison of the restoration effect of underwater blurred images with other methods. It can be seen that the five methods can achieve a certain degree of enhancement. The original image in the first row has a slight blur problem. (b) is the result of the ULAP method. It can be seen that the restored image has improved a certain contrast and the blur has been improved to a certain extent, but the red compensation is too much, resulting in a new color cast. (c) is the result of the IBLA method. The blur in the original image has been removed and the color of the scene has been restored, but the overall tone is dark and the details of the low-grayscale part are not clear. (d) is the result of the UWCC method. The restoration effect is good, but the color restoration is not realistic enough. (e) is the result of the UT method. The details are still blurred and the overall tone is reddish. (f) is the result of the method of the present invention. The contrast of the restored image is greatly improved and the color fidelity is good. The blur problem of the original image in the second row is more serious. (b) is the result of the ULAP method. The details are not enhanced and the red compensation is still too much. (c) is the result of the IBLA method. The brightness of the result image is too dark and the details are almost invisible. (d) is the result of the UWCC method. The contrast is enhanced to a certain extent, but the color is distorted. (e) is the result of the UT method. The contrast is not improved and color cast occurs. (f) is the result of the method of the present invention. The visual enhancement is obvious, the color transition is natural, and the contrast is significantly improved.
[0038] like Figure 3 As shown, Figure 3(a) represents an underwater degraded image, and (b)-(f) correspond to the results of ULAP, IBLA, UWCC, UT and the present invention, respectively. It can be seen that the present invention provides a comparison chart of the restoration effect of underwater color cast images with other methods. The original image in the first row has slight blur and color cast problems. It can be seen that the five methods can achieve a certain degree of enhancement. (b) is the result of ULAP method processing, the contrast is partially improved, but the color cast still exists. (c) is the result of IBLA method processing, the color cast correction is better, but the details are blurred. (d) is the result of UWCC method processing, the restoration effect is better. (e) is the result of UT method processing, the color cast problem is solved, but the contrast improvement is not obvious, and the overall tone is dark. (f) is the result of the present invention method processing, it can be seen that the contrast is effectively improved, the color transition information is retained, and the color fidelity is good. The original image in the second row has a serious color cast problem. In the results of ULAP method corresponding to (b), IBLA method corresponding to (c), and UT method corresponding to (e), the color cast problem is not improved. (d) The contrast of the result processed by the corresponding UWCC method is improved to a certain extent, but the color compensation is incorrect, and a yellowish tint appears. (f) is the result processed by the method of the present invention, the color cast problem is significantly improved, the color is more natural, and the image is clearer.
[0039] like Figure 4 As shown, Figure 4 (a) represents an underwater degraded image, and (b)-(f) correspond to the results of ULAP, IBLA, UWCC, UT and the present invention, respectively. It can be seen that the present invention provides a comparison chart of the restoration effects of other methods for underwater complex scene images. Complex scenes may have factors that affect imaging quality, such as light attenuation, scattering effects, color shift, medium inhomogeneity affecting the propagation path of light, and dynamic environmental changes, thereby affecting the enhancement effect. (b) is the result of the ULAP method, and the contrast is improved, but overcompensation introduces red artifacts that are not in the original image. (c) is the result of the IBLA method, the low grayscale area is not bright enough, and the enhancement effect is not ideal. (d) is the result of the UWCC method, and the contrast is significantly improved, but the color compensation is not accurate enough. (e) is the result of the UT method, and the problem of blurred details still exists and a new color cast is generated. (f) is the result of the present invention method, and it can be seen that the color fidelity is high and the contrast is significantly improved.
[0040] Therefore, compared with other methods, the method of the present invention can better reduce blur, improve contrast, and maintain detail information while correcting color cast, making the image clearer. The performance in the above multiple scenarios shows that the method of the present invention has stronger adaptability. Embodiment 2
[0041] An underwater image restoration system based on fusion model and color balance, comprising: Acquisition module: used to obtain the original image with degraded quality underwater; Processing module: used for decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; Restoration module: used to estimate the dark channel of the color balance map, input the obtained background light value, color balance map and dark channel into the pre-built fusion model, and obtain the restored clear image output by the fusion model.
[0042] The underwater image restoration system based on fusion model and color balance provided in the embodiment of the present invention can execute the underwater image restoration method based on fusion model and color balance provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. Embodiment 3
[0043] An embodiment of the present invention further provides a computer device, including a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to perform the steps of the following method: Obtaining degraded original images underwater; Decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; The dark channel of the color balance map is estimated, and the obtained background light value, color balance map and dark channel are input into a pre-built fusion model to obtain a restored clear image output by the fusion model.
[0044] For the details of each step in this embodiment, please refer to the first embodiment, which will not be described in detail here. Since this embodiment and the first embodiment adopt the same technical concept, it also has the technical effects described in the first embodiment. Embodiment 4
[0045] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the following method are implemented: Obtaining degraded original images underwater; Decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; The dark channel of the color balance map is estimated, and the obtained background light value, color balance map and dark channel are input into a pre-built fusion model to obtain a restored clear image output by the fusion model.
[0046] For the details of each step in this embodiment, please refer to the first embodiment, which will not be described in detail here. Since this embodiment and the first embodiment adopt the same technical concept, it also has the technical effects described in the first embodiment.
[0047] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0048] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.
[0049] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0050] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0051] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. An underwater image restoration method based on fusion model and color balance, characterized in that: include: Obtaining degraded original images underwater; Decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; The dark channel of the color balance map is estimated, and the obtained background light value, color balance map and dark channel are input into a pre-built fusion model to obtain a restored clear image output by the fusion model.
2. The underwater image restoration method based on fusion model and color balance according to claim 1, characterized in that: Pre-built fusion models include: ; In the formula, is the restored clear image, (x, y) corresponds to the pixel coordinates, is a color balance map, represents the dark channel of the color balance map, B is the background light intensity.
3. The underwater image restoration method based on fusion model and color balance according to claim 2, characterized in that: The iterative steps include: Estimate the background light value of each channel of the three-channel image; According to the estimated background light value, the current color cast type is defined, and the background light mean value calculation factor, mean value tolerance and adjustment step size corresponding to the current iteration are determined from the defined color cast type; Determine the background light mean according to the mean calculation factor, and obtain an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted according to the background light mean and the mean tolerance; Determine whether the background light values of each channel converge to an unbiased interval; if not, perform a grayscale linear conversion according to the adjustment step size to obtain an estimated color balance map; The above process is repeated until the background light values of the three channels converge to the unbiased interval, and the final color balance image after adaptive color compensation of each channel is obtained.
4. The underwater image restoration method based on fusion model and color balance according to claim 3 is characterized in that: Estimate the background light value of each channel of the three-channel image, including: Apply Gaussian low-pass filtering to each channel of the obtained three-channel image to obtain a smoothed image; Apply the Sobel operator on the smoothed image to calculate the gradient map; After binarizing the calculated gradient map, the flat area where the background light is located is separated; In the separated flat area, the gray value difference of the three channels in the flat area is analyzed to estimate the background light value.
5. The underwater image restoration method based on fusion model and color balance according to claim 4 is characterized in that: According to the estimated background light value, the current color cast type is defined, and the background light mean value calculation factor, mean value tolerance and adjustment step size corresponding to this round of iteration are determined from the defined color cast type, including: According to the background light value, calculating the range of background light values between channels; The color cast type of the image is determined according to the range, wherein: When the range is less than the threshold, it is a color cast-free type. At this time, the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.1, and the adjustment step is 0.02; When the range is not less than the threshold and the background light value of one channel is greater than the current mean, it is a single-channel color cast type. At this time, the background light values of all channels are used as background light mean calculation factors, the mean tolerance is 0.2, and the adjustment step size is 0.04; When the range is not less than the threshold and the background light values of the two channels are greater than the current mean, it is a dual-channel color cast type. At this time, the maximum and minimum values of the background light estimation value are used as mean calculation factors, the mean tolerance is 0.2, and the adjustment step is 0.
04.
6. The underwater image restoration method based on fusion model and color balance according to claim 1, characterized in that: Determine the background light mean value according to the mean calculation factor, and obtain an unbiased interval for judging whether the grayscale value of each channel needs to be adjusted according to the background light mean value and the mean tolerance, including: Calculate the background light mean value according to the obtained background light mean value calculation factor and the defined color cast type; According to the background light mean and mean tolerance, an unbiased interval for determining whether the grayscale value of each channel needs to be adjusted is obtained.
7. The underwater image restoration method based on fusion model and color balance according to claim 1, characterized in that: Performing a grayscale linear transformation according to the adjustment step size to obtain an estimated color balance map, including: The grayscale linear conversion formula is as follows: ; in, is the gray value after conversion, is the grayscale value before conversion, is the adjustment step size, where: When the background light value estimated by the color cast channel is higher than the background light mean, the gray value is reduced by subtraction to correct the excessively high gray value of the channel pixel; if the gray value is lower than 0, the gray value [0, 2 ] is converted to 1 / 2 of the original value to avoid underflow. The formula is as follows: ; When the background light value estimated by the color cast channel is lower than the background light mean value, the gray value is increased by adding to correct the low gray value of the channel pixel; if the gray value is higher than 1, the gray value is increased. , 1] is converted to [ , 1], to avoid overflow, the formula is as follows: 。 8. An underwater image restoration system based on fusion model and color balance, characterized in that: include: Acquisition module: used to obtain the original image with degraded quality underwater; Processing module: used for decomposing the acquired underwater degraded original image into a three-channel image, performing an iterative step, and obtaining a background light value of the three-channel image and a color balance image after adaptive color compensation of each channel; Restoration module: used to estimate the dark channel of the color balance map, input the obtained background light value, color balance map and dark channel into the pre-built fusion model, and obtain the restored clear image output by the fusion model.
9. A computer device, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.
