Underwater image enhancement method based on water diffusion physical model

Abstract

The application provides an underwater image enhancement method based on a water diffusion physical model, and relates to the technical field of underwater image enhancement. The method comprises the following steps: S101, inputting a local block of an underwater image into a convolutional neural network to predict an attenuation coefficient matrix and a diffusion matrix of water; S102, constructing a water diffusion control equation based on the attenuation coefficient matrix and the diffusion matrix of water; S103, performing multi-scale diffusion on the underwater image in a frequency domain based on the water diffusion control equation to obtain a reconstructed image; S104, restoring a color channel of the reconstructed image through a spectral transmittance model to obtain an enhanced image; and S105, outputting the enhanced image. The method estimates the attenuation and scattering coefficients of light underwater, realizes image enhancement that is more in line with the physical diffusion characteristics of light underwater, effectively avoids color distortion and image blurring, and improves the physical interpretability and reliability of the enhanced result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater image enhancement, in particular to an underwater image enhancement method based on a water diffusion physical model. BACKGROUND

[0002] In many fields such as marine scientific research, underwater engineering detection, and underwater biological observation, underwater images need to be collected and further processed. For example, in marine resource exploration, underwater images of a specified area need to be collected to identify mineral distribution and biological communities; in the industrial field, underwater images of pipelines or ship hulls are collected by underwater robots for defect detection; in the archaeological field, underwater images are relied on to record the appearance of sunken ships or sites. These applications in different fields all require underwater images to have high clarity and authenticity. However, due to the complex optical environment underwater, underwater images often suffer from serious degradation, which directly affects the reliability and accuracy of image recognition and processing. Improving the quality of underwater images not only improves the visual perception effect, but also improves the accuracy of target detection, segmentation, and recognition. Therefore, enhancing underwater images has important scientific significance and practical value.

[0003] Current underwater image enhancement methods mainly fall into two categories: image processing-based methods and deep learning-based methods. Traditional image processing methods such as histogram equalization and color correction highly depend on artificial priori and have poor generalization ability, which can easily lead to over-enhancement or local distortion. While deep learning methods perform well in many scenarios, they rely on a large amount of high-quality data for training and lack physical interpretation, which can easily introduce artifacts or color deviations. The main problem with these methods is that they fail to fully incorporate the physical process of underwater imaging, especially the propagation characteristics of light in water. Underwater image degradation is mainly caused by energy attenuation and color distortion due to light absorption and scattering by water, while scattering effects can be described by a water diffusion physical model. Ignoring this physical property and only enhancing from the image level often fails to accurately restore the true color and structural details of the scene. SUMMARY

[0004] The present application aims to provide an underwater image enhancement method based on a water diffusion physical model, which estimates the attenuation and scattering coefficients of light underwater to achieve image enhancement that is more consistent with physical laws.

[0005] To achieve the above-mentioned application purposes, the present application provides an underwater image enhancement method based on a water diffusion physical model, which comprises:

[0006] S101, inputting an underwater image into a convolutional neural network to predict an attenuation coefficient matrix and a diffusion coefficient matrix of the water body;

[0007] S102, constructing a water diffusion control equation based on the attenuation coefficient matrix and the diffusion coefficient matrix of the water body;

[0008] S103, performing multi-scale diffusion on the underwater image in the frequency domain based on the water diffusion control equation to obtain a reconstructed image;

[0009] S104, restoring the color channel of the reconstructed image through a spectral transmittance model to obtain an enhanced image;

[0010] S105, outputting the enhanced image.

[0011] Further, in step S101, the convolutional neural network includes a first branch and a second branch, the first branch predicts the attenuation coefficient matrix of the water body through a Sigmoid activation function, and the second branch predicts the diffusion coefficient matrix of the water body through a Softplus activation function.

[0012] Further, before step S101, the following operation is performed: generating paired data for training the convolutional neural network by simulating the underwater image degradation process based on a water diffusion physical model through a diffusion model, the paired data including a clear image and a degraded image.

[0013] Further, in step S102, the water diffusion control equation is further constructed in combination with the backscattered light intensity, and the water diffusion control equation is solved by a semi-implicit finite difference method.

[0014] Further, step S103 specifically includes the following operations:

[0015] S201, performing multi-level wavelet transform on the underwater image to obtain subbands of different scales, the subbands of different scales including low-frequency subbands and high-frequency subbands;

[0016] S202, adopting uniform diffusion on the low-frequency subbands;

[0017] S203, adopting edge-aware diffusion on the high-frequency subbands;

[0018] S204, performing weighted fusion on the subbands of different scales to reconstruct the image through inverse wavelet transform.

[0019] Further, step S104 includes the following steps:

[0020] S301, calculating transmittance according to the attenuation coefficient matrix and the scene depth;

[0021] S302, estimating background light;

[0022] S303, restoring the color channel of the reconstructed image according to the color channel intensity of the reconstructed image, the background light and the transmittance to obtain an enhanced image.

[0023] Further, the background light is jointly estimated with the diffuse coefficient matrix through the dark channel prior.

[0024] Further, in step S103, the multi-scale diffusion is accelerated by the FPGA, and the specific operation includes: the diffusion coefficient is pre-stored in the BRAM, and the gradient calculation is realized by adopting the Sobel operator parallel pipeline.

[0025] Further, before the output of the enhanced image, the enhanced image is dynamically compressed according to the brightness standard deviation of different regions of the enhanced image, and the compressed enhanced image is output.

[0026] Further, before step S102, the following operations are performed:

[0027] S401, input the polarization image;

[0028] S402, calculate the polarization degree of the polarization image;

[0029] S403, correct the diffusion coefficient according to the polarization degree.

[0030] Compared with the prior art, the beneficial effects of the present application are:

[0031] The underwater image enhancement method based on the water diffusion physical model provided by the present application estimates the attenuation coefficient matrix and the diffusion coefficient matrix of the water body in the underwater image through the convolutional neural network, constructs the water diffusion control equation according to the attenuation coefficient matrix and the diffusion coefficient matrix, performs multi-scale diffusion on the underwater image in the frequency domain, finally restores each color channel of the image to obtain an enhanced image, so as to realize image enhancement more in line with the physical diffusion characteristics of light under water, effectively avoid color distortion and image blur, and improve the physical interpretability and reliability of the enhanced result. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only preferred embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0033] Figure 1 is a whole flow diagram of the underwater image enhancement method based on the water diffusion physical model provided by the embodiment of the present application.

[0034] Figure 2 is a flow diagram of the underwater image reconstruction provided by the embodiment of the present application.

[0035] Figure 3 is a flow diagram of the image color channel recovery provided by the embodiment of the present application.

[0036] Figure 4 This is a schematic diagram of the diffusion coefficient correction process provided in an embodiment of the present invention. Detailed Implementation

[0037] The principles and features of the present invention are described below with reference to the accompanying drawings. The listed embodiments are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0038] Reference Figure 1 This embodiment provides an underwater image enhancement method based on a water diffusion physical model, the method comprising:

[0039] S101. Input the underwater image into the convolutional neural network to predict the attenuation coefficient matrix and diffusion coefficient matrix of the water body.

[0040] In this embodiment, using This represents the original underwater image to be enhanced, where It represents the two-dimensional coordinates of pixels in an image, and the data type is usually RGB or grayscale matrix.

[0041] Attenuation coefficient matrix This describes the energy attenuation of light as it propagates through water. A higher value indicates faster light intensity decay, meaning the water is more turbid. It's an attenuation coefficient matrix of the same size as the input underwater image, with each pixel position... A corresponding attenuation coefficient value, for example, in the clear water area. turbid area .

[0042] Diffusion coefficient matrix Used to describe the ability of water to scatter light, measured in units of . A higher diffusion coefficient value indicates stronger scattering and a more blurred image. A diffusion coefficient matrix of the same size as the input underwater image is provided, with each pixel position... This corresponds to a diffusion coefficient value.

[0043] In a preferred embodiment, the input to the convolutional neural network is an image patch of an underwater image, which can be 32×32 pixels in size. The convolutional neural network includes a first branch and a second branch. The first branch predicts the attenuation coefficient matrix of the water body using a sigmoid activation function, which can be expressed as:

[0044]

[0045] In the above formula, This indicates the processing of underwater images through the first branch of the convolutional neural network. To process, This indicates the maximum value of the attenuation coefficient set based on the water body type.

[0046] The second branch predicts the diffusion coefficient matrix of the water body through a Softplus activation function, which can be expressed as:

[0047]

[0048] In the above formula, represents the processing of the underwater image by the first branch of the convolutional neural network, represents the maximum value of the diffusion coefficient set based on the water body type.

[0049] As another possible implementation, before step S101, the degradation process of the underwater image is simulated based on the water diffusion physical model through a diffusion model to generate paired data for training the convolutional neural network, the paired data including a clear image and a degraded image. The corresponding expression is:

[0050]

[0051] In the above formula, represents the degraded image, represents the clear image, represents the diffusion kernel, represents the attenuation-related noise. The attenuation of light in water follows the Beer-Lambert law, the stronger the attenuation, the weaker the signal, and the relatively more significant noise.

[0052] This implementation can quickly generate a large amount of high-quality paired data that conforms to the physical propagation characteristics of light underwater by simulating the degradation process of the underwater image through a diffusion model, and improve the performance of the convolutional neural network through training.

[0053] S102, based on the attenuation coefficient matrix and the diffusion matrix of the water body, construct the water diffusion control equation.

[0054] As a possible implementation, the water diffusion control equation is further constructed in step S102 in combination with the backscattering light intensity, and the corresponding expression is:

[0055]

[0056] In the above formula, represents the partial differential operator, which is used to describe the relationship between the image brightness and time t. is a gradient operator used to calculate the spatial variation in the image or physical field. represents the control of the degree of blur caused by scattering, achieving spatial adaptation. represents the control of light intensity attenuation, achieving color distortion correction. is the backscattering weight, is the backscattering light intensity of the underwater image. ​

[0057] In this embodiment, the water diffusion control equation can be solved by semi-implicit finite difference method. The corresponding expression is:

[0058]

[0059] In the above formula, denotes the image matrix of the t-th time step after discretization; denotes the time step; denotes the diffusion term implicit matrix, corresponding to ; denotes the attenuation term explicit matrix, corresponding to .

[0060] S103, performing multi-scale diffusion on the underwater image in the frequency domain based on the water diffusion control equation to obtain a reconstructed image.

[0061] As a possible implementation, referring to Figure 2 , step S103 specifically includes the following operations:

[0062] S201, performing multi-level wavelet transform on the underwater image to obtain subbands of different scales, the subbands of different scales including low-frequency subbands and high-frequency subbands.

[0063] In this embodiment, the low-frequency subband contains the smooth background and color cast of the underwater image; the high-frequency subband contains edges / texture / noise in horizontal, vertical and diagonal directions.

[0064] S202, using uniform diffusion on the low-frequency subband. The corresponding expression is:

[0065]

[0066] In the above formula, denotes the low-frequency subband.

[0067] S203, using edge-aware diffusion on the high-frequency subband. The corresponding expression is:

[0068]

[0069] In the above formula, denotes the high-frequency subband, denotes the gradient of the high-frequency subband, denotes the edge threshold parameter, denotes the divergence operator. When the gradient (strong edge region), the denominator tends to be the square of , the diffusion coefficient tends to 0, thereby preserving the edge. When the gradient (flattened or noise region), the denominator tends to 1, the diffusion coefficient is large, while smoothing the noise to denoise, avoiding edge blurring.

[0070] S204, the subbands of different scales are fused by weighting, and the image is reconstructed by inverse wavelet transform.

[0071] In this embodiment, the expression for reconstructing the image by inverse wavelet transform is:

[0072]

[0073] In the above formula, represents the reconstructed image. represents the high-frequency subband weight, which is adaptively calculated by the local signal-to-noise ratio (SNR). represents the high-frequency subband of the kth level. represents the low-frequency subband weight, which is also adaptively calculated by the local signal-to-noise ratio. represents the low-frequency subband of the kth level. K represents the number of wavelet decomposition, which can be set according to the image resolution.

[0074] For example, the calculation formulas of the high-frequency subband weight and the low-frequency subband weight are as follows:

[0075]

[0076]

[0077]

[0078]

[0079] In the above formula, represents the low-frequency background light estimation value of the kth level. represents the variance calculation function. represents the noise. When the signal-to-noise ratio of the image region is high, the high-frequency subband weight is increased to enhance the detail preservation; when the signal-to-noise ratio of the image region is low, the low-frequency subband weight is increased to suppress the noise and color deviation.

[0080] S104, the color channel of the reconstructed image is recovered by the spectral transmittance model to obtain an enhanced image.

[0081] As a possible implementation, with reference to Figure 3 , step S104 specifically includes the following operations:

[0082] S301, calculate the transmittance according to the attenuation coefficient matrix and the scene depth.

[0083] In this embodiment, the calculation formula of the transmittance is as follows:

[0084]

[0085] In the above formula, denotes transmittance of the c channel at c. denotes attenuation coefficient of the c channel at c. denotes the scene depth, i.e. the distance from the camera to c.

[0086] S302, estimating the background light.

[0087] In this embodiment, the background light is jointly estimated by the dark channel prior and the diffusion model, and the corresponding expression is:

[0088]

[0089] In the above formula, denotes the background light. denotes the local window centered at pixel x to define the neighborhood range for the background light estimation. r is the diffusion radius. is the minimum value of the RGB channel at pixel y , which is the core of the dark channel prior and reflects the degree of light attenuation.

[0090] S303, according to the color channel intensity of the reconstructed image, the background light and the transmittance, the color channel of the reconstructed image is recovered to obtain an enhanced image. The corresponding expression is as follows:

[0091]

[0092] In the above formula, denotes the image after the color channel recovery. denotes the intensity of the original input image c channel. denotes the background light at c.

[0093] S105, outputting the enhanced image.

[0094] As another possible embodiment, in step S103, the multi-scale diffusion is accelerated by FPGA, which specifically includes: the diffusion coefficient is pre-stored in BRAM, and the gradient calculation is realized by adopting the Sobel operator parallel pipeline. BRAM is a special high-speed storage module inside FPGA, which can be read and written in parallel, is suitable for storing frequently accessed data, and preloading these data can avoid delay. And the gradient calculation is realized by adopting the Sobel operator parallel pipeline, which can further improve the calculation speed.

[0095] As another possible embodiment, before outputting the enhanced image, the enhanced image is dynamically range compressed according to the brightness standard deviation of different regions of the enhanced image, and the compressed enhanced image is output. The corresponding expression is:​

[0096]

[0097]

[0098] In the above formula, represents the enhanced image after dynamic range compression, which optimizes the global contrast while preserving the details. represents the enhanced image is subjected to a pixel-level power transformation, which varies with the position. represents the standard deviation of brightness within the local window, which can be obtained by calculating the neighborhood standard deviation of the gray value of . represents the maximum standard deviation of the entire image.

[0099] As another possible implementation, with reference to Figure 4 , the following operations are performed before step S102:

[0100] S401, input the polarization image.

[0101] S402, calculate the polarization degree of the polarization image.

[0102] S403, correct the diffusion coefficient according to the polarization degree. The corresponding expression is:

[0103]

[0104] In the above formula, represents the corrected diffusion coefficient. represents the polarization gain coefficient. represents the polarization degree.

[0105] This implementation optimizes the diffusion coefficient by combining the polarization imaging information, thereby more accurately simulating or correcting the underwater light scattering effect.

[0106] As a possible implementation, after obtaining the enhanced image, the underwater image quality index of the enhanced image is calculated. If the underwater image quality index is lower than the preset quality index threshold, the attenuation coefficient matrix and the diffusion coefficient matrix are re-estimated, and steps S102-S105 are iteratively performed.

[0107] The calculation formula of the underwater image quality index is as follows:

[0108]

[0109] In the above formula, represents the underwater image quality index. represents the color richness, is the weight thereof. represents sharpness, i.e. the sharpness of edges and textures, is a weight for it. represents contrast, embodying the dynamic range of light and dark areas, is a weight for it.

[0110] The above description is merely that of the preferred embodiments of the application, and is not to be taken in a limiting sense, but is made merely for the purpose of disclosure, as the scope of the application will be defined by the claims.

Claims

1. An underwater image enhancement method based on a water diffusion physical model, characterized in that, The method comprises: S101, inputting an underwater image to a convolutional neural network to predict an attenuation coefficient matrix and a diffusion coefficient matrix of the water body, the convolutional neural network comprising a first branch and a second branch, the first branch predicting the attenuation coefficient matrix of the water body through a Sigmoid activation function, and the second branch predicting the diffusion coefficient matrix of the water body through a Softplus activation function; Attenuation coefficient matrix Describes the degree of energy attenuation of light when propagating in a water body, the greater the value, the faster the light intensity attenuates, that is, the more turbid the water body is. The attenuation coefficient matrix is the same size as the input underwater image, and each pixel position Corresponds to an attenuation coefficient value; Diffusion coefficient matrix For describing the scattering ability of water body to light, unit is , the larger the diffusion coefficient value, the stronger the scattering, the more blurred the image, the diffusion coefficient matrix with the same size as the input underwater image, each pixel position corresponds to a diffusion coefficient value; S102, constructing a water diffusion control equation based on the attenuation coefficient matrix and the diffusion coefficient matrix of the water body and combining the backscattering light intensity, and the corresponding expression is: In the above formula, denotes the partial differential operator, which is used to describe the relationship between the image brightness and the time t, is a gradient operator, which is used to calculate the spatial variation in the image or physical field, denotes the control of the degree of blur caused by scattering, and realizes spatial adaptation, denotes the control of light intensity attenuation, and realizes color distortion correction, is the backscattering weight, is the backscattering light intensity of the underwater image, and the water diffusion control equation is solved by semi-implicit finite difference method; S103, performing multi-scale diffusion on the underwater image in the frequency domain based on the water diffusion control equation to obtain a reconstructed image; S104, restoring the color channel of the reconstructed image through a spectral transmittance model to obtain an enhanced image; S105, outputting the enhanced image.

2. The underwater image enhancement method based on water diffusion physical model according to claim 1, characterized in that, Before step S101, the following operation is performed: simulating an underwater image degradation process based on a water diffusion physical model through a diffusion model to generate paired data for training the convolutional neural network, the paired data comprising a clear image and a degraded image.

3. The underwater image enhancement method based on water diffusion physical model according to claim 1, characterized in that, Step S103 specifically comprises the following operations: S201, performing multi-level wavelet transform on the underwater image to obtain subbands of different scales, the subbands of different scales comprising low-frequency subbands and high-frequency subbands; S202, adopting uniform diffusion on the low-frequency subbands; S203, adopting edge-aware diffusion on the high-frequency subbands; S204, performing weighted fusion on the subbands of different scales to reconstruct the image through inverse wavelet transform.

4. The underwater image enhancement method based on water diffusion physical model according to claim 1, characterized in that, Step S104 comprises the following steps: S301, calculating transmittance according to the attenuation coefficient matrix and the scene depth; S302, estimating background light; S303, restoring the color channel of the reconstructed image according to the color channel intensity of the reconstructed image, the background light and the transmittance to obtain the enhanced image.

5. The underwater image enhancement method based on water diffusion physical model according to claim 4, characterized in that, The background light is jointly estimated through dark channel prior and the diffusion coefficient matrix.

6. The underwater image enhancement method based on water diffusion physical model according to claim 1, characterized in that, In step S103, FPGA is used to accelerate the calculation of multi-scale diffusion, specifically comprising: pre-storing the diffusion coefficient in BRAM, and realizing gradient calculation in the manner of Sobel operator parallel pipeline.

7. The underwater image enhancement method based on water diffusion physical model according to claim 1, characterized in that, Before outputting the enhanced image, the enhanced image is dynamically range compressed according to the brightness standard deviation of different regions of the enhanced image, and the compressed enhanced image is output.

8. The underwater image enhancement method based on water diffusion physical model according to claim 1, characterized in that, Before step S102, the following operation is performed: S401, inputting a polarization image; S402, calculating the polarization degree of the polarization image; S403, correcting the diffusion coefficient according to the polarization degree.

Images