Underwater image enhancement method based on water diffusion physical model

Abstract

The invention provides an underwater image enhancement method based on a water diffusion physical model, and relates to the technical field of underwater image enhancement, and the method comprises the steps: S101, inputting a local block of an underwater image to a convolutional neural network, and predicting an attenuation coefficient matrix and a diffusion matrix of a water body; s102, constructing a water diffusion control equation based on the attenuation coefficient matrix and the diffusion matrix of the water body; s103, performing multi-scale diffusion on the underwater image in a frequency domain based on a water diffusion control equation to obtain a reconstructed image; s104, recovering a color channel of the reconstructed image through a spectral transmittance model to obtain an enhanced image; and S105, outputting an enhanced image. According to the method, by estimating the attenuation and scattering coefficients of the light underwater, image enhancement which better conforms to the physical diffusion characteristics of the light underwater is achieved, color distortion and image blurring are effectively avoided, and the physical interpretability and reliability of an enhancement result are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image enhancement, and in particular to an underwater image enhancement method based on a water diffusion physical model. Background Art

[0002] Underwater image acquisition and further processing are essential in a variety of fields, including marine science research, underwater engineering inspection, and underwater biological observation. For example, in marine resource exploration, underwater images of a designated area are collected to identify mineral distribution and biological communities. In industry, underwater robots are used to capture images of pipelines or ship hulls for defect detection. In archaeology, underwater images are used to record the appearance of shipwrecks or ruins. These diverse applications all require underwater images with high clarity and fidelity. However, due to the complex underwater optical environment, underwater images often suffer from severe degradation, directly impacting the reliability and accuracy of image recognition and processing. Improving underwater image quality not only improves visual perception but also enhances the accuracy of object detection, segmentation, and recognition. Therefore, enhancing underwater images is of great scientific and practical value.

[0003] Currently, underwater image enhancement methods are mainly divided into two categories: image processing-based methods and deep learning-based methods. Traditional image processing methods such as histogram equalization and color correction rely heavily on artificial priors, have poor generalization capabilities, and are prone to over-enhancement or local distortion. While deep learning methods perform well in many scenarios, they rely on large amounts of high-quality data for training, lack physical interpretability, and are prone to introducing artifacts or color deviations. The main problem with these methods is that they fail to fully integrate the physical processes of underwater imaging, especially the propagation characteristics of light in water. Underwater image degradation mainly stems from energy attenuation and color distortion caused by light absorption and scattering by water. The scattering effect can be described by the physical model of water diffusion. Ignoring this physical property and enhancing only at the image level often makes it difficult to accurately restore the true color and structural details of the scene. Summary of the Invention

[0004] The purpose of the present invention is to provide an underwater image enhancement method based on a water diffusion physical model, which achieves image enhancement that is more in line with physical laws by estimating the attenuation and scattering coefficient of light underwater.

[0005] To achieve the above-mentioned object, the present invention provides an underwater image enhancement method based on a water diffusion physical model, the method comprising: S101, input the underwater image into the convolutional neural network to predict the attenuation coefficient matrix and diffusion coefficient matrix of the water body; S102, constructing a water diffusion control equation based on the attenuation coefficient matrix and the diffusion coefficient matrix of the water body; S103, performing multi-scale diffusion on the underwater image in the frequency domain based on the water diffusion governing equation to obtain a reconstructed image; S104, restoring the color channels of the reconstructed image using a spectral transmittance model to obtain an enhanced image; S105: Output the enhanced image.

[0006] Furthermore, in step S101, the convolutional neural network includes a first branch and a second branch, the first branch predicts the attenuation coefficient matrix of the water body through the Sigmoid activation function, and the second branch predicts the diffusion coefficient matrix of the water body through the Softplus activation function.

[0007] Furthermore, before step S101, the following operations are performed: a diffusion model is used to simulate the underwater image degradation process based on a water diffusion physical model to generate paired data for training a convolutional neural network, wherein the paired data includes a clear image and a degraded image.

[0008] Furthermore, in step S102, a water diffusion control equation is constructed in combination with the backscattered light intensity, and the water diffusion control equation is solved by a semi-implicit finite difference method.

[0009] Furthermore, step S103 specifically includes the following operations: S201, performing a multi-level wavelet transform on the underwater image to obtain sub-bands of different scales, wherein the sub-bands of different scales include low-frequency sub-bands and high-frequency sub-bands; S202, applying uniform diffusion to the low-frequency sub-band; S203, applying edge-aware diffusion to the high-frequency sub-band; S204: Perform weighted fusion on sub-bands of different scales and reconstruct the image through inverse wavelet transform.

[0010] Furthermore, step S104 includes the following steps: S301, calculating transmittance according to the attenuation coefficient matrix and scene depth; S302, estimating background light; S303 , restoring the color channel of the reconstructed image according to the color channel intensity, background light, and transmittance of the reconstructed image to obtain an enhanced image.

[0011] Furthermore, the background light is jointly estimated using the dark channel prior and the diffusion coefficient matrix.

[0012] Furthermore, in step S103, the multi-scale diffusion is accelerated by FPGA, specifically including: pre-storing the diffusion coefficient in BRAM, and implementing gradient calculation by using a Sobel operator parallel pipeline.

[0013] Furthermore, before outputting the enhanced image, dynamic range compression is performed on the enhanced image according to the brightness standard deviation of different regions of the enhanced image, and the compressed enhanced image is output.

[0014] Furthermore, before step S102, the following operations are performed: S401, input polarization image; S402, calculating the polarization degree of the polarization image; S403: Correct the diffusion coefficient according to the polarization degree.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides an underwater image enhancement method based on a water diffusion physical model. The attenuation coefficient matrix and diffusion coefficient matrix of the water body in the underwater image are estimated through a convolutional neural network. The water diffusion control equation is constructed according to the attenuation coefficient matrix and the diffusion coefficient matrix to perform multi-scale diffusion on the underwater image in the frequency domain. Finally, the enhanced image is obtained by restoring the various color channels of the image, thereby achieving image enhancement that is more in line with the physical diffusion characteristics of light underwater, effectively avoiding color distortion and image blur, and improving the physical interpretability and reliability of the enhancement results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 This is a schematic diagram of the overall process of an underwater image enhancement method based on a water diffusion physical model provided by an embodiment of the present invention.

[0018] Figure 2 4 is a schematic diagram of the underwater image reconstruction process provided by an embodiment of the present invention.

[0019] Figure 3 The figure is a schematic diagram of the image color channel restoration process provided by an embodiment of the present invention.

[0020] Figure 4 It is a schematic diagram of the diffusion coefficient correction process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The principles and features of the present invention are described below with reference to the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0022] Reference Figure 1This embodiment provides an underwater image enhancement method based on a water diffusion physical model, the method comprising: S101. Input the underwater image into the convolutional neural network to predict the attenuation coefficient matrix and diffusion coefficient matrix of the water body.

[0023] In this embodiment, represents the original underwater image to be enhanced, where Represents the two-dimensional coordinates of pixels in an image. The data type is usually an RGB or grayscale matrix.

[0024] Attenuation coefficient matrix Describes the energy attenuation of light when it propagates in water. The larger the value, the faster the light intensity decays, that is, the more turbid the water. The attenuation coefficient matrix of the same size as the input underwater image, each pixel position Corresponding to an attenuation coefficient value, for example, clear water area , turbid area .

[0025] Diffusion coefficient matrix Used to describe the scattering ability of water to light, the unit is The larger the diffusion coefficient value, the stronger the scattering and the more blurred the image. The diffusion coefficient matrix of the same size as the input underwater image, each pixel position Corresponding to a diffusion coefficient value.

[0026] As a preferred embodiment, the input to the convolutional neural network is an image block of an underwater image, which can be 32×32 pixels in size. The convolutional neural network includes a first branch and a second branch. The first branch uses a Sigmoid activation function to predict the attenuation coefficient matrix of the water body, which can be expressed as:

[0027] In the above formula, Represents the underwater image obtained by the first branch of the convolutional neural network To process, Indicates the maximum attenuation coefficient based on the water body type settings.

[0028] The second branch predicts the diffusion coefficient matrix of the water body through the Softplus activation function, which can be expressed as:

[0029] In the above formula, Represents the underwater image obtained by the first branch of the convolutional neural network To process, Indicates the maximum diffusion coefficient value based on the water body type setting.

[0030] As another possible implementation, before step S101, a diffusion model is used to simulate the degradation process of underwater images based on a water diffusion physics model to generate paired data for training a convolutional neural network. The paired data includes a clear image and a degraded image. The corresponding expression is:

[0031] In the above formula, represents the degraded image, Indicates a clear image, represents the diffusion core, Represents attenuation-related noise. The attenuation of light in water follows the Beer-Lambert law: the stronger the attenuation, the weaker the signal, and the more significant the noise.

[0032] This implementation simulates the underwater image degradation process through a diffusion model, which can quickly generate a large amount of high-quality paired data that conforms to the physical propagation characteristics of light underwater, and improve its performance by training the convolutional neural network.

[0033] S102. Construct a water diffusion control equation based on the attenuation coefficient matrix and diffusion matrix of the water body.

[0034] As a possible implementation, in step S102, the backscattered light intensity is further combined to construct a water diffusion control equation, and the corresponding expression is:

[0035] In the above formula, Represents the partial differential operator, which is used to describe the relationship between image brightness and time t. is a gradient operator used to calculate spatial changes in images or physical fields. Indicates the degree of blur caused by controlling scattering to achieve spatial adaptation. Indicates controlling light intensity attenuation to achieve color distortion correction. is the backscattering weight, is the backscattered light intensity of the underwater image.

[0036] In this embodiment, the water diffusion governing equation can be solved by a semi-implicit finite difference method. The corresponding expression is:

[0037] In the above formula, Represents the image matrix of the discretized time step; represents the time step; represents the implicit matrix of diffusion term, corresponding to ; represents the explicit matrix of attenuation term, corresponding to .

[0038] S103 , performing multi-scale diffusion on the underwater image in the frequency domain based on the water diffusion governing equation to obtain a reconstructed image.

[0039] As a possible implementation method, refer to Figure 2 Step S103 specifically includes the following operations: S201 . Perform multi-level wavelet transform on the underwater image to obtain sub-bands of different scales, where the sub-bands of different scales include low-frequency sub-bands and high-frequency sub-bands.

[0040] In this embodiment, the low-frequency sub-band contains the smooth background and color cast of the underwater image; the high-frequency sub-band contains edges / textures / noise in the horizontal, vertical, and diagonal directions.

[0041] S202. Use uniform diffusion for the low-frequency sub-band. The corresponding expression is:

[0042] In the above formula, Represents the low-frequency subband.

[0043] S203, edge-aware diffusion is used for the high-frequency sub-band. The corresponding expression is:

[0044] In the above formula, represents the high frequency subband, represents the gradient of the high-frequency subband, represents the edge threshold parameter, represents the divergence operator. When the gradient (strong edge region), the denominator approaches The square of the gradient approaches 0, thus preserving the edge. (flat or noisy area), the denominator is close to 1, the diffusion coefficient is large, and while smoothing the noise to remove noise, it avoids edge blurring.

[0045] S204: Perform weighted fusion on sub-bands of different scales and reconstruct the image through inverse wavelet transform.

[0046] In this embodiment, the expression for reconstructing the image by inverse wavelet transform is:

[0047] In the above formula, Represents the reconstructed image. Represents the high-frequency subband weight, which is obtained by adaptive calculation of the local signal-to-noise ratio (SNR). Represents the high frequency subband of level k. Represents the low-frequency subband weight, which is also obtained by adaptive calculation of the local signal-to-noise ratio. Represents the low-frequency subband of level k. K represents the number of levels of wavelet decomposition and can be set according to the image resolution.

[0048] Exemplarily, the calculation formulas for the high-frequency sub-band weight and the low-frequency self-band weight are as follows:

[0049]

[0050]

[0051]

[0052] In the above formula, Represents the low-frequency background light estimation value of the kth level. Represents the variance calculation function. Represents noise. When the signal-to-noise ratio (SNR) of an image region is high, the weight of the high-frequency subband is increased to enhance detail preservation. When the SNR of an image region is low, the weight of the low-frequency subband is increased to suppress noise and color cast.

[0053] S104: Restoring the color channels of the reconstructed image using a spectral transmittance model to obtain an enhanced image.

[0054] As a possible implementation method, refer to Figure 3 , step S104 specifically includes the following operations: S301: Calculate transmittance according to the attenuation coefficient matrix and scene depth.

[0055] In this embodiment, the transmittance is calculated as follows:

[0056] In the above formula, express The transmittance of channel c. express The attenuation coefficient of channel c. Indicates the scene depth, that is, the camera to The distance between the two places.

[0057] S302: Estimate the background light.

[0058] In this embodiment, the background light is estimated jointly by the dark channel prior and the diffusion model, and the corresponding expression is:

[0059] In the above formula, Indicates background light. Indicated in pixels xThe local window centered on is used to limit the neighborhood range of background light estimation. r is the diffusion radius. It's a pixel y The minimum value of the RGB channels, the core of the dark channel prior, reflects the degree of light attenuation.

[0060] S303: Restore the color channel of the reconstructed image according to the color channel intensity, background light, and transmittance of the reconstructed image to obtain an enhanced image. The corresponding expression is as follows:

[0061] In the above formula, Represents the image after color channel restoration. Represents the intensity of the c channel of the original input image. express Background light.

[0062] S105: Output the enhanced image.

[0063] As another possible implementation, in step S103, multi-scale diffusion calculations are accelerated using an FPGA. This specifically includes pre-storing the diffusion coefficients in BRAM and implementing gradient calculations using a parallel pipeline of Sobel operators. BRAM is a dedicated high-speed storage module within the FPGA that allows parallel read and write operations. It is suitable for storing frequently accessed data, and pre-loading this data can avoid latency. Implementing gradient calculations using a parallel pipeline of Sobel operators can further improve calculation speed.

[0064] As another possible implementation, before outputting the enhanced image, dynamic range compression is performed on the enhanced image based on the brightness standard deviation of different regions of the enhanced image, and the compressed enhanced image is output. The corresponding expression is:

[0065]

[0066] In the above formula, Represents an enhanced image after dynamic range compression, preserving details while optimizing global contrast. Represents the enhanced image Perform pixel-level power transformation. Change with position. Represents the brightness standard deviation within the local window, which can be obtained by The gray value is obtained by calculating the neighborhood standard deviation. Indicates the maximum standard deviation of the entire image.

[0067] As another possible implementation, refer to Figure 4 Before step S102, perform the following operations: S401: Input a polarization image.

[0068] S402: Calculate the polarization degree of the polarization image.

[0069] S403. Correct the diffusion coefficient according to the polarization degree. The corresponding expression is:

[0070] In the above formula, represents the corrected diffusion coefficient. represents the polarization gain coefficient. Indicates the degree of polarization.

[0071] This embodiment optimizes the diffusion coefficient by combining polarization imaging information, thereby more accurately simulating or correcting the underwater light scattering effect.

[0072] As a possible implementation, after obtaining the enhanced image, an underwater image quality index of the enhanced image is calculated. If the underwater image quality index is lower than a preset quality index threshold, the attenuation coefficient matrix and the diffusion coefficient matrix are re-estimated, and steps S102 to S105 are iteratively performed.

[0073] The calculation formula of underwater image quality index is as follows:

[0074] In the above formula, Indicates the underwater image quality index. Indicates color richness, For its weight. Indicates clarity, that is, the sharpness of edges and textures, For its weight. Indicates contrast, reflecting the dynamic range of light and dark areas, For its weight.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An underwater image enhancement method based on a water diffusion physical model, characterized in that: The method comprises: S101, input the underwater image into the convolutional neural network to predict the attenuation coefficient matrix and diffusion coefficient matrix of the water body; S102, constructing a water diffusion control equation based on the attenuation coefficient matrix and the diffusion coefficient matrix of the water body; S103, performing multi-scale diffusion on the underwater image in the frequency domain based on the water diffusion governing equation to obtain a reconstructed image; S104, restoring the color channels of the reconstructed image using a spectral transmittance model to obtain an enhanced image; S105: Output the enhanced image.

2. The underwater image enhancement method based on the water diffusion physical model according to claim 1, characterized in that: In step S101, the convolutional neural network includes a first branch and a second branch. The first branch predicts the attenuation coefficient matrix of the water body through the Sigmoid activation function, and the second branch predicts the diffusion coefficient matrix of the water body through the Softplus activation function.

3. The underwater image enhancement method based on the water diffusion physical model according to claim 1 or 2, characterized in that: Before step S101, the following operations are performed: the underwater image degradation process is simulated by a diffusion model based on a water diffusion physical model to generate paired data for training a convolutional neural network, where the paired data includes a clear image and a degraded image.

4. The underwater image enhancement method based on the water diffusion physical model according to claim 1, characterized in that: In step S102, the backscattered light intensity is further combined to construct a water diffusion control equation, and the water diffusion control equation is solved by a semi-implicit finite difference method.

5. The underwater image enhancement method based on the water diffusion physical model according to claim 1, characterized in that: Step S103 specifically includes the following operations: S201, performing a multi-level wavelet transform on the underwater image to obtain sub-bands of different scales, wherein the sub-bands of different scales include low-frequency sub-bands and high-frequency sub-bands; S202, applying uniform diffusion to the low-frequency sub-band; S203, applying edge-aware diffusion to the high-frequency sub-band; S204: Perform weighted fusion on sub-bands of different scales and reconstruct the image through inverse wavelet transform.

6. The underwater image enhancement method based on the water diffusion physical model according to claim 1, characterized in that: Step S104 includes the following steps: S301, calculating transmittance according to the attenuation coefficient matrix and scene depth; S302, estimating background light; S303 , restoring the color channel of the reconstructed image according to the color channel intensity, background light, and transmittance of the reconstructed image to obtain an enhanced image.

7. The underwater image enhancement method based on the water diffusion physical model according to claim 6, characterized in that: The background light is jointly estimated using the dark channel prior and the diffusion coefficient matrix.

8. The underwater image enhancement method based on the water diffusion physical model according to claim 1, characterized in that: In step S103, the multi-scale diffusion is accelerated by FPGA, specifically including: pre-storing the diffusion coefficient in BRAM, and implementing gradient calculation by using a Sobel operator parallel pipeline.

9. The underwater image enhancement method based on the water diffusion physical model according to claim 1, characterized in that: Before outputting the enhanced image, dynamic range compression is performed on the enhanced image according to the brightness standard deviation of different areas of the enhanced image, and the compressed enhanced image is output.

10. The underwater image enhancement method based on the water diffusion physical model according to claim 1, characterized in that: Before step S102, perform the following operations: S401, input polarization image; S402, calculating the polarization degree of the polarization image; S403: Correct the diffusion coefficient according to the polarization degree.

Images