Underwater image enhancement method based on depth perception and color restoration

Abstract

The invention relates to the field of image processing, in particular to an underwater image enhancement method based on depth perception and color restoration. The method comprises the following steps: acquiring underwater original image data; preprocessing the image data, and extracting illumination distribution and color shift features; the texture, the edge and the details of the image are enhanced by adopting a convolutional neural network through a depth feature enhancement module; dynamically adjusting the hue, saturation, brightness and contrast of the image through an adaptive color balance module; and reconstructing and outputting the enhanced image. According to the method, the visual quality and availability of underwater images are remarkably improved, the method can be widely applied to the fields of underwater engineering detection, marine resource exploration, unmanned underwater vehicle environment perception, underwater security and protection monitoring and the like, and the method has important economic and social values of improving the operation precision, reducing the manual interpretation risk, saving the monitoring cost and the like.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an underwater image enhancement method based on depth perception and color restoration. Background Technology

[0002] With the widespread application of underwater operations, unmanned underwater vehicles (AUVs), underwater robots, and other equipment in marine engineering, environmental monitoring, and scientific exploration, acquiring clear and accurate underwater images has become a key support for the development of related technologies. However, due to light attenuation and scattering in the underwater environment, the propagation capabilities of different wavelengths of light vary greatly, which can easily lead to problems such as obvious color distortion, insufficient brightness, low contrast, and blurred details in images.

[0003] Currently, common underwater image enhancement methods mainly include the following categories:

[0004] 1. Traditional image processing methods, such as histogram equalization and Retinex algorithm, can improve image contrast and brightness to some extent, but the enhancement results are prone to artifacts and have poor adaptability to different water conditions.

[0005] 2. Image restoration methods based on physical modeling recover the true colors of an image by analyzing the absorption and scattering characteristics of light by water bodies through inverse modeling. However, such methods typically rely on external parameters or water quality models, resulting in poor versatility and real-time performance.

[0006] 3. Deep learning-based image enhancement methods have attracted attention in recent years, enabling end-to-end image processing via convolutional neural networks to achieve joint optimization of color and detail. However, they suffer from strong dependence on training data and high processing instability.

[0007] In summary, existing methods either lack flexibility or are unable to restore color and brightness while preserving detail. Therefore, there is still a need for an underwater image enhancement method with strong adaptability, stability, and enhancement effect. Summary of the Invention

[0008] The purpose of this invention is to provide an underwater image enhancement method based on depth perception and color restoration, which combines multi-channel image analysis and convolutional neural network technology to effectively improve image contrast, brightness balance and color reproduction, and enhance image texture and edge details.

[0009] To achieve the above objectives, this invention provides an underwater image enhancement method based on depth perception and color restoration, the method comprising:

[0010] S1: Acquire raw underwater images to be enhanced;

[0011] S2: Preprocess the image to extract illumination distribution and color shift features;

[0012] S3: Through the deep feature enhancement module, a convolutional neural network is used to enhance the texture, edges and details of the image;

[0013] S4: Dynamically adjusts the hue, saturation, brightness, and contrast of an image through the adaptive color balance module;

[0014] S5: Reconstruct and output the enhanced image.

[0015] Optionally, the image acquisition method described in S1 includes: using underwater cameras, unmanned underwater vehicles, or underwater robots to acquire raw underwater images in real time, and inputting the image data into an image enhancement system for subsequent processing; furthermore, polarization imaging technology can be integrated during the image acquisition stage to suppress backscattering noise and improve the quality of the raw image.

[0016] Optionally, the image preprocessing in step S2 includes the following sub-steps:

[0017] S21: Extract the illumination distribution information in the image using a multi-channel brightness analysis method, and perform brightness correction in combination with water body characteristics;

[0018] Optionally, in S21, the local brightness histogram of the image is used to analyze the illumination changes of the underwater image, and an adaptive algorithm is used to dynamically adjust the image brightness and contrast, thereby eliminating the influence of uneven image illumination.

[0019] S22: Using the local contrast stretching method of the image, the contrast of the image is automatically adjusted and special processing is performed on the dark areas in the underwater environment.

[0020] S23: Employs a gradient-based enhancement algorithm to dynamically adjust the contrast of each region in the image, avoiding the loss of detail caused by over-enhancement.

[0021] Optionally, in S23, enhancement weights are set by analyzing the gradient magnitude distribution of each region of the image, preserving edge details in high-frequency regions and smoothing enhancement in low-frequency regions, thereby improving the clarity of the image structure while suppressing noise.

[0022] Optionally, in S3, the deep feature enhancement module is implemented based on a convolutional neural network with an encoder-decoder structure, including the following sub-steps:

[0023] S31: Multi-scale feature extraction of images is performed through a five-layer encoder and five-layer decoder structure, and skip connections with convolutional block attention modules (CBAM) are introduced between corresponding layers of the encoder and decoder to enhance the texture and detail information in underwater images.

[0024] S32: It combines attention mechanism with skip connections to optimize the edge and detail representation of images, making image details clearer and more natural; furthermore, perceptual loss and adversarial loss can be introduced during the training phase to improve the quality of visual perception.

[0025] Optionally, the convolutional neural network described in S3 can be pre-trained on a training dataset containing underwater images to improve the network's generalization ability and enhancement effect.

[0026] Optionally, in S4, the adaptive color balance module includes the following sub-steps:

[0027] S41: Convert the image to the CIELab color space and quantize the color shift of the a and b channels;

[0028] S42: Combining the gray world hypothesis and the perfect reflector hypothesis, dynamically generate color correction coefficients for each channel;

[0029] S43: Based on the image brightness distribution characteristics, adaptive gain adjustment is performed on the RGB channels to restore the natural colors of the underwater image.

[0030] Optionally, the gain adjustment process adopts a regional weighting mechanism to achieve differentiated processing of the degree of color correction in different regions, so as to adapt to different depths and water quality conditions.

[0031] Optionally, the image reconstruction process in S5 includes:

[0032] The enhanced image undergoes multi-scale feature reconstruction, and an adaptive filtering algorithm based on local noise variance and edge intensity detection is employed to remove potential noise. Specifically, the filtering algorithm calculates the noise variance and edge intensity of each region in the image, applies Gaussian filtering to high-noise, non-edge regions, bilateral filtering to high-noise edge regions, and weak filtering or preservation to low-noise regions. This process suppresses noise while preserving the clarity of image details and edge continuity, ensuring the output image possesses a natural feel and high fidelity. Furthermore, sharpening mask techniques can be introduced to enhance the visual effect of edges.

[0033] This invention discloses an underwater image enhancement method based on depth perception and color restoration. The method includes: acquiring raw underwater image data; preprocessing the image data to extract illumination distribution and color shift features; enhancing the texture, edges, and details of the image using a convolutional neural network through a depth feature enhancement module; dynamically adjusting the hue, saturation, brightness, and contrast of the image through an adaptive color balance module; and reconstructing and outputting the enhanced image. This method improves the visibility of underwater images and is suitable for applications such as underwater operations, unmanned underwater vehicles, and underwater inspection. Attached Figure Description

[0034] Figure 1 This is an overall flowchart of an underwater image enhancement method based on depth perception and color restoration according to the present invention;

[0035] Figure 2 This is a schematic diagram of the image preprocessing module in this invention;

[0036] Figure 3 This is a schematic diagram of the deep feature enhancement module in this invention;

[0037] Figure 4 This is a flowchart of the adaptive color balance module in this invention;

[0038] Figure 5 This is a schematic diagram showing the comparison before and after image reconstruction and enhancement in this invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0040] Please see Figure 1 This invention provides an underwater image enhancement method based on depth perception and color restoration, comprising the following steps:

[0041] S1: Acquire raw underwater images to be enhanced;

[0042] S2: Preprocess the image to extract illumination distribution and color shift features;

[0043] S3: Through the deep feature enhancement module, a convolutional neural network (CNN) is used to enhance the texture, edges, and details of the image;

[0044] S4: Dynamically adjusts the hue, saturation, brightness, and contrast of an image through the adaptive color balance module;

[0045] S5: Reconstruct and output the enhanced image.

[0046] The present invention will be further described below in conjunction with the implementation steps:

[0047] Step S1 involves acquiring the original underwater image to be enhanced, as detailed below:

[0048] Underwater images can be acquired using underwater cameras mounted on equipment such as work platforms, unmanned underwater vehicles (AUVs), or underwater robots. This image acquisition system can acquire RGB format underwater images in real time under different depths and lighting conditions, and transmit them to an image enhancement system via wired or wireless communication modules as input image data for subsequent processing. Preferably, the image resolution is 1920×1080, and the frame rate is 30 frames per second to ensure sufficient preservation of image details.

[0049] Furthermore, considering the complex optical characteristics of the underwater environment, this invention introduces an active illumination compensation mechanism during the image acquisition stage. An adjustable-intensity LED array installed on the acquisition device automatically adjusts the illumination parameters based on water depth and ambient light intensity. The illumination system employs a multi-band LED combination, including three main wavelengths: 450nm blue light, 520nm green light, and 630nm red light, effectively compensating for the differences in light attenuation across different wavelengths underwater. To reduce the impact of backscattering, the illumination system adopts a structured light design, with the light source arranged at a specific angle to the camera axis, and a polarizing filter installed in front of the lens, effectively suppressing scattering noise caused by suspended particles and improving the quality of the original image.

[0050] Step S2 is the image preprocessing module, which extracts illumination distribution and color shift features. The image preprocessing module mainly includes three sub-modules: multi-channel brightness analysis, local contrast enhancement, and gradient enhancement. Please refer to [link / reference]. Figure 2 The details are as follows:

[0051] S21: Multi-channel brightness analysis and brightness correction;

[0052] First, calculate the brightness distribution for the R, G, and B channels of the image respectively:

[0053]

[0054] Subsequently, brightness histogram analysis was performed on local areas of the image, and an adaptive adjustment function was used to compensate for brightness deviation areas.

[0055]

[0056] in, It is a dynamic brightness adjustment factor that varies according to the range of brightness in the area, and its value is usually between 0.8 and 1.5.

[0057] S22: Local contrast stretching;

[0058] To enhance the visibility of dark areas in underwater images, a local contrast enhancement method based on a sliding window is employed in this step. Specifically, the image is divided into several local regions (such as image blocks of fixed size). Within each region, the range of maximum and minimum pixel values ​​is calculated, and the image grayscale is linearly stretched based on its brightness distribution.

[0059] S23: Gradient-based boosting algorithm;

[0060] Further enhance image edges and texture structure, and calculate image gradient magnitude:

[0061]

[0062] Enhancement weights are set based on gradient values ​​in different regions. This achieves the preservation of edge details and smoothing of background areas.

[0063]

[0064] The high-frequency region w∈[0.2,0.5] and the low-frequency region w∈[0.6,1.0].

[0065] Furthermore, this invention introduces a multi-scale gradient fusion technique. Multi-scale gradient information is calculated using Sobel operators of different sizes (3×3, 5×5, 7×7), and the gradients at each scale are weighted and fused. The weighting coefficients are adaptively adjusted according to regional characteristics: in regions with complex textures, the weight of small-scale gradients is increased to enhance details; in flat regions, the weight of large-scale gradients is increased to suppress noise. This multi-scale processing method can effectively maintain the natural visual effect of the image while enhancing edges.

[0066] Step S3 enhances image texture, edges, and details using the depth feature enhancement module. Please refer to [link / reference]. Figure 3 The details are as follows:

[0067] S31: Multi-scale feature extraction and skip connections;

[0068] A convolutional network structure consisting of a 5-layer encoder and a 5-layer decoder is used to extract features at different scales of the image. Each layer of the encoder contains a 3×3 convolution, a batch normalization layer, a ReLU activation function, and a downsampling operation. Each layer of the decoder contains a transposed convolution, a batch normalization layer, and a ReLU activation function. Skip connections are used to fuse the encoder features with the corresponding layer features of the decoder.

[0069] F_decoder = F_decoder + F_encoder

[0070] Where F_decoder is the output feature of the current layer, and F_encoder is the input skip connection feature.

[0071] Furthermore, this invention introduces a dense connection mechanism into the network design. Dense connection blocks are employed within each encoder and decoder layer, with each block containing four convolutional layers. The output of each layer is connected to the input of all subsequent layers. This design enhances feature reuse, improves gradient flow, and enables the network to better learn the complex degradation features of underwater images. Simultaneously, a feature selection mechanism is introduced in the skip connections, generating attention weights through 1×1 convolutions and a sigmoid activation function to adaptively select important features for transmission.

[0072] S32: Attention mechanism optimization;

[0073] Introducing the CBAM attention module, which includes channel attention and spatial attention sub-modules;

[0074] Channel attention is implemented using a combination of global average pooling and max pooling in an MLP:

[0075]

[0076] Spatial attention is achieved through convolution of the channel pooling graph:

[0077]

[0078] The final output is:

[0079]

[0080] The enhanced feature map exhibits better local responsiveness and edge sharpness.

[0081] Step S4 involves color restoration using the adaptive color balance module. Please refer to [link / reference]. Figure 4 The details are as follows:

[0082] S41: Color Shift Quantization;

[0083] The image is converted from the RGB color space to the CIELab color space, and the mean μ_a, μ_b and standard deviation σ_a, σ_b of the a channel and the b channel are calculated respectively to quantify the degree of color shift.

[0084] S42: Color correction coefficient generation;

[0085] Combining the gray world assumption and the perfect reflector assumption, the correction coefficients for each channel are dynamically calculated. For each RGB channel, the deviation of its mean from the ideal gray reference is calculated, and a correction coefficient matrix is ​​generated.

[0086] S43: Adaptive gain adjustment; Based on the brightness distribution of different regions in the image, weighted gain adjustment is applied to the R, G, and B channels to restore color balance.

[0087]

[0088] Where γ is the gain factor adaptively calculated based on the degree of color shift, L(x,y) is the local brightness, and L_avg is the average brightness of the image.

[0089] Furthermore, this invention employs a partitioned color mapping strategy. First, a clustering algorithm divides the image into regions with similar color characteristics, and color correction parameters are calculated independently for each region. During the correction process, region boundary smoothing is introduced, and guided filtering is used to smooth the corrected image while preserving edges, eliminating color discontinuities between regions. This partitioned processing method better maintains local color consistency in the image and avoids color patchiness.

[0090] Step S5 involves reconstructing and outputting the enhanced image, as detailed below:

[0091] The enhanced image may contain local artifacts or noise; therefore, image reconstruction and filtering based on noise detection are performed before output. Specifically, an adaptive filtering algorithm based on local noise variance and edge intensity detection is used.

[0092] First, calculate the noise variance σ for each local region (e.g., an 8×8 pixel block) in the image. n ² and edge strength E. Noise variance is obtained by calculating the variance of pixel values ​​within the region, and edge strength is obtained by calculating the mean gradient magnitude of the region.

[0093] Subsequently, based on the calculated noise level and edge information, a filtering strategy is adaptively selected:

[0094] For high noise (σ n ² For regions that are greater than the set threshold T1 and are not edge regions (E less than the threshold T2), Gaussian filtering is used for smoothing and noise reduction.

[0095] For high noise (σ n For regions that are greater than the threshold T1 and are edges (E is greater than or equal to the threshold T2), a bilateral filter with edge preservation characteristics is used to protect the edge structure while denoising.

[0096] For low noise (σ) n For regions less than or equal to the threshold T1, perform very weak detail-preserving filtering or directly retain the original enhancement result to avoid loss of detail; furthermore, non-local mean filtering can be introduced to process high-noise flat regions to improve the denoising effect.

[0097] Furthermore, this invention introduces a multi-scale denoising method based on wavelet transform. The image is decomposed into three levels of wavelet decomposition, and an adaptive threshold denoising strategy is applied to the coefficients of each sub-band. The threshold is dynamically adjusted according to the noise level and edge information of the sub-band: a softer threshold is used for high-frequency sub-bands containing important edge information to preserve edge details; a harder threshold is used for high-frequency sub-bands mainly containing noise to effectively suppress noise. Finally, the denoised image is obtained through wavelet reconstruction. This method can effectively suppress noise across the entire image range while maintaining the integrity of details.

[0098] This adaptive filtering process effectively suppresses noise and artifacts while preserving the sharpness of image details and the continuity of edges to the maximum extent. Finally, the processed image undergoes an inverse color space transformation (if necessary) and is output, ensuring that the final enhanced result has a high degree of naturalness and fidelity.

[0099] To verify the effectiveness of the underwater image enhancement method based on depth perception and color restoration proposed in this invention, experiments were conducted on the publicly available underwater image dataset UIEB and a self-collected underwater image dataset. The model was implemented on the PyTorch platform, with 200 training epochs, using AdamW as the optimizer, and an input image size of 512×512. The experimental results are shown in Table 1. The proposed method achieved PSNR, SSIM, UCIQE, and UIQM scores of 21.2 dB, 0.75, 0.63, and 2.4, respectively, while the corresponding values ​​for the original images were only 18.3 dB, 0.68, 0.56, and 2.2.

[0100] Evaluation indicators Original image Method of the present invention PSNR (dB) 18.3 21.2 SSIM 0.68 0.75 UIQM 2.2 2.4 UCIQE 0.56 0.63

[0101] Table 1. Quantitative evaluation of the underwater image enhancement effect of the method of the present invention.

[0102] The results show that... (Please refer to...) Figure 5 As shown, the method of the present invention can significantly improve image clarity and structural fidelity, while effectively restoring underwater color and visibility. The enhanced image has natural colors and rich details, and is suitable for underwater target detection and operation monitoring, fully verifying the effectiveness and practical value of the method of the present invention.

[0103] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. Various changes, modifications, substitutions, and variations made to these embodiments without departing from the principles and spirit of the present invention still fall within the protection scope of the present invention.

Claims

1. An underwater image enhancement method based on depth perception and color inpainting, characterized in that, The method includes the following steps: S1: Obtain the original underwater image to be enhanced; S2: Preprocess the original underwater image to extract the illumination distribution and color shift information of the image; S3: The texture, edge and detail features of the image are enhanced by a convolutional neural network (CNN) containing a five-layer encoder and a five-layer decoder, wherein the encoder and decoder are connected by a skip connection with a convolutional block attention module (CBAM). S4: Dynamically adjusts the hue, saturation, brightness, and contrast of the image through the adaptive color balance module; S5: Reconstruct the enhanced and color-adjusted image and output the final enhanced result.

2. The underwater image enhancement method according to claim 1, characterized in that, Step S1 includes: Raw underwater images are acquired using underwater cameras, unmanned underwater vehicles, or underwater robots, and then input into an image processing system.

3. The underwater image enhancement method according to claim 1, characterized in that, Step S2 includes: S21: Extract illumination distribution information from the image using a multi-channel brightness analysis method, and perform adaptive brightness correction by combining the optical properties of the water body; S22: The local contrast stretching method is used to enhance the image contrast, and the dark areas in the underwater image are enhanced with detail. S23: Apply an adaptive enhancement algorithm based on gradient information to adjust the image contrast in different regions to avoid over-enhancement and loss of detail.

4. The underwater image enhancement method according to claim 3, characterized in that, Step S21 includes: By analyzing the local brightness histogram of the image, areas of uneven illumination are identified, and an adaptive correction algorithm is used to achieve illumination equalization.

5. The underwater image enhancement method according to claim 3, characterized in that, Step S23 includes: The enhancement weights are adaptively set according to the gradient magnitude distribution of each region in the image, and detail enhancement is applied to high-frequency edge regions, while noise suppression and moderate enhancement are applied to low-frequency smooth regions.

6. The underwater image enhancement method according to claim 1, characterized in that, The deep feature enhancement module is a self-constructed encoder-decoder network model, and step S3 includes: S31: Construct a five-layer encoder and a five-layer decoder structure. Each layer of the encoder contains a convolutional layer, a batch normalization layer, a ReLU activation function, and a downsampling layer. Each layer of the decoder contains a transposed convolutional layer, a batch normalization layer, and a ReLU activation function. S32: Introduce skip connections with Convolutional Block Attention Module (CBAM) between corresponding layers of the encoder and decoder; S33: Enhances texture detail information and improves edge preservation capabilities through multi-scale feature fusion and attention mechanisms.

7. The underwater image enhancement method according to claim 1, characterized in that, The adaptive color balance module is a self-constructed color correction model, and step S4 includes: S41: Convert the image to the CIELab color space, calculate the mean and standard deviation of the a and b channels, and quantify the degree of color shift; S42: Combining the gray world hypothesis and the perfect reflector hypothesis, dynamically generate color correction coefficients for each channel; S43: Based on the image brightness distribution characteristics, adaptive gain adjustment is performed on the RGB channels to restore natural color performance.

8. The underwater image enhancement method according to claim 1, characterized in that, Step S5 includes: Perform multi-scale feature reconstruction on the enhanced image, and use an adaptive filtering algorithm based on local noise variance and edge intensity detection to remove noise. The filtering algorithm adaptively selects Gaussian filtering or bilateral filtering strategies according to regional characteristics, and finally outputs the enhancement result through color space inverse transformation; further, based on the local noise variance σ n ² and edge intensity E, when σ n ²>T1 (noise variance threshold) and E<T2 (edge intensity threshold), Gaussian filtering is used. When σ n ²>T1 and E≥T2, bilateral filtering is used. When σ n ²≤T1, the original pixel value is retained; where T1∈[20,40] and T2∈[0.05,0.15].

9. The underwater image enhancement method according to claim 8, characterized in that, The specific implementation of the adaptive filtering algorithm is as follows: by calculating the noise variance and edge intensity of each region in the image, Gaussian filtering is applied to high-noise and non-edge regions, bilateral filtering is applied to high-noise edge regions, and weak filtering or preservation is applied to low-noise regions.

10. The underwater image enhancement method according to claim 1, characterized in that, The method is applicable to underwater engineering inspection, marine resource exploration, environmental perception of unmanned underwater vehicles, and underwater security monitoring applications.

Images