An Adaptive Underwater Image Enhancement Method

Abstract

The application discloses a water area adaptive underwater image enhancement method, which comprises the following steps: 1) based on a feature extraction method of multiple groups of convolution and sparse grid attention, an input underwater image is encoded to obtain encoded features of local details and global color contour information; 2) the domain difference of the underwater environment is constructed as a water area classification problem, and the difference information between different water areas is used to generate water area sensitive features to realize image enhancement of different water areas; 3) based on the enhanced image generated by the water area sensitive features, an aggregation loss and a multi-stage training strategy are used to constrain the enhanced result to realize high-quality underwater image enhancement. The water area adaptive method disclosed by the application fully utilizes the domain difference between different water areas and the difference of local and global information, and realizes adaptive and high-quality underwater image enhancement.

Description

Technical Field

[0001] This invention relates to the field of underwater image enhancement technology, and more specifically to an adaptive underwater image enhancement method for water areas. Background Technology

[0002] With economic development and increasing energy demand, the development and utilization of underwater resources are becoming increasingly urgent. However, the unique characteristics of underwater imaging result in lower image quality, and the complexity of the underwater environment leads to diverse degradation phenomena in images, posing challenges to applications in environmental protection and energy development. Underwater image enhancement technology can provide more realistic and reliable underwater visual information by improving the clarity, contrast, and detail restoration of underwater images, thereby supporting underwater operations. Unlike traditional land image enhancement techniques, underwater image enhancement methods need to consider factors such as the underwater environment, color distortion, scattering, and noise effects to present the details and colors of the real scene. However, these factors vary in different aquatic environments, leading to various degradation phenomena in the acquired underwater images. Therefore, underwater image enhancement technology can be considered an image processing problem with inter-domain differences.

[0003] Underwater image enhancement methods can generally be divided into three types: physics-based methods, image-based methods, and learning-based methods. Physics-based methods establish physical models of underwater image degradation processes, such as light propagation, scattering attenuation, and color distortion. These models, along with prior knowledge of underwater optics, are used to invert the degradation process and obtain the original underwater image. These methods can readily understand and simulate underwater optical phenomena, providing a good theoretical foundation and interpretability for underwater image enhancement. However, physics-based methods often rely heavily on extensive prior knowledge and accurate estimation of environmental parameters. Furthermore, new physical models are required for new or complex underwater scenarios, severely limiting the flexibility and versatility of these methods in practical applications. Image-based methods, on the other hand, do not consider the imaging process and directly process the underwater image. They utilize simple image processing algorithms to enhance the underwater image, improving its visualization by directly adjusting pixel values. Image-based enhancement methods are characterized by high computational efficiency and ease of use. However, this method cannot handle illumination attenuation and scattering phenomena well in complex scenes, which may lead to inaccurate enhancement results or other image artifacts. Learning-based underwater image enhancement methods learn features and patterns from large amounts of underwater image data through model training to achieve image enhancement. These methods typically use deep learning techniques, such as convolutional neural networks (CNNs) and generative adversarial networks (GANs). Learning-based methods can extract features from underwater images through learning from large-scale data, thus better handling problems such as illumination attenuation, color distortion, and scattering. However, learning-based methods often cannot simultaneously recover global information and process local details, and they are usually unable to cope with the diversity of underwater image distribution caused by the diversity of water environments. While the above three methods can effectively improve underwater image quality, the robustness and generalization ability of the models are limited, and a single model cannot solve the diversity of underwater image degradation caused by inter-domain differences in the underwater environment.

[0004] The key to underwater image enhancement technology lies in how to effectively learn the mapping relationship from degraded images to clear images, how to utilize the inter-domain differences in the underwater environment to achieve adaptive underwater image enhancement in different waters, and how to use global color contours and local detail textures to ensure that the enhanced image can maintain a high degree of consistency with the real clear image to the greatest extent. Summary of the Invention

[0005] The technical problems to be solved by this invention are: overcoming the poor environmental adaptability of existing algorithms to degraded images in various underwater scenarios, and providing a water-adaptive underwater image enhancement method, which improves the model's sensitivity to water by utilizing inter-domain differences in the underwater environment and a water type classification network; overcoming the loss of details and poor denoising effect of mainstream learning-based algorithm-enhanced underwater images, and proposing a feature extraction method based on multiple sets of convolutions and sparse grid attention; and overcoming the instability of enhancement results of learning-based algorithms, improving the model's stability by utilizing skip connections, and proposing an aggregation loss function and training strategy.

[0006] The technical solution adopted in this invention is: an underwater image enhancement method that adapts to water areas, comprising the following steps:

[0007] Step (1): Feature extraction method based on multiple sets of convolution and sparse grid attention: The input image is pre-processed using convolution operation. For the input features, multiple sets of convolution are used to make the model focus on local details, enhance the local feature model's ability to perceive layout features, and use sparse grid attention to model global information, preserve global color contours, and use max pooling to downsample features in spatial dimension to obtain encoded features. A method for extracting local features using multiple sets of convolution is proposed. Sparse grid attention is used to efficiently model global information, enhance the model's ability to perceive global overall content and local detail information, improve computational efficiency while maintaining high efficiency, and reduce unnecessary computational operations.

[0008] Step (2), Water Adaptation Method Based on Inter-Domain Differences in Underwater Environment: By encoding features in step (1), and using the classification network and cross-entropy loss function constraints, the water type of the input image is obtained. Water-related features are extracted from the water type classification network. Combined with the encoded features, a fusion strategy of spatial and channel dimensions is used to obtain water-sensitive features. The water information of the input image is extracted by utilizing the inter-domain differences of water and the water type classification network. A method for constructing water-sensitive features is proposed. These sensitive features can effectively guide the encoder to reconstruct underwater images in specific water environments, enabling the enhanced image to more accurately restore the details, color, and texture features of the real water environment, and ensuring that the enhancement result and the corresponding water have adaptability and consistency.

[0009] Step (3) Image enhancement based on water-sensitive features and aggregation loss function: Using the water-sensitive features obtained in step (2), the feature extraction method proposed in step (1) is used, combined with Bicubic algorithm interpolation to achieve feature upsampling in the spatial dimension, the aggregation loss function is used to constrain the enhancement results, and a multi-stage training strategy is used to enhance the robustness of the model; the water-sensitive features constructed in step (2) are used as the input of the decoder, and feature information is extracted by using the feature extraction method described in step (1). A loss function based on squared difference loss L2, color loss L2, and gradient loss L2 is proposed. Grad The combined aggregation loss is used to measure the difference between the enhanced image and the clear image. The squared difference loss is used to make the enhanced image closer to the clear image at the pixel level. The color loss is used to avoid unnatural or distorted color effects. The gradient loss is used to help the image maintain its structure and texture.

[0010] Further details of step (1) are as follows:

[0011] For the input degraded underwater image, two convolutional layers are first used to perform preliminary processing and feature extraction of the input data, and different methods are used to focus on local and global information in the features;

[0012] For local information in the features, a multi-group convolution-based local feature extraction method is used. The input features are evenly divided into multiple groups along the channel dimension, with each group containing the same number of input channels. The convolution kernels are divided into multiple groups according to the number of output channels, with each group containing a convolution kernel with the same number of channels. For each group of data, the corresponding convolution kernel is used for convolution operation. For the i-th layer in the encoding network, the multi-group convolution is represented as follows:

[0013]

[0014] Where MGC(·) represents a multi-group convolution operation, consisting of multiple single-group convolution operations G. g (·) Composed of D i-1 ∈R C×H×W The input for the current layer is given by g, where g is the number of groups divided along the channel dimension. Input feature Z i A continuous subset of g features, where Concat(·) concatenates g groups of features along the channel dimension, using batch regularization and ReLU activation function to prevent gradient vanishing and gradient exploding. P (·) represents a point convolution operation;

[0015] For global information in the features, a sparse grid attention feature extraction method is used to obtain the features after multiple convolution operations. As input features for this stage, the feature grid is divided into blocks using a fixed-size (G,G) grid. Obtain window features with global information, and perform a multi-head attention operation within that window to acquire the global information. The multi-head attention operation is defined as follows:

[0016]

[0017]

[0018] in, T represents transpose manipulation, and h represents the number of attention heads. f is a scaling factor pro For the nonlinear mapping function, Softmax(·) is the normalization operation. For the i-th layer in the encoding network, the complete Sparse Grid Attention (SGA) (·) operation flow is as follows:

[0019]

[0020]

[0021] Among them, f gb (·) represents the operation of first meshing and then block-forming the features, LN(·) is the regularization operation, FFN(·) is the feedforward layer, and f ubg (·) is the operation that returns the feature to its shape size (C,H,W), by modifying the feature. Perform convolutional mapping and max pooling operations to downsample and obtain feature D. i ;

[0022] The above steps are iterated four times to obtain a more comprehensive and richer coding feature D4.

[0023] In step one, the local information includes details and texture, while the global information includes color and outline.

[0024] Furthermore, the specific details of step (2) are as follows:

[0025] Using a water type classifier built with a convolutional neural network, the water area information of the encoded features obtained in step (1) is obtained, as shown below:

[0026] D(D4) = Maxpool(f D (D4))

[0027] Where D(·) is the water area classification network, f D (·) represents a non-linear mapping, and Maxpool(·) is the max pooling operation. This classification network has I c , Two outputs;

[0028] I c It is a one-dimensional vector of length equal to the number of water bodies, used to predict water type. The cross-entropy loss function is used to measure the difference between the predicted result and the true label. The cross-entropy loss L... N (·,·) represents the following:

[0029]

[0030] Where C represents the correct water type and c represents the predicted water type, and when C = c, y c =1, otherwise y c =0, where N is the number of water types;

[0031] For the output of the intermediate layer of the water classification network, deconvolution is used to... Mapped to the same dimension as Z4, the two are concatenated in the channel dimension. The concatenated features are then processed by point convolution in the spatial dimension to generate water-sensitive features. This process F(·,·) is represented as follows:

[0032]

[0033] Where F(·,·) is the mapping process for generating water-sensitive features, ConvT(·) is the deconvolution operation, and the generated water-sensitive features I are used to guide the encoding network to reconstruct underwater images in the corresponding water areas.

[0034] Furthermore, the specific details of step (3) are as follows:

[0035] A clear underwater image is reconstructed using a decoding network symmetrical to the encoding network. In addition to decoding information using the feature extraction method proposed in step (1), bilinear interpolation is used to upsample the features. Skip connections are used to concatenate the input of the decoding stage and the output of the corresponding layer in the encoding stage in the channel dimension to avoid information loss. For the i-th layer in the decoding stage, the decoding process is as follows:

[0036] E′ i =SGA(MGC(Concat(E i-1 D 4-i )))

[0037] For output feature D′ i Perform bilinear interpolation upsampling to obtain the current decoding layer output E. i ;

[0038] Repeat the above operation four times to obtain feature E4. Use convolution mapping to obtain an enhanced underwater image Y with 3 channels and the same width and height as the input image. Use aggregation loss to constrain the enhancement of the image, with aggregation loss L. Total(·,·)as follows:

[0039] L Total (Y,Y g )=αL2(Y,Y g )+βL Grad (Y,Y g )+γL color (Y,Y g )

[0040] Among them, Y g Clear the image separately, α, β and γ are weight terms, L2(Y,Y) g )=‖Y g -Y‖2 represents the reconstruction loss. For gradient loss, ΔR, ΔG, and ΔB represent the enhanced image Y and the sharpened image Y, respectively. g The difference between the various color channels in the RGB color space

[0041] The principle of this invention is as follows:

[0042] (1) The feature extraction method based on multiple sets of convolution and sparse grid attention proposed in this invention uses multiple sets of convolution to extract local information, retaining details while removing noise, and uses sparse grid attention to extract global information, removing water mist color while enhancing contours, realizing mutual optimization of different image contents, reducing the computational parameters of the model, and effectively helping the model learn the mapping relationship between degraded parts and clear images in underwater images.

[0043] (2) The water area adaptive method based on the inter-domain differences of the underwater environment proposed in this invention utilizes the difference information of different water areas to generate water area sensitive features, effectively ensuring the accuracy of the enhancement results in different water areas, improving image quality and visualization effect, and realizing image enhancement for different water areas.

[0044] (3) The image enhancement proposed in this invention is based on water-sensitive feature induction and aggregation loss function. It uses water-sensitive features to guide the decoding network to reconstruct underwater images in a specified underwater scene, which effectively improves the consistency between the enhanced image and the water environment. It uses multiple loss functions to provide more feedback signals to the model, guides different content of the image, and proposes a multi-stage training method, which effectively improves the stability of the model and the consistency between the enhanced image and the clear image.

[0045] The advantages of this invention compared to the prior art are:

[0046] 1. This invention proposes a feature extraction method based on multiple convolutions and sparse grid attention, which exhibits excellent information preservation capabilities. On one hand, it focuses on local information through multiple convolution operations; on the other hand, it employs sparse grid attention to preserve global color contours. This effectively solves the problems of detail loss and color distortion, achieves mutual optimization between different types of information, and has fewer computational parameters and lower complexity.

[0047] 2. Compared with existing underwater image enhancement algorithms, the proposed water type classification-based adaptive method improves the generalization ability of the model by adopting corresponding enhancement strategies according to the type of water area, ensuring the consistency between the enhancement results and the degradation scene, and making the enhancement results more accurate and reliable.

[0048] 3. The aggregate loss and multi-stage training strategy proposed in this invention comprehensively consider multiple factors for optimization, providing richer feedback signals, avoiding overfitting and underfitting problems, and better balancing various factors during the optimization process. By progressively optimizing each sub-task, the training at each stage becomes more focused and effective. This strategy helps accelerate the model's convergence speed and learning efficiency, improving the algorithm's stability and reliability. Attached Figure Description

[0049] Figure 1 The flowchart below shows the underwater image enhancement method for water area adaptation described in this invention:

[0050] Figure 2 The image shows a comparison of the enhancement effects of type 1 on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement effects).

[0051] Figure 3 The image shows a comparison of the enhancement effects of type 3 on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement effects).

[0052] Figure 4 The image shows a comparison of the enhancement effects of type 5 on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement effects).

[0053] Figure 5 The image shows a comparison of the enhancement effects of type 7 on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement effects).

[0054] Figure 6The image shows a comparison of the enhancement effects of type 9 on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement effects).

[0055] Figure 7 The image shows a comparison of the enhancement effects of type I on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement effects).

[0056] Figure 8 The image shows a comparison of the augmentation effects of type II on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better augmentation).

[0057] Figure 9 The image shows a comparison of the augmentation effects of type III on a pairwise artificially synthesized underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better augmentation).

[0058] Figure 10 The image shows a comparison of the enhancement effects on the paired real underwater dataset UIEB (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement).

[0059] Figure 11 The image shows a comparison of the enhancement effect on the EUVP real underwater dataset (the top left corner shows the PSNR and SSIM values, with larger values ​​indicating better enhancement).

[0060] Figure 12 The image shows a comparison of the enhancement effects on the paired real underwater dataset UFO-120 (the top left corner shows the PSNR and SSIM values; the higher the values, the better the enhancement effect).

[0061] Figure 13 The image shows a comparison of enhancement effects on the unpaired real-world underwater dataset UCCS, with a blue hue (the top left corner shows the UIQM value; a higher UIQM value indicates better enhancement).

[0062] Figure 14 The image shows a comparison of enhancement effects on the unpaired real underwater dataset UCCS, with a green hue (the top left corner shows the UIQM value; a higher UIQM value indicates better enhancement).

[0063] Figure 15 The image shows a comparison of enhancement effects on the unpaired real underwater dataset UCCS, with a blue-green hue (the top left corner shows the UIQM value; a higher UIQM value indicates better enhancement).

[0064] Figure 16The image shows a comparison of the enhancement effect on the unpaired real underwater dataset UCCS with a degradation level of A (the top left corner shows the UIQM value, and the larger the UIQM value, the better the enhancement effect).

[0065] Figure 17 The image shows a comparison of the enhancement effect on the unpaired real underwater dataset UCCS with a degradation level of B (the top left corner shows the UIQM value, and the larger the UIQM value, the better the enhancement effect).

[0066] Figure 18 The image shows a comparison of the enhancement effect on the unpaired real underwater dataset UCCS with a degradation level of C (the top left corner shows the UIQM value, and the larger the UIQM value, the better the enhancement effect).

[0067] Figure 19 The image shows a comparison of the enhancement effect on the unpaired real underwater dataset UCCS with a degradation level of D (the top left corner shows the UIQM value, and the larger the UIQM value, the better the enhancement effect).

[0068] Figure 20 The image shows a comparison of the enhancement effect on the unpaired real underwater dataset UCCS with a degradation level of E (the top left corner shows the UIQM value, and the larger the UIQM value, the better the enhancement effect).

[0069] Figure 21 This is a comparison chart of statistical results from paired synthetic underwater image datasets;

[0070] Figure 22 A comparative chart of statistical results from paired UIEB, EUVP, and UFO real underwater image datasets;

[0071] Figure 23 This is a comparison chart of statistical results from unpaired UIQS and UCCS real underwater image datasets. Detailed Implementation

[0072] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0073] Figure 1 The overall processing flow of the adaptive underwater image enhancement method for water areas is presented.

[0074] This invention provides an adaptive underwater image enhancement method for water areas, the main steps of which are described below:

[0075] Step (1): Feature extraction method based on multiple sets of convolutions and sparse grid attention:

[0076] For the input degraded underwater image, two convolutional layers are first used to perform preliminary processing and feature extraction on the input data. In order to make the generated encoded features have more comprehensive information, different methods are needed to focus on the local and global information in the features.

[0077] For local information such as details and texture in the features, a local feature extraction method based on multiple convolutions is used. The input features are evenly divided into multiple groups along the channel dimension, with each group containing the same number of input channels. The convolution kernels are divided into multiple groups according to the number of output channels, with each group containing a convolution kernel with the same number of channels. For each group of data, the corresponding convolution kernel is used for convolution operation. For the i-th layer in the encoding network, the multiple convolutions are represented as follows:

[0078]

[0079] Where MGC(·) represents a multi-group convolution operation, consisting of multiple single-group convolution operations G. g Composed of (·), used to learn local information of the corresponding channel features, D i-1 ∈R C×H×W The input for the current layer is given by g, where g is the number of groups divided along the channel dimension. Input feature Z i A continuous subset of g features, where Concat(·) concatenates g groups of features along the channel dimension, using batch regularization and ReLU activation function to prevent gradient vanishing and gradient exploding. P (·) represents point convolution operation, which promotes information interaction between different groups of features;

[0080] For global information such as color and contour in the features, a sparse mesh attention feature extraction method is used to obtain the features after multiple convolution operations. As input features for this stage, the feature grid is divided into blocks using a fixed-size (G,G) grid. Obtain window features with global information, and perform a multi-head attention operation within that window to acquire the global information. The multi-head attention operation is defined as follows:

[0081]

[0082]

[0083] in, T represents transpose manipulation, and h represents the number of attention heads. f is a scaling factor pro For the nonlinear mapping function, Softmax(·) is the normalization operation. For the i-th layer in the encoding network, the complete Sparse Grid Attention (SGA) (·) operation flow is as follows:

[0084]

[0085]

[0086] Among them, f gb (·) represents the operation of first meshing and then block-forming the features, LN(·) is the regularization operation, FFN(·) is the feedforward layer, and f ubg (·) is the operation that returns the feature to its shape size (C,H,W), by modifying the feature. Perform convolutional mapping and max pooling operations to downsample and obtain feature D. i ;

[0087] The above steps are iterated four times to obtain a more comprehensive and richer coding feature D4.

[0088] Step (2), Water Adaptive Method Based on Inter-Domain Differences in Underwater Environment:

[0089] Using a water type classifier built with a convolutional neural network, the water area information of the encoded features obtained in step (1) is obtained, as shown below:

[0090] D(D4) = Maxpool(f D (D4))

[0091] Where D(·) is the water area classification network, f D (·) represents a non-linear mapping, and Maxpool(·) is the max pooling operation. This classification network has I c , Two outputs;

[0092] I c It is a one-dimensional vector of length equal to the number of water bodies, used to predict water type. The cross-entropy loss function is used to measure the difference between the predicted result and the true label. The cross-entropy loss L... N (·,·) represents the following:

[0093]

[0094] Where C represents the correct water type and c represents the predicted water type, and when C = c, y c =1, otherwise y c =0, where N is the number of water types;

[0095] For the output of the intermediate layer of the water classification network, deconvolution is used to... Mapped to the same dimension as Z4, the two are concatenated in the channel dimension. The concatenated features are then processed by point convolution in the spatial dimension to generate water-sensitive features. This process F(·,·) is represented as follows:

[0096]

[0097] Where F(·,·) is the mapping process for generating water-sensitive features, ConvT(·) is the deconvolution operation, and the generated water-sensitive features I are used to guide the encoding network to reconstruct underwater images in the corresponding water areas;

[0098] Step (3): Image enhancement based on water-sensitive feature induction and aggregation loss function:

[0099] A clear underwater image is reconstructed using a decoding network symmetrical to the encoding network. In addition to decoding information using the feature extraction method proposed in step (1), bilinear interpolation is used to upsample the features. Skip connections are used to concatenate the input of the decoding stage and the output of the corresponding layer in the encoding stage in the channel dimension to avoid information loss. For the i-th layer in the decoding stage, the decoding process is as follows:

[0100] E′ i =SGA(MGC(Concat(E i-1 D 4-i )))

[0101] For output feature D′ i Perform bilinear interpolation upsampling to obtain the current decoding layer output E. i ;

[0102] Repeat the above operation four times to obtain feature E4. Use convolution mapping to obtain an enhanced underwater image Y with 3 channels and the same width and height as the input image. Use aggregation loss to constrain the enhancement of the image, with aggregation loss L. Total (·,·)as follows:

[0103] L Total (Y,Y g )=αL2(Y,Y g )+βL Grad (Y,Y g )+γL color (Y,Y g )

[0104] Among them, Y g Clear the image separately, α, β and γ are weight terms, L2(Y,Y) g )=‖Y g -Y‖2 is the reconstruction loss, used to recover basic image information. Gradient loss is used to preserve and enhance detailed textures. ΔR, ΔG, and ΔB represent the enhanced image Y and the sharpened image Y, respectively. g The difference between the various color channels in the RGB color space

[0105] The encoding / decoding structure employed in this invention induces the decoder to generate clear underwater images in a specified water scene by extracting water-sensitive features during the bottleneck period of the model. To this end, a multi-stage training strategy is proposed to progressively optimize each component. Firstly, it utilizes the aggregation loss L... Total The first step involves training the encoder and decoder until the structural similarity index (SSIM) of the enhanced and sharpened images reaches a certain threshold, at which point training stops. This phase aims to enable the encoder to generate meaningful encoded features. The second step trains the water classification network separately, using the encoded features as network input and employing the cross-entropy loss function L. N The (·,·) constraint determines the difference between the output and the actual water type, and the encoder is not updated via backpropagation until the accuracy of the water type prediction reaches a certain threshold, at which point training stops. The third step retrains the encoder and decoder, as well as the mapping process for water-sensitive features, using L... Total The (·,·) constraint enhances the difference between the results and the clear image until the end of training, thus improving the overall performance and robustness of the model.

[0106] This invention is applied to underwater image enhancement processing of publicly available underwater image datasets.

[0107] The multi-frequency dual-branch underwater image enhancement method proposed in this invention is implemented in Python and runs on a Windows 10 64-bit system. The hardware configuration used in the experiment was an Intel i7-8700 processor, 8GB of RAM, and an NVIDIA 2700 graphics card. Figures 2-9 The results comparing the invented algorithm with other advanced algorithms on artificially synthesized, paired underwater image data are presented. Figures 10-12 The results comparing the invented algorithm with other state-of-the-art algorithms on real, paired underwater image synthesis data are presented. Figures 13-20 The results comparing the invented algorithm with other state-of-the-art algorithms on real, unpaired underwater image data are presented. Figures 21-22 Statistical results are given on pairwise datasets. Figure 23 Statistical results are presented on unpaired datasets.

[0108] The technical contents not described in detail in this invention are common knowledge to those skilled in the art.

[0109] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A water-adaptive underwater image enhancement method, characterized in that: Comprising the following steps: Step (1), feature extraction method based on multiple groups of convolution and sparse grid attention: using convolution operation to preliminarily process the input image, for the input feature, using multiple groups of convolution to make the model focus on local details, enhance the perception ability of local feature model to layout features, use sparse grid attention to model global information, retain global color contour, use maximum pooling to realize the down sampling of feature in spatial dimension, obtain the encoding feature; The specific content of step (1) is as follows: For the input degraded underwater image, first use two convolution layers to realize the preliminary processing and feature extraction of the input data, and use different means to pay attention to the local and global information in the feature; For local information in features, a multi-group convolution-based local feature extraction method is used. The input features are evenly divided into multiple groups in the channel dimension, each group containing the same number of input channels. The convolution kernel is divided into multiple groups according to the output channel number, each group containing the same number of convolution kernels as the channel number. For each group of data, the corresponding convolution kernel is used for convolution operation. For the first layer in the encoding network, the multi-group convolution is represented as follows: wherein, is a multi-group convolution operation composed of multiple single-group convolution operations , is the current layer input, is the number of groups divided in the channel dimension, is the input feature 's consecutive subsets, is the group feature concatenated in the channel dimension , which prevents gradient vanishing and gradient explosion by using batch normalization and ReLU activation function, is a point convolution operation; For global information in the feature, a sparse grid attention feature extraction method is used to obtain the feature after a plurality of convolution operations As the input feature of this stage, the feature grid is binarized into a feature size of by using a fixed size grid , and a window feature with global information is obtained, and a multi-head attention operation is performed within the window to obtain global information, and the multi-head attention operation is defined as follows: in, , , , For transpose operation, For the number of attention heads, , As a scaling factor, It is a nonlinear mapping function. For the normalization operation, for the first... Layer, a complete sparse mesh attention The operation process is as follows: wherein, is a pre-griding operation on the features, is a regularization operation, is a feed-forward layer, is an operation that returns the features to the shape size by performing a convolutional mapping and a max-pooling operation on the features to down-sample the features ; The above steps are iterated four times to obtain more informative and comprehensive encoded features ; Step (2), water area adaptive method based on domain difference between underwater environment: through step (1) encoding feature, using classification network and cross entropy loss function constraint, obtain the water area type of input image, extract water area related features from water type classification network, combine the encoding features, use the fusion strategy of spatial and channel dimension, obtain the water area sensitive features; The specific content of step (2) is as follows: Using a water type classifier built by convolutional neural network, the water area information of the encoding feature obtained in step (1) is obtained, which is represented as follows: wherein, is a water body classification network, is a non-linear mapping, is a max-pooling operation, the classification network having two outputs; is a one-dimensional vector of the same length as the number of water areas, used to predict the water type, and the cross-entropy loss function is used to measure the difference between the prediction result and the true label, and the cross-entropy loss is expressed as follows: wherein, is the correct water type, is the predicted water type when , , otherwise , is the number of water types; For the output of the intermediate layer of the water classification network, deconvolution is used to... Mapping to and The two features are concatenated along the same channel dimension. The concatenated features are then processed by point convolution in the spatial dimension to generate water-sensitive features. This process... It is expressed as follows: wherein, is a mapping process to generate water-sensitive features, is a deconvolution operation, the generated water-sensitive features are used to guide the encoding network to reconstruct underwater images in the corresponding water area; Step (3), image enhancement based on water area sensitive feature induction and aggregation loss function: through the water area sensitive feature obtained in step (2), using the feature extraction method proposed in step (1), combining with Bicubic algorithm interpolation to realize the up sampling of feature in spatial dimension, using aggregation loss function constraint to enhance the result, and using multi-stage training strategy to enhance the robustness of the model; The specific content of step (3) is as follows: The decoding network reconstructs clear underwater images using and coding network symmetry, in addition to using the feature extraction method proposed in step (1) to decode information, using bilinear interpolation to realize feature up-sampling, using jump connection to concatenate the input of the decoding stage and the output of the corresponding layer of the coding stage in the channel dimension, which is used to avoid information loss. For the first layer of the decoding stage, the decoding process is as follows: on the output features performing bilinear difference up-sampling to obtain a current decoding layer output ; The above operation is repeated four times, and the features are obtained , and the enhanced underwater image with a channel number of 3, a width and a height equal to the input image is obtained by using the convolution mapping , the enhanced image is constrained by using the aggregation loss, and the aggregation loss is as follows: wherein, respectively clear images, , and are weight terms, is a reconstruction loss, is a gradient loss, , , and respectively enhanced images and clear images a difference between the respective color channels in the RGB color space, .

2. The water adaptive underwater image enhancement method according to claim 1, wherein: The local information in step one is detail, texture, and the global information is color, contour.

Images