Underwater image enhancement method based on color channel interactive attention mechanism

Abstract

The invention relates to the technical field of image processing, in particular to an underwater image enhancement method based on a color channel interactive attention mechanism. Comprising the following steps: performing color channel separation on an original underwater image to obtain R, G and B single-channel images; extracting sparse features of each channel through a sparse feature estimation module; realizing cross-channel feature interaction and information fusion by using a channel interaction attention module; fusion parameters are generated through an environment sensing module, and the interaction feature fusion effect is optimized; and reconstructing the fused features into corresponding channel images, and splicing to obtain an enhanced underwater image. According to the method, the color balance, the structure consistency and the detail retention capability of the underwater image are effectively improved, and the method is suitable for underwater vision application scenes such as underwater robots and ocean exploration.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to an underwater image enhancement method based on a channel interaction attention mechanism, belonging to the category of underwater visual image enhancement technology. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Underwater image enhancement is crucial for ocean exploration and the underwater application of bionic fish. However, due to the absorption and scattering of light by water, underwater images often have problems such as color distortion, low contrast, and blurred details. Since light of different wavelengths attenuates differently in water, there are significant differences in the performance of different color channels in the image. Specifically, the red light has the longest wavelength and is most easily absorbed when propagating in water, usually being completely attenuated within a short distance; while the green and blue lights have shorter wavelengths and stronger penetration ability, thus retaining more scene information. This wavelength-dependent light attenuation characteristic causes underwater images to usually present a blue-green tone and have a serious lack of red information, resulting in a strong color imbalance.

[0004] Existing underwater image enhancement methods mainly include methods based on image enhancement, image restoration, and deep learning. Traditional image enhancement methods such as histogram equalization and color correction, although having a certain interpretability, have limited performance in complex underwater scenes. Physical model-based restoration methods, such as dark channel prior and inverse underwater imaging model, improve the image quality by constructing an underwater imaging model, but have poor adaptability in practical applications. In recent years, deep learning methods such as Water-Net, UWCNN, and U-Transformer, etc., have achieved good enhancement effects relying on end-to-end learning strategies. However, most methods ignore the potential correlation between multiple color channels in underwater images and have the problem of independent channel processing, failing to fully utilize the complementary information of multi-channel images. There is still a certain consistency between different color channels in terms of edges, contours, textures, etc. If information sharing between channels can be achieved, stronger color channels can provide structural guidance when restoring degraded color channels, thereby further improving the overall image quality. Summary of the Invention

[0005] The present invention starts from the reasons for the underwater image enhancement solution:

[0006] To solve the above problems, the present invention proposes an underwater image enhancement method based on a color channel interaction attention mechanism. The present invention separates the original image into three color single channels through a channel separation operation, and then obtains the sparse features of different channels through a sparse feature estimation module. The channel interaction attention module is used to achieve cross-color channel information fusion, and at the same time, the environment perception module generates fusion parameters based on global features. The fused features are reconstructed into the corresponding image color channels and stitched together to obtain the enhanced image. Finally, a loss function is used to improve the performance of the overall network model.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] An underwater image enhancement method based on a color channel interaction attention mechanism, comprising the following steps:

[0009] Perform a color channel separation operation on the input original underwater image to obtain three color single channel images of R, G, and B;

[0010] Use a sparse feature estimation module to extract sparse features from the three color single channel images of R, G, and B respectively;

[0011] Input the extracted sparse features into the channel interaction attention module, establish the feature dependence relationship between channels, calculate the attention weights between channels by constructing Query, Key, and Value feature vectors, and realize the feature interaction and information fusion between different channels through the attention mechanism;

[0012] Based on the global feature content, adaptively adjust the interaction intensity between channels through the environment perception module to generate fusion parameters and optimize the fusion effect of the interaction features;

[0013] Use convolution operation to reconstruct the fused features into the corresponding channel images, and stitch the enhanced R, G, and B three-channel images to obtain the enhanced underwater image;

[0014] Finally, train and optimize the network model based on a preset multiple loss function to improve the underwater image enhancement performance.

[0015] Compared with the prior art, the beneficial effects of the present invention are:

[0016] The present invention takes into account the differences in the physical attenuation characteristics of different color channels, and estimates the significant features from different color channels through a sparse feature estimation module based on a convolutional sparse coding model. A channel interaction attention module is constructed to establish the feature interaction relationship between channels, enabling the channels with stronger information retention to guide the structural recovery of the channels with weaker information retention, thereby improving the structural consistency and color balance. An environment perception module is constructed to adaptively adjust the interaction intensity between channels according to the global content features of the input image, enhancing the adaptability of the model to various underwater environments. Compared with existing underwater image enhancement algorithms, the present invention effectively improves the quality of underwater image enhancement, achieving excellent results in color correction and structural detail preservation, and providing a more interpretable and robust enhancement strategy for underwater vision systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention, and the schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0018] Figure 1 is the overall flowchart of at least one embodiment of the present invention;

[0019] Figure 2 is the architecture diagram of the sparse feature estimation module of at least one embodiment of the present invention;

[0020] Figure 3 is the architecture diagram of the channel interaction attention module of at least one embodiment of the present invention;

[0021] Figure 4 is the architecture diagram of the environment perception module of at least one embodiment of the present invention;

[0022] Figure 5 is the comparison diagram of the enhancement effects between the present invention and other underwater enhancement algorithms. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0025] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] An underwater image enhancement method based on a channel interaction attention mechanism includes the following steps:

[0027] S1: Perform color channel separation operations on the input original underwater image to obtain three color single-channel images of R, G, and B;

[0028] S2: Use a sparse feature estimation module to extract sparse features from the three color single-channel images of R, G, and B respectively;

[0029] S3: Input the extracted sparse features into the channel interaction attention module to establish the feature dependence relationship between channels. By constructing Query, Key, and Value feature vectors, calculate the attention weights between channels, and realize the feature interaction and information fusion between different channels through the attention mechanism;

[0030] S4: Based on the global feature content, adaptively adjust the channel interaction intensity through the environmental perception module to generate fusion parameters and optimize the fusion effect of the interaction features;

[0031] S5: Use convolution operations to reconstruct the corresponding channel images from the fused features, and splice the enhanced R, G, and B three-channel images to obtain the enhanced underwater image;

[0032] S6: Finally, train and optimize the network model based on a preset multi-loss function to improve the underwater image enhancement performance.

[0033] Specifically, in step S1, the original underwater image is defined as , where H and W are the height and width of the image respectively. Through color channel separation operations, three color single-channels of R, G, and B are obtained , and three separate models are used to represent the three color channels. The whole process can be described by the following formula:

[0034]

[0035] Among them, is the splitting operation according to the color channel.

[0036] Specifically, the sparse feature estimation module described in step S2 is constructed based on a convolutional sparse coding model. Under the sparsity constraint condition, the convolutional sparse coding model can extract meaningful structures from images and use convolutional sparse features to represent color channels , where K represents the number of convolutional filters; the sparse feature estimation module uses regularization constraints to improve the sparsity of features, further highlighting significant features. A convolutional expansion module using the learned iterative shrinkage threshold algorithm is used to solve the regularized convolutional sparse coding problem. In the sparse feature estimation module, multiple iterative modules are stacked to continuously iteratively update to obtain the optimal significant sparse features of the color channels.

[0037] Specifically, the implementation process of the sparse feature estimation module described in step S2 is as follows:

[0038] Step 101: Use convolutional sparse features to represent color channels , and the mathematical expression is as follows:

[0039]

[0040] where is a learnable convolutional operation;

[0041] Step 102: Use regularization constraints to improve the sparsity of features, further highlighting significant features. Estimate by solving the following equation:

[0042]

[0043] where is the regularization parameter, used to control the strength of the regularization term;

[0044] Step 103: Use a convolutional expansion module of the learned iterative shrinkage threshold algorithm (LISTA) to construct an iterative module to solve the regularized convolutional sparse coding problem, stack multiple iterative modules to iteratively update to obtain significant sparse features. The mathematical expression of the whole process is as follows:

[0045]

[0046] where is the estimate of at the th iteration, , and is the learnable convolutional layer corresponding to the th iteration, is the soft threshold function, defined as:

[0047]

[0048] is the sign function, is the threshold, invalidates the input values with absolute values less than , improves sparsity and highlights the significant features in the input color channels. Different and different learnable convolutional layers are used for each iteration to improve the estimation accuracy.

[0049] Specifically, the channel interaction attention module described in step S3 performs a convolutional transformation on the sparse features of each color channel to obtain Query, Key, and Value feature vectors, then reshapes the feature vectors into a form suitable for matrix multiplication, calculates the attention weight matrix between different channels using matrix multiplication, obtains the channel interaction features, reshapes the obtained interaction feature information back to the original feature size, and performs weighted fusion on the features of each channel to obtain new features containing channel interaction information.

[0050] Specifically, the implementation process of the channel interaction attention module described in step S3 is as follows:

[0051] Step 201: Perform a convolutional transformation on the sparse features of each channel to obtain Query, Key, and Value feature vectors. The calculation formulas are as follows:

[0052]

[0053] where , , is the convolutional transformation function, , , is the number of channels after transformation. Let ;

[0054] Step 202: To calculate the attention weights between different channels, reshape the feature vectors into a form suitable for matrix multiplication:

[0055]

[0056] where is the operation of flattening the spatial dimension. After transposing , is obtained;

[0057] Step 203: Calculate the attention weight matrix from channel to channel , and the formula is as follows:

[0058]

[0059] where represents the attention transferred from channel to channel , and is the scaling factor;

[0060] Step 204: Use the attention matrix to obtain the inter-channel interaction features, and the formula is as follows:

[0061]

[0062] where represents the feature information transferred from channel to channel ;

[0063] Step 205: Reshape the inter-channel interaction features back to the original feature size, and the formula is as follows:

[0064]

[0065] Step 206: Fuse the inter-channel interaction features back into the original color channels, and the formula is as follows:

[0066]

[0067] where is the learnable fusion parameter used to control the degree of information exchange between channels.

[0068] Specifically, the environmental perception module described in step S4 splices the sparse features of the R, G, and B color channels, obtains the global features through global average pooling, inputs the global features into two fully connected layers in sequence. The first layer performs feature dimensionality reduction and uses ReLU activation, and the second layer maps to a three-dimensional channel fusion intensity vector. The fusion intensity vector is normalized to the interval (0, 1) through the Sigmoid activation function and used as the final fusion parameter to fuse the channel interaction information back into the input channels.

[0069] Specifically, the specific process of the environmental perception module described in step S4 is as follows:

[0070] Step 301: Aggregate the sparse features of the three color channels through channel-level joint operations, and the formula is as follows:

[0071]

[0072] Step 302: Obtain global features through global average pooling. The formula is as follows:

[0073]

[0074] Step 303: Process the global features through two consecutive fully connected network layers. The first fully connected network reduces the dimension of the high-dimensional features and uses the ReLU activation function. The second layer maps the intermediate features to the three-dimensional fusion intensity space. The whole process can be expressed as:

[0075]

[0076] where and are weight matrices, and are bias terms, is the intermediate layer dimension;

[0077] Step 304: Use the activation function to obtain the fusion parameters for generating three color channels based on the environmental features. The formula is as follows:

[0078]

[0079] where is the weight matrix, is the bias term, is the Sigmoid activation function, which is used to perform element-wise calculations on the three elements of the vector and map the fusion parameter to the range of (0, 1).

[0080] Specifically, in step S5, convolution operation is used to restore the sparse features that have fused the information between color channels into an enhanced color single channel , and channel-level joint operation is used to aggregate the three color single channels to obtain the image enhanced by channel interaction attention. The mathematical expression of the whole process is as follows:

[0081]

[0082] where is the convolution operation.

[0083] Specifically, the loss function described in step S6 includes five loss terms: intensity similarity loss, texture similarity loss, structural similarity loss, channel consistency loss, and environmental adaptability loss. The algorithm is constrained by these five loss terms to optimize the underwater image enhancement performance;

[0084] The intensity similarity loss is expressed as for constraining the enhanced image and the reference image The pixel intensity similarity is as follows:

[0085]

[0086] The texture similarity loss is expressed as and constrains the edge and texture information of the image through the Sobel gradient operator The mathematical expression is as follows:

[0087]

[0088] The structural similarity loss is expressed as for ensuring the structural consistency between two images. The mathematical expression is as follows:

[0089]

[0090] The channel consistency loss is expressed as for constraining the structural consistency between color channels. The mathematical expression is as follows:

[0091]

[0092] where is the weight coefficient between channel and channel This coefficient is determined according to the attenuation characteristics of various color channels in the underwater environment, aiming to let the stronger signal channel affect the structure of the weaker signal channel and represent edge extraction, which is performed using the Sobel gradient operator and aims to extract the structural features of the image;

[0093] The environment adaptation loss is expressed as including the diversity loss and the smoothness loss for adjusting the distribution rationality and stability of the channel fusion parameters. The diversity loss ensures that the channel fusion parameters have sufficient information dispersion to prevent parameter collapse or aggregation. Its mathematical expression is:

[0094]

[0095] where is the entropy of the fusion parameter The higher the entropy value, the more uniform the information distribution and the greater the diversity of the fusion parameters;

[0096] The smoothness loss It is used to suppress the drastic fluctuations between the weights of adjacent channels, avoid overfitting of the model to a specific channel or imbalance in channel interaction, and generate a more stable and natural fusion effect. Its mathematical definition is as follows:

[0097]

[0098] in, Represent the indices of adjacent channels in a cyclic manner, ensuring that all adjacent channel pairs are included in the loss calculation, environment adaptive loss The overall mathematical expression is as follows:

[0099]

[0100] in, and is the weight coefficient. After combining all the above loss terms, the total loss function mathematical expression is:

[0101]

[0102] in, , , , and are the weight coefficients of different loss terms.

[0103] To verify the enhancement effect of this embodiment, this embodiment is compared with three other most advanced deep learning-based methods, namely FUnIE-GAN, PUIE-NET, and UWNet.

[0104] Figure 5 The experimental effect diagram of the embodiment and other algorithms after enhancing the color-shifted underwater images is shown in the first row of the figure. The image enhancement effect comparison from the LSUI dataset is shown. The FUnIE-GAN model makes the image appear grayish white as a whole. PUIE-NET makes the overall tone of the image greenish. LiteEnhanceNet removes the green color, but introduces noise. The present invention not only removes the green color well and completes the color correction, but also the enhanced image has a higher structural similarity with the real image. The second row shows the image enhancement effect comparison from the UIEBC dataset. FUnIE-GAN reduces the image quality, PUIE-NET makes the overall tone of the image greenish, and LiteEnhanceNet does not remove the blue color. The present invention successfully corrects the color and generates an image with finer structural details.

[0105] In order to ensure the reliability of the evaluation, this embodiment uses three objective indicators, PSNR, SSIM, and UIQM, to perform comparative analysis on the LSUI and UIEBC datasets. For specific data, see Table 1.

[0106] Table 1: Comparison of objective evaluation indicators between the present invention and other underwater image enhancement algorithms

[0107]

[0108] Among them, PSNR is the peak signal-to-noise ratio, which is used to measure the difference between the enhanced image and the original image; SSIM is the structural similarity index, which is used to measure the structural similarity between the enhanced image and the original image; UIQM is the underwater image quality assessment, which is used to measure the color richness and balance of the image. Higher PSNR and SSIM values indicate a higher degree of structural and textural similarity between the enhanced image and the original image, and a higher UIQM value indicates that the obtained image is more in line with human visual perception. It can be seen from Table 1 that the method of the present invention is superior to other methods in objective evaluation indicators, proving that the method of the present invention can effectively improve the texture details, color richness and contrast of the image, and further proving the effectiveness of the present invention in enhancing the quality and perceptual fidelity of underwater images.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than imposing any form of limitation on the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements all fall within the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image enhancement method based on a color channel interactive attention mechanism, characterized in that: It includes the following steps: Perform color channel separation operation on the input original underwater image to obtain three color single-channel images of R, G, and B; Use the sparse feature estimation module to extract sparse features from the three color single-channel images of R, G, and B respectively; Input the extracted sparse features into the channel interaction attention module to establish the feature dependence relationship between channels. By constructing Query, Key, and Value feature vectors, calculate the attention weights between channels, and realize the feature interaction and information fusion between different channels through the attention mechanism; Based on the global feature content, adaptively adjust the interaction intensity between channels through the environment perception module to generate fusion parameters and optimize the fusion effect of interaction features; Use convolution operation to reconstruct the fused features into corresponding channel images, and splice the enhanced R, G, and B three-channel images to obtain the enhanced underwater image; Finally, train and optimize the overall network based on the preset loss function to improve the underwater image enhancement performance.

2. The underwater image enhancement method based on the color channel interactive attention mechanism according to claim 1, wherein: Define the original underwater image as , where H and W are the height and width of the image respectively. Through the color channel separation operation, three single-color channels of R, G, and B are obtained , and three separate models are used to represent the three color channels.

3. The underwater image enhancement method based on the color channel interactive attention mechanism according to claim 1, wherein: The sparse feature estimation module is constructed based on the convolutional sparse coding model. Under the sparse constraint condition, the convolutional sparse coding model can extract meaningful structures from images and use convolutional sparse features to represent color channels , where K represents the number of convolutional filters; The sparse feature estimation module uses regularization constraints , to improve the sparsity of features, further highlight significant features, and use a convolutional expansion module with the learning iterative shrinkage threshold algorithm to solve the regularized convolutional sparse coding problem. In the sparse feature estimation module, multiple iterative modules are stacked to continuously iteratively update to obtain the optimal significant sparse features for the color channels.

4. The underwater image enhancement method based on the color channel interactive attention mechanism according to claim 1, characterized in that: The channel interaction attention module performs convolution transformation on the sparse features of each color channel to obtain Query, Key, and Value feature vectors, reshapes the feature vectors into a form suitable for matrix multiplication, uses matrix multiplication to calculate the attention weight matrix between different channels, and performs weighted fusion on the features of each channel according to the attention weights to obtain new features containing channel interaction information.

5. An underwater image enhancement method based on a color channel interactive attention mechanism according to claim 1, characterized in that: The specific process of the channel interaction attention module is as follows: Step 101: Perform convolution transformation on the sparse features of each channel to obtain Query, Key, and Value feature vectors. The calculation formula is as follows: ； Among them, , , is a convolution transformation function, , , is the number of channels after conversion. Let ; Step 102: In order to calculate the attention weights between different channels, reshape the feature vectors into a form suitable for matrix multiplication: ； Among them, is an operation to flatten the spatial dimension, and after transposing we get ; Step 103: Calculate the attention weight matrix from channel to channel , and the formula is as follows: ； Among them, represents the attention transferred from channel to channel , and is the scaling factor; Step 104: Use the attention matrix to obtain the interaction features between channels. The formula is as follows: ； Among them, represents the characteristic information transmitted from channel to channel . Step 105: Reshape the interaction features between channels back to the original feature size. The formula is as follows: ； Step 106: Fuse the interaction features between channels back to the original color channels. The formula is as follows: ； Among them, is a learnable fusion parameter that controls the degree of information exchange between channels.

6. The underwater image enhancement method based on the color channel interactive attention mechanism according to claim 1, wherein: The environment perception module splices the sparse features of the R, G, and B color channels, performs global average pooling operation on the spliced features to extract global features; the global features are sequentially input into two fully connected layers. The first layer performs feature dimensionality reduction and uses ReLU activation, and the second layer maps to a three-dimensional channel fusion intensity vector. The fusion intensity vector is normalized to the interval (0,1) through the Sigmoid activation function and used as the final fusion parameter to fuse the channel interaction information back into the input channels.

7. An underwater image enhancement method based on a color channel interactive attention mechanism according to claim 1, characterized in that: The specific process of the environment perception module is as follows: Step 201: Aggregate the sparse features of the three color channels through channel-level joint operation. The formula is as follows: ； Step 202: Obtain global features through global average pooling. The formula is as follows: ； Step 203: Process the global features through two consecutive fully connected network layers. The first fully connected network reduces the dimension of the high-dimensional features and uses the ReLU activation function. The second layer maps the intermediate features to the three-dimensional fusion intensity space. The whole process can be expressed as: ； Among them, and are weight matrices, and are bias terms, is the dimension of the intermediate layer; Step 204: Use the activation function to obtain the fusion parameters of the three color channels generated based on the environmental features. The formula is as follows: ； Among them, is the weight matrix, is the bias term, is the Sigmoid activation function, which is used to perform element-wise calculations on the three elements of the vector and map the fusion parameter to the range of (0, 1).

8. An underwater image enhancement method based on a color channel interactive attention mechanism according to claim 1, characterized in that: Using convolution operations, restore the sparse features that integrate information between color channels into an enhanced single color channel , and then use channel-level joint operations to aggregate the three single color channels to obtain an image enhanced by channel interaction attention .

9. The underwater image enhancement method based on the color channel interactive attention mechanism according to claim 1, characterized in that: The loss function includes intensity similarity loss, texture similarity loss, structural similarity loss, channel consistency loss, and environment adaptation loss. The intensity similarity loss constrains the pixel intensity similarity between the enhanced image and the reference image; the texture similarity loss constrains the edge and texture information of the image; the structural similarity loss is used to ensure the structural consistency between the two images; the channel consistency loss is used to constrain the structural consistency between color channels; the environment adaptation loss includes diversity loss and smoothness loss, which are used to adjust the rationality and stability of the distribution of channel fusion parameters.

Images