An underwater image enhancement method based on color channel interactive attention mechanism

Abstract

This invention relates to the field of image processing technology, and in particular to an underwater image enhancement method based on a color channel interactive attention mechanism. The method includes: separating the color channels of the original underwater image to obtain R, G, and B single-channel images; extracting sparse features for each channel using a sparse feature estimation module; achieving cross-channel feature interaction and information fusion using a channel interactive attention module; generating fusion parameters through an environment perception module to optimize the interactive feature fusion effect; reconstructing the fused features into corresponding channel images; and stitching them together to obtain the enhanced underwater image. This invention effectively improves the color balance, structural consistency, and detail preservation of underwater images, and is suitable for underwater vision applications such as underwater robots and ocean exploration.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to an underwater image enhancement method based on a channel interaction attention mechanism, which falls under the category of underwater visual image enhancement technology. Background Technology

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Underwater image enhancement is crucial for marine exploration and underwater applications of biomimetic robotic fish. However, underwater images often suffer from color distortion, low contrast, and blurred details due to the absorption and scattering of light by water. Because different wavelengths of light attenuate to varying degrees in water, the performance of different color channels in the image differs significantly. Specifically, red light, with its longest wavelength, is most easily absorbed in water and is typically completely attenuated over short distances; while green and blue light, with shorter wavelengths, have stronger penetrating power and thus retain more scene information. This wavelength-dependent light attenuation characteristic results in underwater images often exhibiting a blue-green hue and a severe lack of red information, causing a strong color imbalance.

[0004] Existing underwater image enhancement methods mainly include those based on image enhancement, image restoration, and deep learning. Traditional image enhancement methods, such as histogram equalization and color correction, while possessing some interpretability, have limited performance in complex underwater scenes. Restoration methods based on physical models, such as dark channel priors and inverse underwater imaging models, improve image quality by constructing underwater imaging models, but their adaptability in practical applications is poor. In recent years, deep learning methods such as Water-Net, UWCNN, and U-Transformer have achieved good enhancement results through end-to-end learning strategies; however, most methods neglect the potential correlations between multiple color channels in underwater images, resulting in the problem of independent channel processing and failure to fully utilize the complementary information of multi-channel images. Different color channels still exhibit certain consistency in terms of edges, contours, and textures. If information sharing between channels can be achieved, stronger color channels can provide structural guidance when restoring degraded color channels, thereby further improving the overall image quality. Summary of the Invention

[0005] This invention starts with the reasons for underwater image enhancement schemes:

[0006] To address the aforementioned issues, this invention proposes an underwater image enhancement method based on a color channel interactive attention mechanism. The invention separates the original image into three single-color channels through a channel separation operation, and then obtains sparse features for each channel using a sparse feature estimation module. A channel interactive attention module is used to achieve cross-color channel information fusion, while an environment awareness module generates fusion parameters based on global features. The fused features are reconstructed into the corresponding image color channels and then stitched together to obtain the enhanced image. Finally, a loss function is used to improve the overall network model performance.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] An underwater image enhancement method based on a color channel interactive attention mechanism includes the following steps:

[0009] The input raw underwater image is subjected to color channel separation to obtain three single-channel color images: R, G, and B.

[0010] The sparse feature estimation module is used to extract sparse features from the three single-channel images of color R, G, and B respectively.

[0011] The extracted sparse features are input into the channel interaction attention module to establish feature dependencies between channels. By constructing Query, Key and Value feature vectors, the attention weights between channels are calculated, and feature interaction and information fusion between different channels are realized through the attention mechanism.

[0012] Based on global feature content, the interaction intensity between channels is adaptively adjusted through the environment perception module to generate fusion parameters and optimize the interaction feature fusion effect;

[0013] Convolution operations are used to reconstruct the corresponding channel images from the fused features, and the enhanced R, G, and B channel images are stitched together to obtain the enhanced underwater image.

[0014] Finally, the network model is trained and optimized based on a set of pre-defined loss functions to improve the underwater image enhancement performance.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] This invention considers the differences in physical attenuation characteristics of different color channels and estimates their salient features from different color channels through a sparse feature estimation module built based on a convolutional sparse coding model. A channel interaction attention module is constructed to establish feature interaction relationships between channels, enabling channels with stronger information preservation to guide the structural recovery of channels with weaker information preservation, thereby improving structural consistency and color balance. An environment perception module is constructed to adaptively adjust the interaction strength between channels based on the global content features of the input image, enhancing the model's adaptability to various underwater environments. Compared with existing underwater image enhancement algorithms, this invention effectively improves the quality of underwater image enhancement, achieving superior results in color correction and structural detail preservation, and providing a more interpretable and robust enhancement strategy for underwater vision systems. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0018] Figure 1 This is an overall flowchart of at least one embodiment of the present invention;

[0019] Figure 2 This is an architecture diagram of a sparse feature estimation module according to at least one embodiment of the present invention;

[0020] Figure 3 This is an architecture diagram of the channel interaction attention module according to at least one embodiment of the present invention;

[0021] Figure 4 This is an architecture diagram of an environmental perception module according to at least one embodiment of the present invention;

[0022] Figure 5 This is a comparison chart of the enhancement effects of this invention with other underwater enhancement algorithms. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] An underwater image enhancement method based on channel interaction attention mechanism includes the following steps:

[0027] S1: Perform color channel separation on the input raw underwater image to obtain three single-channel color images: R, G, and B.

[0028] S2: Use the sparse feature estimation module to extract sparse features from the three color single-channel images of R, G, and B respectively;

[0029] S3: Input the extracted sparse features into the channel interaction attention module to establish the feature dependency relationship between channels. By constructing Query, Key and Value feature vectors, calculate the attention weight between channels, and realize feature interaction and information fusion between different channels through the attention mechanism;

[0030] S4: Based on global feature content, the interaction intensity between channels is adaptively adjusted through the environment perception module to generate fusion parameters and optimize the interaction feature fusion effect;

[0031] S5: Use convolution operations to reconstruct the corresponding channel images from the fused features, and stitch the enhanced R, G, and B channel images together to obtain the enhanced underwater image;

[0032] S6: Finally, the network model is trained and optimized based on a preset multi-loss function to improve the underwater image enhancement performance.

[0033] Specifically, in step S1, the original underwater image is defined as... H and W represent the height and width of the image, respectively. Through color channel separation, three single color channels (R, G, and B) are obtained. Using three separate models to represent the three color channels, the entire process can be described by the following formula:

[0034]

[0035] in, It is a split operation based on color channels.

[0036] Specifically, the sparse feature estimation module described in step S2 is built based on a convolutional sparse coding model. Under sparsity constraints, the convolutional sparse coding model can extract meaningful structures from images and use convolutional sparse features. To express color channels K represents the number of convolutional filters; the sparse feature estimation module uses Regularization constraints To improve feature sparsity and further highlight salient features, a convolutional expansion module based on a learning iterative shrinking threshold algorithm is used to address this issue. The problem of regularized convolutional sparse coding is solved by iteratively updating multiple iterative modules in the sparse feature estimation module. This yields the most significant sparse features for the color channels.

[0037] Specifically, the implementation process of the sparse feature estimation module described in step S2 is as follows:

[0038] Step 101: Use convolutional sparse features To express color channels The mathematical expression is as follows:

[0039]

[0040] in, It is a learnable convolution operation;

[0041] Step 102: Use Regularization constraints To improve the sparsity of features and further highlight salient features, the following equation is used to estimate... :

[0042]

[0043] in, It is a regularization parameter used to control the strength of the regularization term;

[0044] Step 103: Construct an iterative module using the convolutional expansion module of the Learning Iterative Shrinking Thresholding Algorithm (LISTA) to solve... The problem of regularized convolutional sparse coding involves stacking multiple iterative modules for iterative updates. This yields significant sparse features, and the mathematical expression for the entire process is as follows:

[0045]

[0046] in, It is During the next iteration The estimate, , and It is The learnable convolutional layer corresponding to the next iteration, It is a soft threshold function, defined as:

[0047]

[0048] It is a symbolic function. It is a threshold. Make the absolute value less than Invalid input values ​​improve sparsity and highlight salient features in the input color channels. Different values ​​are used for each iteration. Different learnable convolutional layers can be used to improve estimation accuracy.

[0049] Specifically, in step S3, the channel interaction attention module performs convolution transformation on the sparse features of each color channel to obtain Query, Key, and Value feature vectors. Then, the feature vectors are reshaped into a form suitable for matrix multiplication. Matrix multiplication is used to calculate the attention weight matrix between different channels to obtain the interaction features between channels. The obtained interaction feature information is reshaped back to the original feature size, and the features of each channel are weighted and fused to obtain new features containing channel interaction information.

[0050] Specifically, the implementation process of the channel interaction attention module described in step S3 is as follows:

[0051] Step 201: Sparse features for each channel The convolution transformation is performed to obtain the Query, Key, and Value feature vectors. The calculation formula is as follows:

[0052]

[0053] in, , , It is the convolution transform function. , , Let be the number of channels after conversion. ;

[0054] Step 202: To calculate the attention weights between different channels, the feature vectors are reshaped into a form suitable for matrix multiplication:

[0055]

[0056] in, It is an operation that flattens out the spatial dimensions. After transposing, we get ;

[0057] Step 203: Calculate from channel To the passage The attention weight matrix is ​​calculated using the following formula:

[0058]

[0059] in, Indicates from the channel Transmitted to channel attention, It is the scaling factor;

[0060] Step 204: Use the attention matrix to obtain the inter-channel interaction features, as shown in the following formula:

[0061]

[0062] in, Indicates from the channel Transmitted to channel Feature information;

[0063] Step 205: Reshape the inter-channel interaction features back to their original feature size, using the following formula:

[0064]

[0065] Step 206: Fuse the inter-channel interaction features back into the original color channels, using the following formula:

[0066]

[0067] in, These are learnable fusion parameters used to control the degree of information exchange between channels.

[0068] Specifically, in step S4, the environment perception module concatenates the sparse features of the R, G, and B color channels and obtains global features through global average pooling. The global features are then input into two fully connected layers. The first layer performs feature dimensionality reduction and uses ReLU activation. The second layer maps the feature to a three-dimensional channel fusion intensity vector. The fusion intensity vector is normalized to the (0,1) interval using the Sigmoid activation function and used as the final fusion parameter to fuse the channel interaction information back into the input channel.

[0069] Specifically, the specific process of the environmental perception module described in step S4 is as follows:

[0070] Step 301: Aggregate the sparse features of the three color channels through channel-level joint operations, as shown in the following formula:

[0071]

[0072] Step 302: Obtain global features through global average pooling, as shown in the following formula:

[0073]

[0074] Step 303: Process global features through two consecutive fully connected network layers. The first fully connected network layer reduces the dimensionality of high-dimensional features and uses the ReLU activation function. The second layer maps intermediate features to a three-dimensional fusion intensity space. The entire process can be represented as:

[0075]

[0076] in, and It is a weight matrix. and It is a bias term. It is an intermediate layer dimension;

[0077] Step 304: Use the activation function to obtain the fusion parameters for generating three color channels based on environmental features, as shown in the following formula:

[0078]

[0079] in, It is a weight matrix. It is a bias term. It is the Sigmoid activation function, used to activate vectors. Calculate each of the three elements element by element and then combine the parameters. Mapped to the range (0, 1).

[0080] Specifically, in step S5, convolution operations are used to restore the sparse features that have fused information between color channels into enhanced single color channels. Using channel-level union operations, the three color single channels are combined. The images are aggregated to obtain the images enhanced by channel interaction attention. The mathematical expression for the entire process is as follows:

[0081]

[0082] in, It's a convolution operation.

[0083] Specifically, the loss function described in step S6 includes five loss terms: intensity similarity loss, texture similarity loss, structural similarity loss, channel consistency loss, and environment adaptation loss. These five loss terms constrain the algorithm to optimize the underwater image enhancement performance.

[0084] Intensity similarity loss is expressed as Used for constrained image enhancement Compared with reference image The pixel intensity similarity is expressed mathematically as follows:

[0085]

[0086] Texture similarity loss is expressed as Through the Sobel gradient operator The mathematical expression for constraining the edge and texture information of an image is as follows:

[0087]

[0088] Structural similarity loss is expressed as This is used to ensure the structural consistency between two images, and its mathematical expression is as follows:

[0089]

[0090] Channel consistency loss is expressed as This is used to constrain the structural consistency between color channels, and its mathematical expression is as follows:

[0091]

[0092] in, For channel With channel The weighting coefficients between these coefficients are determined based on the attenuation characteristics of various color channels in the underwater environment. The purpose is to allow stronger signal channels to influence weaker signal channels. and This indicates edge extraction, an operation performed using the Sobel gradient operator, designed to extract structural features from an image;

[0093] The environment adaptive loss is expressed as Including the loss of diversity With smoothness loss Used to adjust the distribution rationality and stability of channel fusion parameters, and to reduce diversity loss. To ensure that the channel fusion parameters have sufficient information dispersion and prevent parameter collapse or clustering, its mathematical expression is as follows:

[0094]

[0095] in, These are fusion parameters The higher the entropy value, the more uniform the information distribution and the greater the diversity of fusion parameters;

[0096] Smoothing loss It is used to suppress drastic fluctuations between the weights of adjacent channels, avoid overfitting the model to specific channels or causing channel interaction imbalances, and generate a more stable and natural fusion effect. Its mathematical definition is as follows:

[0097]

[0098] in, The indices of adjacent channels are represented in a circular manner to ensure that all adjacent channel pairs are included in the loss calculation, enabling environment-adaptive loss. The overall mathematical expression is as follows:

[0099]

[0100] in, and These are the weighting coefficients. After considering all the loss terms, the mathematical expression for the total loss function is:

[0101]

[0102] in, , , , and These are the weighting coefficients for different loss terms.

[0103] To verify the enhancement effect of this embodiment, this embodiment is compared with three other state-of-the-art deep learning-based methods: FUnIE-GAN, PUIE-NET, and UWNet.

[0104] Figure 5 The images show experimental results of enhancing underwater images with color casts using this embodiment and other algorithms. The first row of the figures compares the image enhancement effects from the LSUI dataset. The FUnIE-GAN model renders the image as grayish-white. PUIE-NET makes the image appear greenish. LiteEnhanceNet removes the green but introduces noise. This invention not only effectively removes the green and completes color correction but also produces an enhanced image with higher structural similarity to the real image. The second row shows a comparison of the image enhancement effects from the UIEBC dataset. FUnIE-GAN reduces image quality, PUIE-NET makes the image appear greenish, and LiteEnhanceNet does not remove the blue. This invention successfully corrects the color and generates images with more detailed structural features.

[0105] To ensure the reliability of the evaluation, this embodiment uses three objective indicators—PSNR, SSIM, and UIQM—for comparative analysis on the LSUI and UIEBC datasets. Specific data can be found in Table 1.

[0106] Table 1: Comparison of Objective Evaluation Indicators of the Invention with Other Underwater Image Enhancement Algorithms

[0107]

[0108] Among them, PSNR (Peak Signal-to-Noise Ratio) measures the difference between the enhanced image and the original image; SSIM (Structural Similarity Index) measures the structural similarity between the enhanced and original images; and UIQM (Underwater Image Quality Assessment) measures the color richness and balance of the image. Higher PSNR and SSIM values ​​indicate a higher degree of structural and texture similarity between the enhanced and original images, while a higher UIQM value indicates that the obtained image is more consistent with human visual perception. Table 1 shows that the method of this invention outperforms other methods in terms of objective evaluation metrics, proving that the method of this invention can effectively improve the texture details, color richness, and contrast of images, thus demonstrating the effectiveness of this invention in enhancing underwater image quality and perceptual fidelity.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention in any way. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and such modifications or substitutions all fall within the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image enhancement method based on a color channel interactive attention mechanism, characterized by: Includes the following steps: The input raw underwater image is subjected to color channel separation to obtain three single-channel color images: R, G, and B. The sparse feature estimation module is used to extract sparse features from the three single-channel images of color R, G, and B respectively. The extracted sparse features are input into the channel interaction attention module to establish feature dependencies between channels. By constructing Query, Key and Value feature vectors, the attention weights between channels are calculated, and feature interaction and information fusion between different channels are realized through the attention mechanism. Based on global feature content, the interaction intensity between channels is adaptively adjusted through the environment perception module to generate fusion parameters and optimize the interaction feature fusion effect; Convolution operations are used to reconstruct the corresponding channel images from the fused features, and the enhanced R, G, and B channel images are stitched together to obtain the enhanced underwater image. Finally, the entire network is trained and optimized based on a preset loss function to improve underwater image enhancement performance; The specific process of the channel environment perception module is as follows: Step 101: Aggregate the sparse features of the three color channels through channel-level joint operations, as shown in the following formula: ； Step 102: Obtain global features through global average pooling, as shown in the following formula: ； Step 103: Global features are processed through two consecutive fully connected network layers. The first fully connected network layer reduces the dimensionality of high-dimensional features and uses the ReLU activation function. The second layer maps intermediate features to a three-dimensional fusion intensity space. The entire process can be represented as follows: ； in, and It is a weight matrix. and It is a bias term. It is an intermediate layer dimension; Step 104: Use an activation function to obtain the fusion parameters for generating three color channels based on environmental features, as shown in the following formula: ； in, It is a weight matrix. It is a bias term. It is the Sigmoid activation function, used to activate vectors. Calculate each of the three elements element by element and then combine the parameters. Mapped to the range (0, 1).

2. The underwater image enhancement method based on color channel interactive attention mechanism as described in claim 1, characterized in that: Define the original underwater image as H and W represent the height and width of the image, respectively. Through color channel separation, three single color channels (R, G, and B) are obtained. Three color channels are represented using three separate models.

3. The underwater image enhancement method based on color channel interactive attention mechanism as described in claim 1, characterized in that: The sparse feature estimation module is built on a convolutional sparse coding model. Under sparsity constraints, the convolutional sparse coding model can extract meaningful structures from images and use convolutional sparse features. To express color channels K represents the number of convolutional filters; Sparse feature estimation module uses Regularization constraints To improve feature sparsity and further highlight salient features, a convolutional expansion module based on a learning iterative shrinking threshold algorithm is used to address this issue. The problem of regularized convolutional sparse coding is solved by iteratively updating multiple iterative modules in the sparse feature estimation module. This yields the most significant sparse features for the color channels.

4. The underwater image enhancement method based on color channel interactive attention mechanism as described in claim 1, characterized in that: The channel interaction attention module performs convolution transformation on the sparse features of each color channel to obtain Query, Key and Value feature vectors. The feature vectors are then reshaped into a form suitable for matrix multiplication. Matrix multiplication is used to calculate the attention weight matrix between different channels. Based on the attention weights, the features of each channel are weighted and fused to obtain new features containing channel interaction information.

5. The underwater image enhancement method based on color channel interactive attention mechanism as described in claim 1, characterized in that: The specific process of the channel interaction attention module is as follows: Step 201: Perform convolution transformation on the sparse features of each channel to obtain the Query, Key, and Value feature vectors. The calculation formula is as follows: ； in, , , It is the convolution transform function. , , Let be the number of channels after conversion. ; Step 202: To calculate the attention weights between different channels, the feature vectors are reshaped into a form suitable for matrix multiplication: ； in, It is an operation that flattens out the spatial dimensions, After transposing, we get ; Step 203: Calculate from channel To the passage The attention weight matrix is ​​calculated using the following formula: ； in, Indicates from the channel Transmitted to channel attention, It is the scaling factor; Step 204: Use the attention matrix to obtain the inter-channel interaction features, as shown in the following formula: ； in, Indicates from the channel Transmitted to channel Feature information; Step 205: Reshape the inter-channel interaction features back to their original feature size, using the following formula: ； Step 206: Fuse the inter-channel interaction features back into the original color channels, using the following formula: ； in, These are learnable fusion parameters that control the degree of information exchange between channels.

6. The underwater image enhancement method based on color channel interactive attention mechanism as described in claim 1, characterized in that: The environment perception module concatenates the sparse features of the R, G, and B color channels, and performs global average pooling on the concatenated features to extract global features. The global features are then input into two fully connected layers. The first layer performs feature dimensionality reduction and ReLU activation, while the second layer maps the features to a three-dimensional channel fusion intensity vector. The fusion intensity vector is normalized to the (0,1) interval using the Sigmoid activation function and used as the final fusion parameter to fuse the channel interaction information back into the input channel.

7. The underwater image enhancement method based on color channel interactive attention mechanism as described in claim 1, characterized in that: Using convolution operations, sparse features that incorporate information from different color channels are restored into enhanced single-channel color models. Then, use channel-level joint operations to combine the three color single channels. The images are aggregated to obtain the images enhanced by channel interaction attention. .

8. The underwater image enhancement method based on color channel interactive attention mechanism as described in claim 1, characterized in that: The loss function includes intensity similarity loss, texture similarity loss, structural similarity loss, channel consistency loss, and environment adaptation loss. Intensity similarity loss constrains the pixel intensity similarity between the enhanced image and the reference image; texture similarity loss constrains the edge and texture information of the image; structural similarity loss is used to ensure the structural consistency between the two images; channel consistency loss is used to constrain the structural consistency between each color channel; and environment adaptation loss includes diversity loss and smoothness loss, which are used to adjust the rationality and stability of the distribution of channel fusion parameters.

Images