Underwater Image Enhancement Method and System Combining Multi-Convolution Model and Attention Mechanism

Abstract

This application belongs to the field of image processing technology, and specifically relates to an underwater image enhancement method and system that combines a multi-convolution model with an attention mechanism. This technology is composed of the DICAM model and the improved Shallow-UWnet model. The DICAM model mainly performs color restoration by compensating for the uneven tone attenuation of underwater images, and can optimize problems such as underwater noise and fogging. The improved Shallow-UWnet model adds a max pooling layer and an Inc multi-convolution module on the basis of Shallow-UWnet. The max pooling layer effectively reduces network parameters and computational overhead through dimensionality reduction operations, not only improving the operation efficiency, but also enhancing the ability to extract key features and avoiding detail loss. Inc can extract features from different scales to enhance the feature capture ability of the model.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to an underwater image enhancement method and system that combines multiple convolutional models with an attention mechanism. Background Art

[0002] Underwater images provide crucial information for numerous engineering and research tasks, such as the condition monitoring of energy infrastructure, seabed terrain mapping, garbage detection, and the identification and classification of underwater objects and events.

[0003] However, due to the interference of factors such as water turbidity, light attenuation, and underwater particulate matter, original underwater images and videos often have serious visual distortion problems, manifested as uneven color deviation, blurred images, and deficiencies in quality, contrast, and brightness.

[0004] To address these challenges and improve the clarity of underwater images to better serve practical applications, researchers have proposed a variety of underwater image enhancement techniques. These techniques aim to obtain clearer images by optimizing contrast, adjusting color distribution, and eliminating blur effects. Early research mainly focused on contrast enhancement techniques, such as contrast-limited adaptive histogram equalization. However, the degree of degradation of underwater images is closely related to the distance between the object and the camera. For example, in the original image, the content in areas far from the camera is often more blurred than that in areas close to the camera. In the enhanced image, the visibility of these areas has been significantly improved. Therefore, traditional image enhancement methods have limited effects when dealing with underwater images. Summary of the Invention

[0005] This application addresses problems such as underwater image fogging noise, scale degradation, and non-uniform color attenuation, and proposes a method and system based on a neural network architecture that combines multiple convolutional models with an attention mechanism, achieving the effect of significantly improving the color, brightness, contrast, and details of underwater images. The technical solution is as follows:

[0006] An underwater image enhancement method that combines multiple convolutional models with an attention mechanism, comprising the following steps:

[0007] S1. In the DICAM model, the Inc module extracts features from different scales through multiple convolutional kernels or pooling operations of different sizes, and the CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features;

[0008] S2. Input the picture processed by the DICAM model into the improved Shallow-UWnet model. The improved Shallow-UWnet model includes multiple stacked ConvBlock and Inc modules, and a skip connection mechanism is introduced after each Inc module;

[0009] The ConvBlock is responsible for extracting local features such as details and textures. The Inc module extracts structural information at different scales through multi-branch convolutions, and the skip connection fuses the shallow input with the deep features.

[0010] Preferably, the DICAM network includes an Inc module, and the Inc module contains four parallel branch paths:

[0011] The first branch uses a 1×1 convolution followed by a 3×3 convolution to extract texture and edge information at a set scale;

[0012] The second branch uses a 5×5 convolution based on the 1×1 convolution, which can capture broader context information and helps to recover blurred regions;

[0013] The third branch suppresses local noise through 3×3 max pooling combined with 1×1 convolution;

[0014] The fourth branch directly uses a 1×1 convolution to retain the original local features and color information;

[0015] By concatenating the outputs of the four branches in the channel dimension, the Inc module achieves efficient fusion of multi-scale features;

[0016] The DICAM model applies the Inc module to the R, G, and B channels of the image respectively, which can process the specific degradation effects suffered by each channel and extract more distinguishable color features.

[0017] Preferably, the DICAM model includes a CAM module. The CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features. The specific method is as follows:

[0018] Step 1: For the m-th feature map F m extracted by the Inc module, first compress it using global average pooling to obtain a descriptive vector in the channel dimension , which represents the statistical average of the overall response of each channel. The formula is as follows:

[0019] ;

[0020] Step 2: Reduce the dimension of the GAP vector through a fully connected layer, and then increase the dimension through another fully connected layer. The formula is expressed as:

[0021] ;

[0022] where, and respectively represent the fully connected weight matrices for dimension reduction and dimension increase. It represents the Softsign activation function, and r is the compression ratio;

[0023] Step 3. Attention coefficient Coeff m is re-expanded to have the same shape as the original feature map, and feature adjustment is performed through per-channel multiplication:

[0024] ;

[0025] The CAM module can enhance the channel selectivity of the feature map and is inserted into multiple stages of the network.

[0026] Preferably, the ConvBlock is a lightweight convolutional block that processes the input feature map through two layers of convolution and the activation function ReLU, extracts local features such as texture and details, and prevents overfitting through Dropout. After being processed by the ConvBlock, the feature map enters the Inc module.

[0027] Preferably, the Inc module in the improved Shallow-UWnet model is a multi-branch convolutional neural network module that enhances the expression ability of the image processing model through feature extraction at different scales. The Inc module includes four parallel branches, each branch uses different convolutional operations, and the concatenation operation concatenates the outputs of each branch in the channel dimension.

[0028] Preferably, the first branch of the Inc module first reduces the number of channels of the input feature map through 1x1 convolution while retaining the key information of the input image. Subsequently, after passing through the 3x3 convolutional layer, local features are further extracted to capture a wider range of local context information;

[0029] The first branch uses LeakyReLU as its activation function, and the formula is:

[0030] ;

[0031] where x is the neuron input, that is, the value at each pixel position can be regarded as a neuron input, is a constant;

[0032] The second branch first compresses the number of channels through 1x1 convolution; then, a 5x5 convolution is used to obtain a larger receptive field. The second branch also uses LeakyReLU as the activation function;

[0033] The third branch first performs a 3x3 max pooling operation to reduce the spatial resolution through pooling, reduce the size of the feature map, and focus on the global information of the image. The pooled feature map further adjusts the number of channels through 1x1 convolution and increases the non-linearity through the LeakyReLU activation function to enhance the feature extraction ability of the network;

[0034] The fourth branch adjusts the number of channels through a 1x1 convolution operation without performing spatial feature expansion;

[0035] Preferably, the concatenation operation of the Inc module concatenates these four tensors in the channel dimension, and its mathematical representation is as follows:

[0036] ;

[0037] O 1 ,O 2 ,O 3 ,O 4 are the tensors of the four branches in the Inc module respectively, and the output tensor dimension of each branch is , where B represents the batch size, C represents the number of channels, H and W are the height and width respectively. Then the output tensor dimension after concatenation will become:

[0038] ;

[0039] The number of channels expands from the original C to 4C, thus integrating the different features extracted by multiple branches.

[0040] Preferably, the loss function of the improved Shallow-UWnet model:

[0041] Introduces pixel-level mean squared error loss to measure the difference between the network output image and the real clear image at the pixel level. The formula is:

[0042] ;

[0043] Where represents the model output image, is the corresponding clear image, and N is the total number of pixels;

[0044] To improve the quality of the image in terms of structure perception, the network introduces a second loss:

[0045] ;

[0046] I is the generated image, is the real image, and the features are extracted through the layer of the VGG network , is the real image at the layer of the VGG network; is the L2 norm, that is, calculating the Euclidean distance between the feature representations; is the weight coefficient of the l-th layer feature, used to balance the influence of features in different layers;

[0047] The final training loss function is composed of the addition of the above two parts, that is:

[0048] ;

[0049] , is the weight coefficient, .

[0050] An underwater image enhancement system combining a multi-convolution model with an attention mechanism, including a data acquisition unit, a data processing unit and a display unit;

[0051] Data acquisition unit: Obtain underwater image data;

[0052] Data processing unit: Process the complex distortion problems in underwater images by combining the DICAM model and the improved Shallow-UWnet model;

[0053] Display unit: Visual output of images.

[0054] Compared with the prior art, the beneficial effects of this application are as follows:

[0055] This technology is composed of the combination of the DICAM model and the improved Shallow-UWnet model. The DICAM model mainly performs color restoration by compensating for the inconsistent tone attenuation of underwater images. Compared with the original DICAM model, the improved Shallow-UWnet model can optimize problems such as underwater noise and fogging while further correcting colors. The improved Shallow-UWnet model is based on Shallow-UWnet and adds a max pooling layer and an Inc multi-convolution module. The max pooling layer effectively reduces network parameters and computational overhead through dimensionality reduction operations, not only improving the running efficiency but also enhancing the ability to extract key features and avoiding detail loss. Inc can extract features from different scales to enhance the feature capture ability of the model.

[0056] This technology effectively improves the ability to handle complex distortion problems in underwater images, such as color shift, contrast decline, detail blurring, etc., by combining the DICAM model and the improved Shallow-UWnet model. Brief Description of the Drawings

[0057] Figure 1 is the overall neural network architecture of this method;

[0058] Figure 2 is the Inc module;

[0059] Figure 3 is the CAM module;

[0060] Figure 4 is the effect comparison diagram. Specific implementation manner

[0061] The technical solution of the present application will be described in detail below through specific embodiments and the accompanying drawings. It should be understood that the specific features in the embodiments of the present application are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. The specific technical features can be combined with each other.

[0062] An underwater image enhancement method combining a multi-convolution model and an attention mechanism includes the following steps:

[0063] S1. In the DICAM model, the Inc module extracts features from different scales through multiple convolution kernels or pooling operations of different sizes, and the CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features and suppress unimportant features;

[0064] S2. Input the image processed by the DICAM model into the improved Shallow-UWnet model. The improved Shallow-UWnet model includes multiple stacked ConvBlock and Inc modules, and a skip connection mechanism is introduced after each Inc module;

[0065] The ConvBlock is responsible for extracting local features such as details and textures. The Inc module extracts structural information of different scales through multi-branch convolution, and the skip connection fuses the shallow input and the deep features.

[0066] I. DICAM model:

[0067] In the DICAM network, the Inc and CAM modules constitute the core architecture design of the entire model. They are responsible for extracting multi-scale spatial features and strengthening the information expression at the channel level respectively. They complement each other in function and make the model able to more effectively cope with complex distortion problems in underwater images, such as color shift, contrast decline, detail blurring, etc.

[0068] The design of the Inc module is derived from the classic Inception structure. Its main idea is to extract features from different scales through multiple convolutional kernels or pooling operations of different sizes, so as to enhance the model's feature capture ability. This module contains four parallel branch paths: the first branch uses a 1×1 convolution followed by a 3×3 convolution, which is good at extracting medium-scale texture and edge information; the second branch uses a 5×5 convolution based on a 1×1 convolution, has a larger receptive field, can capture broader context information, and helps to restore blurred areas; the third branch combines 3×3 max pooling with 1×1 convolution to enhance feature robustness and suppress local noise; while the fourth branch directly uses a 1×1 convolution to retain the original local features and color information. By concatenating the outputs of the four branches in the channel dimension, the Inc module achieves efficient fusion of multi-scale features. This structure not only improves the feature expression ability but also maintains controllability of computational efficiency, making it very suitable for deployment on devices with limited resources. It is worth mentioning that DICAM applies the Inc module to the R, G, and B channels of the image respectively, which is a clever design for the problem of uneven color distortion in underwater images and can process the specific degradation effects suffered by each channel separately to extract more distinguishable color features.

[0069] The basic idea of CAM is: learn an attention weight coefficient for each channel through the global statistical information of the feature map (i.e., the average value of each channel) to enhance important features and suppress unimportant features. Specifically, for the m-th feature map F m extracted by the Inc module, we first compress it using global average pooling (GAP) to obtain a description vector in the channel dimension. This vector represents the statistical average value of the overall response of each channel, and the formula is as follows:

[0070] ;

[0071] To improve the non-linear expression ability of attention learning, we borrow the "compression–excitation" mechanism of the SE module. First, reduce the dimension (compress) the GAP vector through a fully connected layer, and then increase the dimension (restore) it through another fully connected layer, where the compression ratio r is empirically set to 4. Formally expressed as:

[0072] ;

[0073] Among them, and The fully connected weight matrices representing dimensionality reduction and dimensionality increase respectively, and ψ represents the Softsign activation function, which is used to compress the output into the range of [−1, 1]. Different from ReLU and Sigmoid used in the SE module, Softsign can learn negative weights, which is particularly effective for color deviation correction in underwater images (such as overstrong blue / green channels). Through this mechanism, the model can effectively suppress noise channels and improve realism and color balance.

[0074] Finally, the attention coefficient Coeff m is re-expanded to have the same shape as the original feature map, and feature adjustment is performed through channel-wise multiplication:

[0075] ;

[0076] In this way, the CAM module completes the channel-selective enhancement of the feature map with extremely low computational overhead and can be flexibly inserted into multiple stages of the network, such as the initial color channel feature extraction and the final color correction stage.

[0077] The introduced CAM module further enhances the model's ability to focus on effective features. This module first performs global average pooling on the input feature map to generate a global description vector at the channel level, representing the overall response of each channel; then constructs a channel attention map through two fully connected layers and the Softsign activation function to evaluate the importance of each channel; finally, remaps this attention weight back to the original feature map to perform weighted enhancement on each channel. This channel attention mechanism can automatically learn to focus on important feature channels, thereby suppressing redundant or inefficient information and strengthening key features, especially in color correction.

[0078] In DICAM, CAM is used for attention enhancement of the R / G / B three-channel features respectively and also for re-weighting the overall fused features, enabling the model to dynamically select the most effective feature channels during the information integration process. Overall, the Inc module provides rich and hierarchical spatial features, while the CAM module guides the model to learn more reasonable feature channel weights from a global semantic perspective. The former focuses on the extraction of feature "breadth", and the latter focuses on the strengthening of feature "depth". This spatial-channel complementary structure enables DICAM to more comprehensively understand the complex degradation features in underwater images and finally output a clearer, more natural-color and detail-rich enhanced image.

[0079] II. Improved Shallow-UWnet Model:

[0080] In an underwater environment, due to the fact that the propagation of light in water is affected by various physical processes such as absorption, scattering, and reflection, the quality of the acquired images is generally poor. These phenomena are mainly manifested as problems such as color imbalance (usually showing an obvious blue-green tone), reduced contrast, blurred details, uneven brightness, and a decrease in the overall clarity of the image. To address these challenges, the DICAM model mainly focuses on correcting the color imbalance and brightness unevenness of underwater images. By introducing depth information and color attenuation characteristics, it effectively alleviates the problems of color cast and uneven brightness distribution in the images.

[0081] However, the DICAM model still has certain limitations in aspects such as noise reduction, defogging, and detail enhancement. Therefore, this technology has optimized the structure and improved the performance of the Shallow-UWnet model to make up for the deficiencies of DICAM in underwater image preprocessing. The optimized model not only improves the clarity and detail retention ability of the images but also enhances the robustness to underwater fog and noise. In addition, in terms of image color correction, this method further integrates multi-scale feature extraction and attention mechanism, enabling the model to better adapt to the illumination changes in different underwater scenes while maintaining natural color restoration, and finally achieving all-round enhancement and repair of underwater images.

[0082] The preprocessing part of the network includes a 3×3 convolutional layer for mapping the input three-channel RGB underwater image into 64-channel features. The purpose of this step is to transform the low-dimensional input into a higher-dimensional feature space to extract richer semantic information. Immediately afterwards, max pooling is used to perform downsampling on the feature map. In underwater images, due to the more serious blurring and degradation of distant targets, expanding the receptive field helps the model understand the overall background information of these blurred areas. Subsequently, upsampling is performed through learnable transposed convolution to restore the spatial dimension of the image and make it consistent with the input for the convenience of aligning the final output image.

[0083] Next, the feature extraction module is introduced. The feature extraction module is the core of the entire network and is composed of multiple stacked ConvBlock and Inc modules. ConvBlock is a lightweight convolutional block that contains multiple convolutional, ReLU activation, and Dropout operations. ReLU is a common activation function that sets the part of the input value less than 0 to 0, while for inputs greater than or equal to 0, ReLU directly retains the input value. In this way, ReLU introduces non-linearity in the neural network, enabling the network to learn more complex patterns. In addition, the calculation of ReLU is very simple, usually accelerating the training process and effectively alleviating the problem of gradient disappearance. The formula is:

[0084] f(x)=max(0,x);

[0085] where x is the input of the neuron.

[0086] Dropout is a regularization method that prevents model overfitting by randomly dropping a portion of neurons during training. In each training session, Dropout randomly selects neurons and sets their outputs to zero, forcing the network not to rely on a single neuron or feature but to learn more robust representations. In this way, Dropout can effectively improve the generalization ability of the model, especially when the data is insufficient or the model complexity is high. The ConvBlock mainly processes the input feature map through two layers of convolution and the ReLU activation function, extracts local features such as texture and details, and uses Dropout to prevent overfitting. After being processed by the ConvBlock, the feature map enters the Inc module.

[0087] The Inc module is a multi-branch convolutional neural network module designed to enhance the expressive power of image processing models through feature extraction at different scales. The design of this module is inspired by the Inception network structure, but it combines multiple branches to capture image features at different levels and scales, thereby improving the network's feature representation ability. The Inc module consists of four parallel branches, each branch using different convolutional operations, and finally concatenating the outputs of each branch in the channel dimension. The specific implementation and role of each branch in this module will be analyzed in detail below.

[0088] The first branch first reduces the number of channels of the input feature map through 1x1 convolution, thereby reducing the computational amount and the number of parameters. This operation can effectively reduce unnecessary computations while retaining the key information of the input image. Subsequently, through a 3x3 convolutional layer, local features are further extracted. The 3x3 convolution has a larger receptive field and can capture a wider range of local context information. The LeakyReLU formula is:

[0089] ;

[0090] where x is the neuron input, that is, the value at each pixel position can be regarded as a neuron input, is a very small constant set to 0.01. In this way, LeakyReLU gives a very small slope to negative inputs, usually a very small constant a, instead of directly outputting negative values as 0. Through this method, even when the input is negative, LeakyReLU will output a non-zero negative number, avoiding the problem of "dead neurons". Using the LeakyReLU activation function introduces a non-linear transformation to enhance the learning ability of the network. Through such a structure, the first branch can extract local features in a relatively small area.

[0091] The second branch is similar to the first branch. It also first compresses the number of channels through a 1x1 convolution. Then, it uses a 5x5 convolution instead of a 3x3 convolution to obtain a larger receptive field. Compared with the 3x3 convolution, the 5x5 convolution can capture a larger range of local features, especially when dealing with regions with large spatial variations. Like the first branch, the second branch also uses LeakyReLU as the activation function, enabling the network to learn more complex non-linear features.

[0092] The design of the third branch is different. It first performs a 3x3 max pooling operation. By pooling, it reduces the spatial resolution, decreases the size of the feature map, and focuses on the global information of the image. The pooling operation helps reduce the computational amount and enhances the translational invariance of the model, making the network more robust to small changes in the input. The pooled feature map is further adjusted in terms of the number of channels through a 1x1 convolution and the non-linearity is increased through the LeakyReLU activation function to enhance the feature extraction ability of the network.

[0093] The fourth branch is the simplest one. It only adjusts the number of channels through a 1x1 convolution operation and does not perform spatial feature expansion. The 1x1 convolution in this branch is mainly used to extract the relationships between channels while maintaining the dimension of the feature map. This branch has a relatively small computational amount and can effectively capture the subtle changes between the channels of the input feature map.

[0094] The concatenation operation mainly appears at the end of the Inc module and is used to fuse the feature maps extracted by the four different convolutional branches along the channel dimension. These four branches respectively use different convolutional kernels (1×1, 3×3, 5×5) and pooling structures to extract the local and global features of the input feature map at different scales. The output of each branch is transformed through the non-linear activation function LeakyReLU, and finally four tensors are obtained: O 1 ,O 2 ,O 3 ,O 4 . To integrate these multi-scale features into a unified feature map, the Inc module uses the concatenation operation, which essentially concatenates these four tensors along the channel dimension (dim = 1). Its mathematical representation is as follows:

[0095] ;

[0096] The dimension of the output tensor of each branch is , where B represents the batch size, C represents the number of channels, H and W are the height and width respectively. Then the dimension of the concatenated output tensor will become: This means that the concatenation operation expands the number of channels from the original C to 4C without changing the spatial dimensions (H and W), thus integrating the different features extracted by multiple branches. This operation not only enriches the feature representation but also enhances the model's perception ability of different structural information. Through this structured concatenation operation, the Inc module effectively fuses the information in different convolutional paths, enabling the network to simultaneously capture the details, edges, and context of larger regions in the image, providing more comprehensive feature support for subsequent image processing.

[0097] To address the problem of structural information loss in underwater images, the network introduces a skip connection mechanism after each Inc module. A core function of the skip connection is information retention. Neural networks usually extract low-level detail features such as edges, colors, and textures in the previous layers, and these features are particularly crucial in image enhancement tasks. Through the skip connection, these shallow detail information can be directly fused with the deep abstract semantic features, thereby enhancing the overall feature representation ability of the network, making the output image have both global perception and delicate local details. In addition, the skip connection helps to construct a smoother gradient propagation path, making the network more stable and converging more rapidly during the training process. Concatenate the original input image and the current feature map in the channel dimension. This design is very crucial as it can reintroduce shallow structural information (such as edges, textures, object contours) into the deep features, thereby improving the spatial consistency and detail performance of the image. The number of channels of this fused feature is 256, and then a 1×1 convolution is used to compress it back to 64 channels to control the computational complexity of the network while retaining the effective information after fusion.

[0098] The skip connection mechanism, together with the ConvBlock module and the Inc module, jointly constructs a multi-scale feature extraction architecture of "depth + width + cross-layer fusion". The ConvBlock is responsible for extracting local features such as details and textures, the Inc module extracts structural information of different scales through multi-branch convolution, and the skip connection fuses the shallow input and deep features, effectively retaining the details of the original image and alleviating the problem of gradient disappearance. The three work together, enabling the network to maintain details while having strong structural understanding and restoration ability in image enhancement tasks.

[0099] In the output stage, the network uses a 3×3 convolution to map the 64-channel high-dimensional feature back to a 3-channel image, outputting the final result map. The output image will have a more natural color distribution, clearer texture information, and more balanced brightness distribution, significantly superior to the original degraded image.

[0100] This model is an efficient neural network for underwater image enhancement tasks, with powerful feature modeling capabilities and image restoration capabilities. Its core advantages are: modular design, multi-scale feature extraction, skip information fusion, and a flexible depth control mechanism. These features enable the model to adapt to various complex underwater environments and provide a stable, clear, and high-quality image basis for underwater visual perception.

[0101] III. Loss Function:

[0102] When training a neural network for underwater image enhancement tasks, the model adopts a multi-loss function strategy, aiming to constrain the quality of the generated images from multiple levels. This strategy not only focuses on the restoration accuracy of the image at the pixel level but also emphasizes the clarity and structural consistency of the image at the perceptual level. By combining two loss functions with complementary characteristics, the model can generate enhanced images that are superior both subjectively and objectively.

[0103] First, the pixel-level mean squared error loss (MSELoss) is introduced during training to measure the difference between the network output image and the real clear image at the pixel level. The mean squared error calculates the sum of the squared differences between the two images at each pixel point, and its mathematical expression is:

[0104] ;

[0105] where I represents the model output image, and I ∗ is the corresponding clear image (GroundTruth), and N is the total number of pixels. This loss can ensure that the generated image is as close as possible to the real image at the basic levels such as color and brightness, but it usually cannot effectively express the differences in the structure and texture of the image.

[0106] To improve the quality of the image in terms of structural perception, the network introduces a second loss: the perceptual loss (PerceptualLoss). This loss is calculated based on the activation features of a certain convolutional layer in the pre-trained VGG-19 network. The perceptual loss constrains the network to retain edge clarity, texture structure, and spatial consistency as much as possible during the restoration process by comparing the distances between the enhanced image and the real image in the high-level semantic feature space. Specifically, the model inputs the enhanced image and the real image into the VGG network, extracts the feature maps at the specified convolutional layer, and calculates the Euclidean distance between these two feature maps as the perceptual loss. This method effectively guides the model to generate images that are more in line with human visual perception.

[0107] The pixel-level mean squared error loss is introduced to measure the difference between the network output image and the real clear image at the pixel level, and the formula is:

[0108] ;

[0109] where represents the model output image, is the corresponding clear image, and N is the total number of pixels;

[0110] To improve the quality of the image in terms of structure perception, the network introduces a second loss:

[0111] ;

[0112] I is the generated image, is the real image, and the features are extracted through the layer of the VGG network ; is the real image at the layer of the VGG network; is the L2 norm, that is, calculating the Euclidean distance between the feature representations; is the weight coefficient of the feature at the l-th layer, used to balance the influence of features in different layers;

[0113] The final training loss function is composed of the addition of the above two parts, that is:

[0114] ;

[0115] , are weight coefficients.

[0116] This combination strategy achieves a balance between pixel-level accuracy and semantic-level structure restoration, ensuring both the authenticity of image details and the improvement of the overall perceptual quality.

[0117] In the underwater image enhancement task, this multi-loss design is particularly important. Due to color drift, edge blurring, and detail loss in underwater images, a single MSE loss cannot restore the visual effect of the real underwater scene. By introducing the VGG perceptual loss, the network can effectively identify and restore the blurred object structures and textures in underwater images, making the enhanced images more natural and clear. This loss design provides strong training constraints for the images, significantly improving the enhancement ability and generalization performance of the network.

[0118] IV. Differences and connections between the Inc module in the DICAM network and the Inc module in the Shallow-UWnet model:

[0119] In this system design, although the DICAM model and the improved Shallow-UWnet model network each independently use Inc modules, the two are closely linked overall, forming a layered, synergistic feature extraction system. DICAM's Inc module preserves shallow image details through feature enhancement and passes them as feature input to the improved Shallow-UWnet model, serving as the original input to the improved Shallow-UWnet model and participating in feature concatenation through a skip connection mechanism. The Inc module in the improved Shallow-UWnet model further deepens this information in the mid- and high-level feature streams, thereby achieving continuous feature modeling from shallow to deep layers and from local to global. The Inc modules in both parts share the same structure, both extracting and fusing feature information from different receptive fields through multi-branch, multi-scale convolutions. Their core purpose is to fully capture the rich and complex features of underwater images.

[0120] In the DICAM part, the Inc module primarily performs preliminary feature enhancement on the input image. DICAM applies the Inc module independently to each color channel, extracting multi-scale detailed features from the raw pixel level. It then uses the Channel Attention (CAM) mechanism to enhance key information, effectively improving image quality before the image enters the backbone network. At this stage, the Inc module targets low-level, fine-grained image features, aiming to restore the image's underlying details and local structure.

[0121] In the improved Shallow-UWnet, the Inc module is deployed in the intermediate feature stream, acting on mid- and high-order feature maps that have undergone preliminary convolution processing. Instead of directly processing raw pixel information, the Inc module further refines and enhances semantic information and multi-scale texture relationships within the feature space. By continuously inserting Inc modules into the feature stream, combined with convolutional blocks (ConvBlocks) and feature concatenation, the network can gradually accumulate and fuse local and global information at different levels, supporting more accurate image reconstruction.

[0122] From an overall perspective, the Inc modules in DICAM and the improved Shallow-UWnet model actually form a relationship of echoing before and after and connecting up and down in function: the former focuses on the original image, emphasizing detail compensation and quality improvement; the latter focuses on abstract features and further deepens information modeling and context understanding. Using the Inc module continuously in two stages completes a comprehensive extraction and enhancement process from the low-level features of the input image (such as edges and color distributions) to the deep semantic features (such as object structures and regional consistency). Through this cross-level, multi-scale, and progressive feature modeling strategy, the network can model simultaneously at different spatial scales and semantic depths, greatly enhancing the ability to repair complex degradation phenomena in underwater images (such as light attenuation, color distortion, and detail loss). Using the Inc module continuously in two stages enables the network to cover the feature extraction requirements from shallow details to deep semantics, constructing a cross-level, multi-scale, and progressive feature processing chain.

[0123] IV. Experimental Design:

[0124] To comprehensively verify the effectiveness of the underwater image enhancement method proposed in this paper, the experiments were conducted under the Windows operating system, using the Python 3.7 programming environment, and a model was built based on the advanced deep learning framework PyTorch 1.12.1. The designed enhancement network structure is compact and logically clear, giving full play to the advantages of deep learning in feature extraction and nonlinear modeling. During the network training process, the Adam optimizer was selected as the parameter update strategy. This optimizer has the ability to adjust the learning rate adaptively, can effectively combine the first-order and second-order momentum information of the gradient, and enhances the stability of the model training process while improving the convergence efficiency.

[0125] All experiments were completed on a high-performance server equipped with an NVIDIA GeForce RTX3090 GPU (24GB of video memory), ensuring the efficient and stable processing of images. This hardware platform provides strong computing power support for the training and inference of the model on large-scale datasets. The test data is shown in Table 1.

[0126] Table 1 Image Enhancement Quality Evaluation of Different Methods Based on the EUVP Dataset

[0127] .

[0128] In terms of experimental parameter settings, the initial learning rate was set to 0.0002. The number of training samples in each batch was 16, balancing the computing resources and memory consumption, which helped to improve the training efficiency and image quality. The model was trained for 200 epochs to ensure that the results have good stability and reliability.

[0129] To comprehensively evaluate the performance and practical value of the proposed method, this paper adopts an image quality evaluation system that combines subjective and objective methods. In terms of objective evaluation, three widely recognized and commonly used indicators in the industry are selected to quantitatively analyze the performance of the enhanced images in various scenarios. These three indicators are as follows:

[0130] 1. Peak Signal-to-Noise Ratio (PSNR): It is a commonly used image quality evaluation index based on pixel accuracy. It mainly judges the enhancement effect by calculating the error between the enhanced image and the reference image (usually the original high-quality image). Its essence lies in evaluating the intensity of noise in the image. The higher the value, the smaller the difference between the enhanced image and the original image, that is, the less image distortion and the better the image quality. Therefore, the larger the PSNR, the more excellent the enhanced algorithm performs in retaining image details and structures. The following are the calculation formulas for the peak signal-to-noise ratio and the variance used:

[0131] ;

[0132] ;

[0133] Structural Similarity Index (SSIM): It is a similarity evaluation method from the perspective of image structure information. Different from traditional indicators based only on pixel errors, SSIM pays more attention to the matching of images in three aspects: brightness, contrast, and structural details, and tries to measure image quality from the perspective of human visual perception. A higher SSIM value usually means that the enhanced image has better consistency with the reference image in terms of texture, edges, and local structures, indicating that the method has strong advantages in detail retention and visual fidelity.

[0134] Underwater Image Quality Metric (UIQM): It is a comprehensive index designed specifically for evaluating the quality of underwater images and is widely used in related research. This index comprehensively evaluates image quality from three key dimensions: color fidelity, image contrast, and clarity. UIQM can effectively reflect the comprehensive ability of the enhancement algorithm to cope with underwater image degradation phenomena (such as light scattering and absorption). Usually, the higher the UIQM value, the more excellent the proposed enhancement method performs in restoring the true color of the image, enhancing visual contrast, and detail clarity, indicating that the method has stronger practicality and the ability to improve visual effects.

[0135] Figure 1This is the overall neural network architecture of the method, which is generally divided into two module methods. The first module method is the DICAM module, which aims to fully exploit the multi-scale spatial features contained in different channels (R, G, B) of the image. At the same time, it enhances the response ability of the key feature channels through the CAM module, thereby enhancing the expression effect and reconstruction ability of the image. The entire model takes a three-channel color image as input and splits it into three separate channels, R, G, and B, which are respectively fed into three feature extraction Inc modules with shared structures but independent parameters. The feature maps extracted from each channel will be further fed into the CAM module. After extracting and enhancing the features in the three color channels respectively, the model concatenates them in the channel dimension to form a fused feature map. This fused feature map will go through another Inc module to extract higher-level features and use the channel attention module again to enhance the expression ability. The final output part uses two convolutional operations to reduce the number of channels to 3 channels, and limits the output range through an activation function to generate a standard image format. The second module method is a deep convolutional neural network for underwater image enhancement, which is an improvement of the Shallow-UWnet model. Its design aims to restore clear and natural visual effects from degraded underwater images. The model structure is relatively more concise, facilitating analysis and rapid training, but still retains key enhancement mechanisms such as multi-scale feature extraction and skip connections. The input of the model is a three-channel color image. First, a convolutional layer is used to map the number of channels to 64 to form an initial feature map. Subsequently, a max-pooling operation is performed to extract more globally perceptive features. Then, upsampling is achieved through a learnable transposed convolutional layer to restore the feature map to the original resolution. An activation function is used in this process to enhance the non-linear expression ability. In the backbone part, the model constructs a stacked structure composed of multiple alternating multi-scale convolutional modules and Inc modules. After each Inception module, the model concatenates its output with the original input image to explicitly introduce the low-level information of the original image. This concatenation leads to a sharp increase in the number of channels. To control the computational overhead, the model reduces the number of channels to 64 through a 1×1 convolutional layer, retaining the key information while maintaining the computational efficiency. Finally, the model uses an output convolutional layer to restore the 64-channel feature map to a three-channel RGB image as the enhanced image output.

[0136] Figure 2It is the Inc module. The Inc module is a multi-branch convolutional structure built based on the Inception idea, aiming to extract multi-scale features of images. It processes the input through four parallel sub-branches with different receptive fields respectively, and then stitches these features together in the channel dimension, thereby enhancing the richness and diversity of feature representation. In this method's Inc module, the first branch uses a 1×1 convolution for dimensionality reduction followed by a 3×3 convolution, which is suitable for extracting medium-scale texture information; the second branch has a similar structure but uses a 5×5 convolution with a larger receptive field, which is beneficial for capturing a wider range of context features; the third branch first performs max pooling to extract smooth or background region information and then integrates the results through a 1×1 convolution; the fourth branch only uses a 1×1 convolution to quickly extract local responses. The LeakyReLU activation function is applied after each convolutional layer to introduce non-linearity and alleviate the "neuron death" problem of ReLU in the negative interval. The outputs of the four branches are finally stitched together to form an output tensor that fuses multi-scale features. This design not only improves the model's perception ability of image content at different scales but also effectively maintains the computational efficiency of the network, and is very suitable for tasks such as image enhancement that require high requirements for detail restoration and context understanding.

[0137] Figure 3 It is the CAM module. The CAM module is an attention mechanism for extracting the importance weights in the channel dimension, aiming to enhance the network's response ability to key feature channels and suppress irrelevant or redundant channel information by explicitly modeling the relationships between channels, thereby improving the image expression effect. Specifically, CAM first compresses the spatial information of each channel into a single value through global average pooling to form a channel description vector, representing the global semantics of the entire channel. Then, a fully connected network composed of two fully connected layers is used to model this channel description, with the Softsign activation function used in the middle to introduce non-linearity. The first linear layer is used for dimensionality reduction to capture a compact cross-channel representation; the second linear layer then increases the dimension back to the original number of channels and outputs the attention coefficients for each channel. Finally, the tensor formed by these attention coefficients will be multiplied with the original input feature map channel by channel through the broadcast mechanism to achieve channel weight adjustment. The entire process does not change the spatial size of the input but will significantly improve the information selectivity between channels, enabling the model to focus more on the feature channels that contribute more to image enhancement.

[0138] Figure 4This is a comparison of the effect diagrams of this method with the original image and other underwater image enhancement methods. The method of Module 1 is the DICAM model method, and the method of Module 2 is the Shallow-UWnet method. The method of Module 1 is mainly used to solve the restoration degree of color cast images in underwater images and is more suitable for underwater image restoration in deep water areas. However, it may not handle the blurring and grayish state of the image well during image enhancement. The image of Model 2 has good effects in terms of image structure and brightness, but it may introduce other color casts. The effect diagrams intuitively show that there has been a significant improvement in all five effect diagrams compared with the original image, and there are also significant differences among the five methods. The method of adaptive histogram equalization has a poor effect on restoring color cast images in underwater image enhancement, but it has good effects in improving image brightness and image details. The fast underwater image enhancement method has good improvements in both image brightness and color cast degree. In contrast, the method in this paper has better effects in removing color cast, blurring and occlusion, and improving image contrast, etc.

[0139] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of this application, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of this application.

Claims

1. A method for underwater image enhancement using a multi-convolutional model combined with an attention mechanism, characterized in that: The following steps are involved: S1. In the DICAM model, the Inc module extracts features from different scales using multiple convolution kernels or pooling operations of different sizes. The CAM module learns an attention weight coefficient for each channel based on the global statistical information of the feature map to enhance important features. S2. The image processed by the DICAM model is fed into the improved Shallow-UWnet model. The improved Shallow-UWnet model consists of multiple layers of stacked ConvBlock and Inc modules, with a skip connection mechanism introduced after each Inc module. ConvBlock is responsible for extracting local features, Inc module extracts structural information of different scales through multi-branch convolution, and skip connection fuses shallow input with deep features; The DICAM model includes the Inc module, which contains four parallel branch paths: The first branch uses 1×1 convolution followed by 3×3 convolution to extract texture and edge information of a set scale; The second branch uses 5×5 convolution on the basis of 1×1 convolution to capture contextual information; The third branch suppresses local noise by combining 3×3 maximum pooling with 1×1 convolution; The fourth branch directly uses 1×1 convolution to retain the original local features and color information; By splicing the outputs of four branches in the channel dimension, the Inc module achieves efficient fusion of multi-scale features; The DICAM model applies the Inc module to the R, G, and B channels of the image, which can process the degradation of each channel separately and extract recognizable color features. The DICAM model includes a CAM module. The CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features. The specific method is as follows: Step 1: For the mth feature map F extracted by the Inc module m , first use global average pooling to compress it and obtain the description vector on the channel dimension , which represents the statistical mean of the overall response of each channel, and the formula is as follows: ； Step 2: Reduce the dimension of the GAP vector through a fully connected layer and then increase the dimension through another fully connected layer. The formula is expressed as: ； in, and Represent the fully connected weight matrices of dimensionality reduction and dimensionality increase, represents the Softsign activation function, r is the compression ratio; Step 3: Attention Coefficient Coeff m It is re-expanded to the same shape as the original feature map and feature adjustment is performed by channel-by-channel multiplication: ； The CAM module can selectively enhance the channels of the feature map and is inserted into multiple stages of the network.

2. The underwater image enhancement method according to claim 1, characterized in that: ConvBlock is a lightweight convolution block that processes the input feature map through two layers of convolution and the activation function ReLU, extracts local features, and prevents overfitting through Dropout. After processing by ConvBlock, the feature map enters the Inc module.

3. The underwater image enhancement method using a multi-convolutional model combined with an attention mechanism according to claim 1, characterized in that: The Inc module in the improved Shallow-UWnet model is a multi-branch convolutional neural network module that enhances the expressiveness of the image processing model by extracting features at different scales. The Inc module consists of four parallel branches, each of which uses a different convolution operation. The splicing operation splices the outputs of each branch in the channel dimension.

4. The underwater image enhancement method using a multi-convolutional model combined with an attention mechanism according to claim 3, characterized in that: The first branch of the Inc module first reduces the number of channels of the input feature map through 1x1 convolution while retaining the key information of the input image. Subsequently, it passes through a 3x3 convolution layer to further extract local features and capture a wider range of local context information. The first branch uses LeakyReLU as its activation function, and the formula is: ; Where x is the neuron input, that is, the value of each pixel position can be regarded as a neuron input, is a constant; The second branch first compresses the number of channels through 1x1 convolution; then uses 5x5 convolution, and the second branch also uses LeakyReLU as the activation function; The third branch first uses a 3x3 maximum pooling operation to reduce the spatial resolution and the size of the feature map, focusing on the global information of the image. The pooled feature map is further adjusted with a 1x1 convolution and nonlinearity is added through the LeakyReLU activation function to enhance the network's feature extraction capabilities. The fourth branch adjusts the number of channels through 1x1 convolution operation and does not perform spatial feature expansion.

5. The underwater image enhancement method according to claim 4, characterized in that: The concatenation operation of the Inc module is to concatenate these four tensors in the channel dimension. Its mathematical representation is as follows: ； O 1 ,O 2 ,O 3 ,O 4 They are the tensors of the four branches in the Inc module, and the output tensor dimension of each branch is , where B represents the batch size, C represents the number of channels, H and W are the height and width respectively, then the output tensor dimension after splicing will become: ； The number of channels is expanded from the original C to 4C, thereby integrating different features extracted by multiple branches.

6. The underwater image enhancement method according to claim 4, characterized in that: Improved Shallow-UWnet model loss function: The pixel-level mean square error loss is introduced to measure the difference between the network output image and the real clear image at the pixel level. The formula is: ； in represents the model output image, is the corresponding clear image, N is the total number of pixels; In order to improve the quality of the image in terms of structural perception, the network introduces a second loss: ； I is the generated image, For real images, the first Layer Extraction Features ; Is a real image In the VGG network Feature representation of the layer; is the L2 norm, that is, the Euclidean distance between feature representations; It is The weight coefficient of the layer feature is used to balance the influence of different layer features; The final training loss function is: ； 、 is the weight coefficient.

7. An underwater image enhancement system using a multi-convolutional model combined with an attention mechanism, employing the underwater image enhancement method using a multi-convolutional model combined with an attention mechanism as described in any one of claims 1-6, characterized in that: It includes a data acquisition unit, a data processing unit and a display unit; Data acquisition unit: acquire underwater image data; Data processing unit: It processes the distortion problem in underwater images by combining the DICAM model with the improved Shallow-UWnet model; Display unit: image visualization output.

Images