Underwater image enhancement method and system based on combination of multiple convolution models and attention mechanism

Abstract

The invention belongs to an image processing technology, and particularly relates to an underwater image enhancement method and system based on combination of multiple convolution models and an attention mechanism, and the technology is formed by combination of a DIAM model and a Shallow-UWnet improved model. The DIAM model is mainly used for carrying out color restoration in a mode of complementing hue weakening and non-uniformity of the underwater image, and can solve the problems of underwater noise, atomization and the like. According to the Shell-UWnet improved model, on the basis of Shell-UWnet, a highest pooling layer and an Inc multi-convolution module are added, the highest pooling layer effectively reduces network parameters and calculation overhead through dimension reduction operation, the operation efficiency is improved, the key feature extraction capacity is enhanced, detail loss is avoided, Inc can extract features from different scales, and the algorithm is simple and convenient to operate. And the feature capture capability of the model is enhanced.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and particularly relates to an underwater image enhancement method and system that combines a multi-convolution model with an attention mechanism. Background Art

[0002] Underwater images provide crucial information for numerous engineering and research tasks, such as the condition monitoring of energy infrastructure, seabed topography mapping, garbage detection, and the recognition and classification of underwater objects and events.

[0003] However, due to the interference of factors such as water turbidity, light attenuation, and underwater particulate matter, raw underwater images and videos often suffer from severe visual distortion problems, manifested as uneven color deviation, blurred images, and deficiencies in quality, contrast, and brightness.

[0004] To address these challenges and improve the clarity of underwater images for better service in practical applications, researchers have proposed various underwater image enhancement techniques. These techniques aim to obtain clearer images by optimizing contrast, adjusting color distribution, and eliminating blur effects. Early research mainly focused on contrast enhancement techniques such as contrast-limited adaptive histogram equalization. However, the degree of degradation of underwater images is closely related to the distance between the object and the camera. For example, in the original image, the content in the area far from the camera is often more blurred than the area close to the camera. In the enhanced image, the visibility of these areas has been significantly improved. Therefore, traditional image enhancement methods have limited effectiveness in processing underwater images. Summary of the Invention

[0005] This application addresses problems such as underwater image fogging noise, scale degradation, and non-uniform color attenuation, and proposes a method and system based on a neural network architecture that combines a multi-convolution model with an attention mechanism, achieving a significant improvement in the color, brightness, contrast, and details of underwater images. The technical solution is as follows: An underwater image enhancement method that combines a multi-convolution model with an attention mechanism, comprising the following steps: S1. In the DICAM model, the Inc module extracts features from different scales through multiple convolution kernels or pooling operations of different sizes, and the CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features; S2. Input the image processed by the DICAM model into the improved Shallow-UWnet model. The improved Shallow-UWnet model includes multiple stacked ConvBlock and Inc modules, and a skip connection mechanism is introduced after each Inc module; The ConvBlock is responsible for extracting local features such as details and textures. The Inc module extracts structural information at different scales through multi-branch convolutions, and the skip connection fuses the shallow input with the deep features.

[0006] Preferably, the DICAM network includes an Inc module, and the Inc module contains four parallel branch paths: The first branch uses a 1×1 convolution followed by a 3×3 convolution to extract texture and edge information at a set scale; The second branch uses a 5×5 convolution based on the 1×1 convolution, which can capture broader context information and helps to recover blurred regions; The third branch suppresses local noise through 3×3 max pooling combined with 1×1 convolution; The fourth branch directly uses a 1×1 convolution to preserve the original local features and color information; By concatenating the outputs of the four branches in the channel dimension, the Inc module achieves efficient fusion of multi-scale features; The DICAM model applies the Inc module to the R, G, and B channels of the image respectively, which can process the specific degradation effects received by each channel and extract more distinguishable color features.

[0007] Preferably, the DICAM model includes a CAM module. The CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features. The specific method is as follows: Step 1: For the m-th feature map F extracted by the Inc module m , first compress it using global average pooling to obtain a descriptive vector in the channel dimension , which represents the statistical average of the overall response of each channel. The formula is as follows: ; Step 2: Reduce the dimension of the GAP vector through a fully connected layer, and then increase the dimension through another fully connected layer. The formula is expressed as: ; Among them, and represent the fully connected weight matrices for dimension reduction and dimension increase respectively, represents the Softsign activation function, and r is the compression ratio; Step 3: The attention coefficient Coeff m is re-expanded to the same shape as the original feature map, and the feature is adjusted through channel-wise multiplication: ; The CAM module can enhance the channel selectivity of the feature map and be inserted into multiple stages of the network.

[0008] Preferably, the ConvBlock is a lightweight convolutional block that processes the input feature map through two layers of convolution and the activation function ReLU, extracts local features such as texture and details, and prevents overfitting through Dropout. After being processed by the ConvBlock, the feature map enters the Inc module.

[0009] Preferably, the Inc module in the improved Shallow-UWnet model is a multi-branch convolutional neural network module that enhances the expressive ability of the image processing model through feature extraction at different scales. The Inc module includes four parallel branches, each branch uses different convolutional operations, and the concatenation operation concatenates the outputs of each branch in the channel dimension.

[0010] Preferably, the first branch of the Inc module first reduces the number of channels of the input feature map through 1x1 convolution while retaining the key information of the input image. Subsequently, through a 3x3 convolutional layer, it further extracts local features and captures a wider range of local context information; The first branch uses LeakyReLU as its activation function, and the formula is: ; where x is the neuron input, that is, the value at each pixel position can be regarded as a neuron input, is a constant; The second branch first compresses the number of channels through 1x1 convolution; then, it uses a 5x5 convolution to obtain a larger receptive field. The second branch also uses LeakyReLU as the activation function; The third branch first performs a 3x3 max pooling operation to reduce the spatial resolution through pooling, reduce the size of the feature map, and focus on the global information of the image. The pooled feature map further adjusts the number of channels through 1x1 convolution and increases the non-linearity through the LeakyReLU activation function to enhance the feature extraction ability of the network; The fourth branch adjusts the number of channels through 1x1 convolution operation without performing spatial feature expansion; Preferably, the concatenation operation of the Inc module concatenates these four tensors in the channel dimension, and its mathematical representation is as follows: ; O 1 ,O 2 ,O 3 ,O 4Tensors of the four branches in the Inc module respectively, and the output tensor dimension of each branch is , where B represents the batch size, C represents the number of channels, H and W represent the height and width respectively. Then the output tensor dimension after concatenation will become: ; The number of channels is expanded from the original C to 4C, thus integrating different features extracted by multiple branches.

[0011] Preferably, the loss function of the improved Shallow-UWnet model: Introduce the pixel-level mean square error loss, which is used to measure the difference between the network output image and the real clear image at the pixel level. The formula is: ; Where represents the model output image, is the corresponding clear image, and N is the total number of pixels; To improve the quality of the image in terms of structure perception, the network introduces a second loss: ; I is the generated image, is the real image, and the features are extracted through the layer of the VGG network , is the real image at the layer of the VGG network; is the L2 norm, that is, calculate the Euclidean distance between the feature representations; is the weight coefficient of the feature at the l-th layer, which is used to balance the influence of features at different layers; The final training loss function is composed of the addition of the above two parts, that is: ; , are weight coefficients, .

[0012] An underwater image enhancement system combining a multi-convolution model with an attention mechanism, including a data acquisition unit, a data processing unit, and a display unit; Data acquisition unit: Obtain underwater image data; Data processing unit: Process complex distortion problems in underwater images by combining the DICAM model and the improved Shallow-UWnet model; Display unit: Visualize and output images.

[0013] Compared with the prior art, the beneficial effects of this application are as follows: This technology is composed of the combination of the DICAM model and the improved Shallow-UWnet model. The DICAM model mainly repairs colors by compensating for the inconsistent tone attenuation of underwater images. Compared with the original DICAM model, while further correcting colors, the improved Shallow-UWnet model can optimize problems such as underwater noise and fogging. The improved Shallow-UWnet model is based on Shallow-UWnet and adds a max pooling layer and an Inc multi-convolution module. The max pooling layer effectively reduces network parameters and computational overhead through dimensionality reduction operations, not only improving the running efficiency but also enhancing the ability to extract key features and avoiding detail loss. Inc can extract features from different scales to enhance the model's feature capture ability.

[0014] This technology effectively improves the ability to handle complex distortion problems in underwater images, such as color offset, contrast decline, detail blur, etc., through the combination of the DICAM model and the improved Shallow-UWnet model. Brief Description of the Drawings

[0015] Figure 1 is the overall neural network architecture of this method; Figure 2 is the Inc module; Figure 3 is the CAM module; Figure 4 is the comparison chart of effects. Specific Embodiments

[0016] The technical solutions of this application will be described in detail below through specific embodiments and the accompanying drawings. It should be understood that the specific features in the embodiments of this application are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. The specific technical features can be combined with each other.

[0017] An underwater image enhancement method combining a multi-convolution model with an attention mechanism includes the following steps: S1. In the DICAM model, the Inc module extracts features from different scales through multiple convolution kernels or pooling operations of different sizes, and the CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to achieve the enhancement of important features and the suppression of unimportant features; S2. Input the pictures processed by the DICAM model into the improved Shallow-UWnet model. The improved Shallow-UWnet model includes multiple stacked ConvBlock and Inc modules, and a skip connection mechanism is introduced after each Inc module; The ConvBlock is responsible for extracting local features such as details and textures. The Inc module extracts structural information at different scales through multi-branch convolutions, and the skip connection fuses the shallow input with the deep features.

[0018] I. DICAM Model: In the DICAM network, the Inc and CAM modules constitute the core architecture design of the whole model. They are respectively responsible for extracting multi-scale spatial features and strengthening the information expression at the channel level. They complement each other in function, enabling the model to more effectively cope with complex distortion problems in underwater images, such as color shift, contrast reduction, detail blurring and other phenomena.

[0019] The design of the Inc module is derived from the classic Inception structure. Its main idea is to extract features at different scales through multiple convolution kernels or pooling operations of different sizes to enhance the feature capture ability of the model. This module contains four parallel branch paths: the first branch uses a 1×1 convolution followed by a 3×3 convolution, which is good at extracting medium-scale texture and edge information; the second branch uses a 5×5 convolution based on a 1×1 convolution, with a larger receptive field, which can capture broader context information and helps to restore blurred areas; the third branch enhances feature robustness and suppresses local noise through 3×3 max pooling combined with 1×1 convolution; while the fourth branch directly uses a 1×1 convolution to retain the original local features and color information. By concatenating the outputs of the four branches in the channel dimension, the Inc module realizes the efficient fusion of multi-scale features. This structure not only improves the feature expression ability but also maintains the controllability of computational efficiency, making it very suitable for deployment on devices with limited resources. It is worth mentioning that DICAM applies the Inc module to the R, G, and B channels of the image respectively, which is a clever design for the uneven color distortion problem in underwater images and can separately handle the specific degradation effects received by each channel and extract more distinguishable color features.

[0020] The basic idea of CAM is: to learn an attention weight coefficient for each channel through the global statistical information of the feature map (i.e., the average value of each channel), so as to enhance important features and suppress unimportant features. Specifically, for the m-th feature map F extracted by the Inc module m , we first compress it using global average pooling (GAP) to obtain a description vector in the channel dimension . This vector represents the statistical average of the overall response of each channel, and the formula is as follows: ; To improve the non - linear representation ability of attention learning, we draw on the "compression - excitation" mechanism of the SE module. First, the GAP vector is dimension - reduced (compressed) through a fully - connected layer and then dimension - increased (restored) through another fully - connected layer, where the compression rate r is empirically set to 4. It is formally expressed as: ; where, and represent the fully - connected weight matrices for dimension - reduction and dimension - increase respectively, and ψ represents the Softsign activation function, which is used to compress the output into the range of [−1,1]. Different from ReLU and Sigmoid used in the SE module, Softsign can learn negative weights, which is particularly effective for color deviation correction in underwater images (such as over - strong blue / green channels). Through this mechanism, the model can effectively suppress noise channels and improve the realism and color balance.

[0021] Finally, the attention coefficient Coeff m is re - expanded to the same shape as the original feature map, and feature adjustment is performed through channel - wise multiplication: ; In this way, the CAM module completes the channel - selective enhancement of the feature map with extremely low computational overhead and can be flexibly inserted into multiple stages of the network, such as the initial color channel feature extraction and the final color correction stage.

[0022] The introduced CAM module further improves the model's ability to focus on effective features. This module first performs global average pooling on the input feature map to generate a channel - level global description vector representing the overall response of each channel; then constructs a channel attention map through two fully - connected layers and the Softsign activation function to evaluate the importance of each channel; finally, remaps this attention weight back to the original feature map to perform weighted enhancement on each channel. This channel attention mechanism can automatically learn to focus on important feature channels, thereby suppressing redundant or inefficient information and strengthening key features, especially in color correction.

[0023] In DICAM, the CAM is used for attention enhancement of the features in the R / G / B three channels respectively, and also for re-weighting the overall fused features, enabling the model to dynamically select the most effective feature channels during the process of integrating information. Overall, the Inc module provides rich and hierarchical spatial features, while the CAM module guides the model to learn more reasonable feature channel weights from a global semantic perspective. The former focuses on the extraction of feature "breadth", and the latter focuses on the enhancement of feature "depth". This spatial-channel complementary structure enables DICAM to more comprehensively understand the complex degradation features in underwater images and finally output enhanced images that are clearer, have natural colors, and are rich in details.

[0024] II. Improved Shallow-UWnet Model: In the underwater environment, due to the influence of various physical processes on the propagation of light in water, such as absorption, scattering, and reflection, the quality of the acquired images is generally poor. These phenomena are mainly manifested as problems such as color imbalance (usually showing an obvious blue-green tone), reduced contrast, blurred details, uneven brightness, and decreased overall image clarity. To address these challenges, the DICAM model mainly focuses on correcting the color imbalance and uneven brightness of underwater images, and effectively alleviates the problems of color cast and uneven brightness distribution by introducing depth information and color attenuation characteristics.

[0025] However, the DICAM model still has certain limitations in aspects such as noise reduction, defogging, and detail enhancement. Therefore, this technology optimizes the structure and improves the performance of the Shallow-UWnet model to make up for the deficiencies of DICAM in underwater image preprocessing. The optimized model not only improves the clarity and detail retention ability of the image, but also enhances the robustness to underwater fog and noise. In addition, in terms of image color correction, this method further integrates multi-scale feature extraction and attention mechanism, enabling the model to better adapt to the illumination changes in different underwater scenes while maintaining natural color restoration, and finally realizing the all-round enhancement and repair of underwater images.

[0026] The preprocessing part of the network includes a 3×3 convolutional layer to map the input three-channel RGB underwater image into 64-channel features. The purpose of this step is to transform the low-dimensional input into a higher-dimensional feature space to extract richer semantic information. Immediately afterwards, max pooling is used to perform downsampling on the feature map. In underwater images, since the blurring and degradation of distant targets are relatively serious, expanding the receptive field helps the model understand the overall background information of these blurred areas. Subsequently, upsampling is performed through learnable transposed convolution to restore the spatial dimension of the image and keep it consistent with the input for the alignment of the final output image.

[0027] The feature extraction module is introduced below. The feature extraction module is the core of the entire network and is composed of multiple stacked ConvBlock and Inc modules. ConvBlock is a lightweight convolutional block that contains multiple convolutional operations, ReLU activations, and Dropout operations. ReLU is a common activation function that sets the part of the input value less than 0 to 0, while for inputs greater than or equal to 0, ReLU directly retains the input value. In this way, ReLU introduces non-linearity in the neural network, enabling the network to learn more complex patterns. In addition, the calculation of ReLU is very simple, usually accelerating the training process and effectively alleviating the vanishing gradient problem. The formula is: f(x)=max(0,x); where x is the input of the neuron.

[0028] Dropout is a regularization method that prevents model overfitting by randomly discarding some neurons during training. In each training, Dropout randomly selects neurons and sets their outputs to zero, which forces the network not to rely on a single neuron or feature, but to learn more robust representations. In this way, Dropout can effectively improve the generalization ability of the model, especially when the data is insufficient or the model complexity is high. ConvBlock mainly processes the input feature map through two layers of convolution and the activation function ReLU, extracts local features such as texture and details, and prevents overfitting through Dropout. After being processed by ConvBlock, the feature map enters the Inc module.

[0029] The Inc module is a multi-branch convolutional neural network module designed to enhance the expressive power of the image processing model through feature extraction at different scales. The design of this module is inspired by the Inception network structure, but it combines multiple branches to capture image features at different levels and scales, thereby improving the feature representation ability of the network. The Inc module consists of four parallel branches, each branch uses different convolutional operations, and finally the outputs of each branch are concatenated in the channel dimension. The specific implementation and role of each branch in this module will be analyzed in detail below.

[0030] The first branch first reduces the number of channels of the input feature map through 1x1 convolution, thereby reducing the computational amount and the number of parameters. This operation can effectively reduce unnecessary calculations while retaining the key information of the input image. Subsequently, through the 3x3 convolutional layer, local features are further extracted. The 3x3 convolution has a large receptive field and can capture a wider range of local context information. The LeakyReLU formula is: ; where x is the neuron input, i.e., the value at each pixel position can be regarded as a neuron input. is a very small constant, set to 0.01. In this way, LeakyReLU assigns a very small slope to negative inputs, usually a very small constant a, instead of directly outputting negative values as 0. In this way, even when the input is negative, LeakyReLU will output a non-zero negative number, avoiding the problem of "dead neurons". Using the LeakyReLU activation function introduces a non-linear transformation, enhancing the learning ability of the network. Through such a structure, the first branch can extract local features in a relatively small area.

[0031] The second branch is similar to the first branch. It also first compresses the number of channels through a 1x1 convolution. Then, it uses a 5x5 convolution instead of a 3x3 convolution to obtain a larger receptive field. Compared with the 3x3 convolution, the 5x5 convolution can capture a larger range of local features, especially when dealing with areas with large spatial variations. Like the first branch, the second branch also uses LeakyReLU as the activation function, enabling the network to learn more complex non-linear features.

[0032] The design of the third branch is different. It first adopts a 3x3 max pooling operation. By pooling, it reduces the spatial resolution, decreases the size of the feature map, and focuses on the global information of the image. The pooling operation helps reduce the computational amount and enhances the translational invariance of the model, making the network more robust to small changes in the input. The pooled feature map is further adjusted in terms of the number of channels through a 1x1 convolution and increased non-linearity through the LeakyReLU activation function, enhancing the feature extraction ability of the network.

[0033] The fourth branch is the simplest one. It only adjusts the number of channels through a 1x1 convolution operation and does not perform spatial feature expansion. The 1x1 convolution in this branch is mainly used to extract the relationships between channels while maintaining the dimension of the feature map. The computational amount of this branch is small, and it can effectively capture the subtle changes between channels of the input feature map.

[0034] The concatenation operation mainly appears at the end of the Inc module and is used to fuse the feature maps extracted by the four different convolutional branches along the channel dimension. These four branches respectively use different convolutional kernels (1×1, 3×3, 5×5) and pooling structures to extract local and global features of the input feature map at different scales. The output of each branch is transformed through the non-linear activation function LeakyReLU, and finally four tensors are obtained: O 1 ,O 2 ,O3 ,O 4 . To integrate these multi-scale features into a unified feature map, the Inc module adopts a concatenation operation, which essentially concatenates these four tensors along the channel dimension (dim = 1). Its mathematical representation is as follows: ; The output tensor dimension of each branch is , where B represents the batch size, C represents the number of channels, H and W are the height and width respectively. Then the output tensor dimension after concatenation will become: . This means that the concatenation operation expands the number of channels from the original C to 4C without changing the spatial dimensions (H and W), thus integrating the different features extracted by multiple branches. This operation not only enriches the feature representation but also enhances the model's perception ability of different structural information. Through this structured concatenation operation, the Inc module effectively fuses the information in different convolutional paths, enabling the network to simultaneously capture the details, edges, and context of larger regions in the image, providing more comprehensive feature support for subsequent image processing.

[0035] To address the problem of structural information loss in underwater images, the network introduces a skip connection mechanism after each Inc module. A core role of the skip connection is information retention. Neural networks usually extract low-level detail features such as edges, colors, and textures in the previous layers, and these features are particularly crucial in image enhancement tasks. Through the skip connection, these shallow detail information can be directly fused with the deep abstract semantic features, thereby enhancing the overall feature expression ability of the network, making the output image have both global perception and delicate local details. In addition, the skip connection helps to construct a smoother gradient propagation path, making the network more stable and convergent during training. Concatenate the original input image and the current feature map along the channel dimension. This design is very crucial as it can reintroduce shallow structural information (such as edges, textures, object contours) into the deep features, thereby improving the spatial consistency and detail performance of the image. The number of channels of this fused feature is 256, and then a 1×1 convolution is used to compress it back to 64 channels to control the computational complexity of the network while retaining the effective information after fusion.

[0036] The skip connection mechanism, combined with the ConvBlock module and the Inc module, constructs a multi-scale feature extraction architecture of "depth + width + cross-layer fusion". The ConvBlock is responsible for extracting local features such as details and textures. The Inc module extracts structural information at different scales through multi-branch convolutions. The skip connection fuses the shallow input with the deep features, effectively preserving the details of the original image and alleviating the vanishing gradient problem. The three work together, enabling the network to maintain details while having strong structural understanding and restoration capabilities in the image enhancement task.

[0037] In the output stage, the network uses a 3×3 convolution to map the 64-channel high-dimensional feature map back to a 3-channel image, outputting the final result image. This output image will have a more natural color distribution, clearer texture information, and a more balanced brightness distribution, significantly superior to the original degraded image.

[0038] This model is an efficient neural network for underwater image enhancement tasks, with strong feature modeling and image restoration capabilities. Its core advantages are: modular design, multi-scale feature extraction, skip information fusion, and a flexible depth control mechanism. These features enable the model to adapt to various complex underwater environments and provide a stable, clear, and high-quality image basis for underwater visual perception.

[0039] III. Loss Function: When training a neural network for underwater image enhancement tasks, the model adopts a multi-term loss function strategy, aiming to constrain the quality of the generated images from multiple levels. This strategy not only focuses on the restoration accuracy of the image at the pixel level but also emphasizes the clarity and structural consistency of the image at the perceptual level. By combining two loss functions with complementary characteristics, the model can generate enhanced images that are superior both subjectively and objectively.

[0040] First, the pixel-level mean squared error loss (MSELoss) is introduced during training to measure the difference between the network output image and the real clear image at the pixel level. The mean squared error calculates the sum of the squared differences between the two images at each pixel point, and its mathematical expression is: ; where I represents the model output image, and I ∗ is the corresponding clear image (GroundTruth), and N is the total number of pixels. This loss can ensure that the generated image is as close as possible to the real image at the basic levels such as color and brightness, but it usually cannot effectively express the differences in structure and texture of the image.

[0041] To improve the quality of images in terms of structural perception, the network introduces a second loss: the Perceptual Loss. This loss is calculated based on the activation features of a certain convolutional layer in the pre-trained VGG-19 network. The Perceptual Loss constrains the network to preserve edge sharpness, texture structure, and spatial consistency as much as possible during the restoration process by comparing the enhanced image with the real image in the high-level semantic feature space. Specifically, the model inputs the enhanced image and the real image into the VGG network, extracts the feature maps at the specified convolutional layer, and calculates the Euclidean distance between these two feature maps as the Perceptual Loss. This method effectively guides the model to generate images that are more in line with human perception.

[0042] The pixel-level mean squared error loss is introduced to measure the difference between the network output image and the real clear image at the pixel level. The formula is: ; where represents the model output image, is the corresponding clear image, and N is the total number of pixels; To improve the quality of images in terms of structural perception, the network introduces a second loss: ; I is the generated image, is the real image, and features are extracted through the layer of the VGG network ; is the real image at the layer of the VGG network; is the L2 norm, that is, the Euclidean distance between the feature representations is calculated; is the weight coefficient of the features at the l-th layer, used to balance the influence of features in different layers; The final training loss function is composed of the sum of the above two parts, that is: ; and are weight coefficients.

[0043] This combination strategy achieves a balance between pixel-level accuracy and semantic-level structure restoration, ensuring both the authenticity of image details and the improvement of the overall perceptual quality.

[0044] In the underwater image enhancement task, this multi-loss design is particularly important. Due to color drift, edge blurring, and detail loss in underwater images, a single MSE loss cannot restore the visual effect of the real underwater scene. By introducing the VGG perceptual loss, the network can effectively identify and restore the blurred object structures and textures in underwater images, making the enhanced images more natural and clear. This loss design provides strong training constraints for the images, significantly improving the network's enhancement ability and generalization performance.

[0045] IV. Differences and connections between the Inc module in the DICAM network and the Inc module in the Shallow-UWnet model: In the design of this system, although the DICAM model and the Shallow-UWnet improved model network independently use the Inc module respectively, there is a close connection between the two as a whole, forming a feature extraction system that progresses layer by layer and works synergistically. The Inc module of DICAM preserves the details of the shallow-layer image through feature enhancement and passes it as a feature input to the Shallow-UWnet improved model, serving as the original input of the Shallow-UWnet improved model and participating in feature splicing through the skip connection mechanism; the Inc module in the Shallow-UWnet improved model further deepens this information in the middle and high-level feature streams, thus realizing continuous feature modeling from shallow to deep and from local to global. The structures of the Inc modules in both parts are the same, and they both extract and fuse the feature information of different receptive fields through multi-branch and multi-scale convolutions. The core purpose is to fully capture the rich and complex features in underwater images.

[0046] In the DICAM part, the Inc module is mainly used for preliminary feature enhancement of the input image. DICAM independently applies the Inc module on each color channel, extracts multi-scale detail features from the original pixel level, and combines the channel attention (CAM) mechanism to strengthen key information, thus effectively improving the image quality before the image enters the backbone network. At this stage, the Inc module is oriented towards low-level and fine-grained image features, with the goal of restoring the basic details and local structures of the image.

[0047] In the Shallow-UWnet improved part, the Inc module is deployed in the middle feature stream and acts on the middle and high-order feature maps that have undergone preliminary convolution processing. At this time, the Inc module no longer directly processes the original pixel information, but further refines and enhances the semantic information and multi-scale texture relationships in the feature space. By continuously inserting the Inc module in the feature stream and cooperating with the convolutional block (ConvBlock) and feature splicing, the network can gradually accumulate and fuse local and global information at different levels to support more accurate image reconstruction.

[0048] From an overall perspective, the Inc modules in DICAM and the improved Shallow-UWnet model actually form a relationship of echoing before and after and connecting up and down in function: the former focuses on the original image, emphasizing detail compensation and quality improvement; the latter focuses on abstract features and further deepens information modeling and context understanding. Using the Inc module continuously in two stages completes a comprehensive extraction and enhancement process from the low-level features of the input image (such as edges and color distributions) to the deep semantic features (such as object structures and regional consistency). Through this cross-level, multi-scale, and progressive feature modeling strategy, the network can simultaneously model at different spatial scales and semantic depths, greatly enhancing the restoration ability for complex degradation phenomena in underwater images (such as light attenuation, color distortion, and detail loss). Using the Inc module continuously in two stages enables the network to cover the feature extraction requirements from shallow details to deep semantics and constructs a cross-level, multi-scale, and progressive feature processing chain.

[0049] IV. Experimental Design: To comprehensively verify the effectiveness of the underwater image enhancement method proposed in this paper, the experiments were carried out under the Windows operating system, using the Python 3.7 programming environment, and building the model based on the advanced deep learning framework PyTorch 1.12.1. The designed enhancement network structure is compact and logically clear, giving full play to the advantages of deep learning in feature extraction and non-linear modeling. During the network training process, the Adam optimizer was selected as the parameter update strategy. This optimizer has the ability to adaptively adjust the learning rate, can effectively combine the first-order and second-order momentum information of the gradient, and enhances the stability of the model training process while improving the convergence efficiency.

[0050] All experiments were completed on a high-performance server equipped with an NVIDIA GeForce RTX3090 GPU (24GB of video memory), ensuring the efficient and stable processing of images. This hardware platform provides powerful computing power support for the training and inference of the model on large-scale datasets. The test data is shown in Table 1.

[0051] Table 1 Image Enhancement Quality Evaluation of Different Methods Based on the EUVP Dataset 。

[0052] In terms of experimental parameter settings, the initial learning rate was set to 0.0002. The number of training samples in each batch was 16, balancing the computing resources and memory consumption, which helped to improve the training efficiency and image quality. The model was trained for 200 epochs to ensure good stability and reliability of the results.

[0053] To comprehensively evaluate the performance and practical value of the proposed method, this paper adopts an image quality evaluation system that combines subjective and objective evaluations. In terms of objective evaluation, three widely recognized and commonly used indicators in the industry are selected to quantitatively analyze the performance of the enhanced images in various scenarios. These three indicators are as follows: 1. Peak Signal-to-Noise Ratio (PSNR): It is a commonly used image quality evaluation indicator based on pixel accuracy. It mainly judges the enhancement effect by calculating the error between the enhanced image and the reference image (usually the original high-quality image). Its essence lies in evaluating the intensity of noise in the image. The higher the value, the smaller the difference between the enhanced image and the original image, that is, the less distortion and better quality of the image. Therefore, the larger the PSNR, the better the enhanced algorithm performs in preserving image details and structures. The following are the calculation formulas for the peak signal-to-noise ratio and the variance used: ; ; Structural Similarity Index (SSIM): It is a similarity evaluation method from the perspective of image structure information. Different from traditional indicators based only on pixel errors, SSIM pays more attention to the matching of images in three aspects: brightness, contrast, and structural details, and tries to measure image quality from the perspective of human visual perception. A higher SSIM value usually means that the enhanced image has better consistency with the reference image in terms of texture, edges, and local structures, indicating that the method has strong advantages in detail preservation and visual fidelity.

[0054] Underwater Image Quality Metric (UIQM): It is a comprehensive indicator designed specifically for evaluating the quality of underwater images and is widely used in related research. This indicator comprehensively evaluates the image quality from three key dimensions: color fidelity, image contrast, and sharpness. UIQM can effectively reflect the comprehensive ability of the enhancement algorithm in dealing with underwater image degradation phenomena (such as light scattering and absorption). Usually, the higher the UIQM value, the better the proposed enhancement method performs in restoring the true color of the image, enhancing visual contrast, and detail sharpness, indicating that the method has stronger practicality and the ability to improve visual effects.

[0055] Figure 1This is the overall neural network architecture of the method. Generally speaking, it is divided into two module methods. The first module method is the DICAM module, which aims to fully exploit the multi-scale spatial features contained in different channels (R, G, B) of the image. At the same time, it enhances the response ability of the key feature channels through the CAM module, thereby enhancing the expression effect and reconstruction ability of the image. The entire model takes a three-channel color image as the input and splits it into three separate channels, R, G, and B, which are respectively fed into three feature extraction Inc modules with shared structures but independent parameters. The feature maps extracted from each channel will be further fed into the CAM module. After extracting and enhancing the features in the three color channels respectively, the model concatenates them in the channel dimension to form a fused feature map. This fused feature map will go through another Inc module to extract higher-level features and use the channel attention module again to enhance the expression ability. The final output part uses two convolutional operations to reduce the number of channels to 3 channels and limits the output range through an activation function to generate a standard image format. The second module method is a deep convolutional neural network for underwater image enhancement, which is an improvement of the Shallow-UWnet model. Its design aims to restore clear and natural visual effects from degraded underwater images. The model structure is relatively more concise, facilitating analysis and fast training, but still retains key enhancement mechanisms such as multi-scale feature extraction and skip connections. The input of the model is a three-channel color image. First, a convolutional layer is used to map the number of channels to 64 to form an initial feature map. Subsequently, a max-pooling operation is performed to extract more globally perceptive features. Then, an upsampling is achieved through a learnable transposed convolutional layer to restore the feature map to the original resolution. An activation function is used in this process to enhance the non-linear expression ability. In the backbone part, the model constructs a stacked structure composed of multiple alternating multi-scale convolutional modules and Inc modules. After each Inception module, the model concatenates its output with the original input image to explicitly introduce the low-level information of the original image. This concatenation results in a sharp increase in the number of channels. To control the computational cost, the model reduces the number of channels to 64 through a 1×1 convolutional layer, retaining the key information while maintaining the computational efficiency. Finally, the model uses an output convolutional layer to restore the 64-channel feature map to a three-channel RGB image as the enhanced image output.

[0056] Figure 2It is the Inc module. The Inc module is a multi-branch convolutional structure built based on the Inception idea, aiming to extract multi-scale features of images. It processes the input through four parallel sub-branches with different receptive fields respectively, and then concatenates these features in the channel dimension, thereby enhancing the richness and diversity of feature expression. In this method's Inc module, the first branch uses a 1×1 convolution for dimensionality reduction followed by a 3×3 convolution, which is suitable for extracting medium-scale texture information; the second branch has a similar structure but uses a 5×5 convolution with a larger receptive field, which is beneficial for capturing a wider range of context features; the third branch first performs max pooling to extract smooth or background region information, and then integrates the results through a 1×1 convolution; the fourth branch only uses a 1×1 convolution to quickly extract local responses. The LeakyReLU activation function is applied after each convolutional layer to introduce non-linearity and alleviate the "neuron death" problem of ReLU in the negative interval. The outputs of the four branches are finally concatenated together to form an output tensor that fuses multi-scale features. This design not only improves the model's perception ability of image content at different scales but also effectively maintains the computational efficiency of the network, and is very suitable for tasks such as image enhancement that require high requirements for detail restoration and context understanding.

[0057] Figure 3 It is the CAM module. The CAM module is an attention mechanism for extracting importance weights in the channel dimension, aiming to enhance the network's response ability to key feature channels and suppress irrelevant or redundant channel information by explicitly modeling the relationships between channels, thereby improving the image expression effect. Specifically, CAM first compresses the spatial information of each channel into a single value through global average pooling to form a channel description vector, representing the global semantics of the entire channel. Then, a fully connected network composed of two fully connected layers is used to model this channel description, with the Softsign activation function used in the middle to introduce non-linearity. The first linear layer is used for dimensionality reduction to capture a compact cross-channel representation; the second linear layer then increases the dimension back to the original number of channels and outputs the attention coefficients for each channel. Finally, the tensor formed by these attention coefficients will be multiplied with the original input feature map channel by channel through the broadcast mechanism to achieve channel weight adjustment. The entire process does not change the spatial size of the input but will significantly improve the information selectivity between channels, enabling the model to focus more on the feature channels that contribute more to image enhancement.

[0058] Figure 4This is a comparison of the effect diagrams of this method with the original image and other underwater image enhancement methods. Among them, the method of Module 1 is the DICAM model method, and the method of Module 2 is the Shallow-UWnet method. The method of Module 1 is mainly used to solve the problem of the restoration degree of color cast images in underwater images, and is more suitable for the restoration of underwater images in deep water areas. However, it may not handle the blurring and grayness of the image well during the image enhancement process. The image of Model 2 has good effects in terms of image structure and brightness, but it may introduce other color casts. The effect diagrams intuitively show that all five effect diagrams have been greatly improved compared with the original image, and there are also significant differences among the five methods. The method of adaptive histogram equalization has a poor effect on the restoration of color cast images in the enhanced underwater images, but it has good effects on improving image brightness and image details. The fast underwater image enhancement method has good improvements in both image brightness and color cast degree. In contrast, the method in this paper has better effects in removing color cast, blurring and occlusion, and improving image contrast, etc.

[0059] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art of this technology, without departing from the technical principle of this application, several improvements and deformations can still be made, and these improvements and deformations should also be regarded as the protection scope of this application.

Claims

1. An underwater image enhancement method combining a multi-convolution model with an attention mechanism, characterized in that It includes the following steps: S1. In the DICAM model, the Inc module extracts features from different scales through multiple convolution kernels or pooling operations of different sizes, and the CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features; S2. Input the image processed by the DICAM model into the improved Shallow-UWnet model. The improved Shallow-UWnet model includes multiple stacked ConvBlocks and Inc modules, and a skip connection mechanism is introduced after each Inc module; The ConvBlock is responsible for extracting local features. The Inc module extracts structural information of different scales through multi-branch convolution, and the skip connection fuses the shallow input with the deep features.

2. The underwater image enhancement method combining a multi-convolution model with an attention mechanism according to claim 1, characterized in that, The DICAM model includes an Inc module, and the Inc module contains four parallel branch paths: The first branch uses a 1×1 convolution followed by a 3×3 convolution to extract texture and edge information of a set scale; The second branch uses a 5×5 convolution based on the 1×1 convolution to capture context information; The third branch suppresses local noise through 3×3 max pooling combined with 1×1 convolution; The fourth branch directly uses a 1×1 convolution to retain the original local features and color information; By concatenating the outputs of the four branches in the channel dimension, the Inc module realizes the efficient fusion of multi-scale features; The DICAM model applies the Inc module to the R, G, and B channels of the image respectively, can process the degradation effects suffered by each channel separately, and extract distinguishable color features.

3. The underwater image enhancement method combining a multi-convolution model with an attention mechanism according to claim 1, characterized in that The DICAM model includes a CAM module. The CAM module learns an attention weight coefficient for each channel through the global statistical information of the feature map to enhance important features. The specific method is as follows: Step 1. For the m-th feature map F extracted by the Inc module m , first compress it using global average pooling to obtain a description vector in the channel dimension , which represents the statistical average of the overall responses of each channel. The formula is as follows: ； Step 2. Reduce the dimension of the GAP vector through a fully connected layer, and then increase the dimension through another fully connected layer. The formula is expressed as: ； Among them, and respectively represent the fully connected weight matrices for dimensionality reduction and dimensionality increase, represents the Softsign activation function, and r is the compression ratio; Step 3. Attention Coefficient Coeff m is re-expanded to have the same shape as the original feature map, and feature adjustment is performed through per-channel multiplication: ； The CAM module can enhance the channel selectivity of the feature map and be inserted into multiple stages of the network.

4. The underwater image enhancement method combining a multi-convolution model with an attention mechanism according to claim 1, characterized in that, The ConvBlock is a lightweight convolution block that processes the input feature map through two layers of convolution and the activation function ReLU to extract local features, and uses Dropout to prevent overfitting. After being processed by the ConvBlock, the feature map enters the Inc module.

5. The underwater image enhancement method combining a multi-convolution model and an attention mechanism according to claim 1, characterized in that, The Inc module in the improved Shallow-UWnet model is a multi-branch convolutional neural network module that enhances the expression ability of the image processing model through feature extraction of different scales. The Inc module includes four parallel branches, each branch uses different convolution operations, and the concatenation operation concatenates the outputs of each branch in the channel dimension.

6. The underwater image enhancement method combining a multi-convolution model and an attention mechanism according to claim 5, characterized in that The first branch of the Inc module first reduces the number of channels of the input feature map through a 1x1 convolution while retaining the key information of the input image. Subsequently, through a 3x3 convolutional layer, it further extracts local features and captures a wider range of local context information; The first branch uses LeakyReLU as its activation function. The formula is: ; where x is the neuron input, i.e., the value at each pixel position can be regarded as a neuron input, is a constant; The second branch first compresses the number of channels through a 1x1 convolution; then a 5x5 convolution is used, and the second branch also uses LeakyReLU as the activation function; The third branch first performs a 3x3 max pooling operation to reduce the spatial resolution through pooling, decrease the size of the feature map, and focus on the global information of the image. The pooled feature map further adjusts the number of channels through a 1x1 convolution and increases the non-linearity through the LeakyReLU activation function to enhance the feature extraction ability of the network; The fourth branch adjusts the number of channels through a 1x1 convolution operation without performing spatial feature expansion.

7. The underwater image enhancement method combining a multi-convolution model and an attention mechanism according to claim 6, characterized in that, The concatenation operation of the Inc module concatenates these four tensors in the channel dimension, and its mathematical representation is as follows: ； O 1 ,O 2 ,O 3 ,O 4 Tensors of four branches in the Inc module respectively, and the output tensor dimension of each branch is , where B represents the batch size, C represents the number of channels, H and W are the height and width respectively, then the output tensor dimension after concatenation will become: ； The number of channels is expanded from the original C to 4C, thus integrating the different features extracted by multiple branches.

8. The underwater image enhancement method using a multi-convolution model combined with an attention mechanism according to claim 6, characterized in that Improved loss function of the Shallow-UWnet model: The pixel-level mean squared error loss is introduced to measure the difference between the network output image and the real clear image at the pixel level. The formula is: ； Among them represents the model output image, is the corresponding clear image, and N is the total number of pixels; To improve the quality of the image in terms of structure perception, the network introduces a second loss: ； I is the generated image, is the real image, and its features are extracted through the th layer of the VGG network; ; is the real image in the th layer of the VGG network; is the L2 norm, that is, calculating the Euclidean distance between feature representations; is the weight coefficient of the th layer features, used to balance the influence of features in different layers; The final training loss function is: ； , is the weight coefficient.

9. An underwater image enhancement system that combines a multi-convolution model with an attention mechanism, which adopts the underwater image enhancement method that combines a multi-convolution model with an attention mechanism as described in any one of claims 1-8, characterized in that, It includes a data acquisition unit, a data processing unit, and a display unit; Data acquisition unit: Obtain underwater image data; Data processing unit: Process the distortion problem in underwater images by combining the DICAM model with the improved Shallow-UWnet model; Display unit: Visualize and output the image.

Images