Underwater Image Enhancement Method Based on Dense Reuse and Skip Connections for Multi-Layer Color Features

Abstract

This invention relates to the field of underwater image enhancement technology, specifically to an underwater image enhancement method based on dense reuse and skip connections for multi-layer color features, comprising the following steps: S1, a feature extraction branch network CFCNN receives the original image and transmits it to each channel stitching layer through a skip connection strategy. Features are extracted through a densely connected convolutional module DCBlock, and the feature extraction branch network CFCNN extracts high-level features for each color space; S2, an underwater image enhancement model MCF-Net based on multi-layer color features extracts features in the RGB, HSV, and Lab color spaces through a parallel feature extraction branch network CFCNN. After channel stitching, an attention mechanism module CBAM is introduced for adaptive feature selection and weighting, and then convolutional operations are used to achieve image enhancement, outputting the result. This application enriches the diversity of feature representation and enhances the model's sensitivity to underwater image color features.

Description

Technical Field

[0001] This invention relates to the field of underwater image enhancement technology, and more specifically to an underwater image enhancement method based on dense reuse and skip connections for multi-layer color features. Background Technology

[0002] Due to the complexity of underwater environments and lighting conditions, underwater images are affected by wavelength-dependent absorption and scattering, often resulting in degradation problems such as color distortion and low contrast. These phenomena greatly limit the practical application of underwater images in marine biology, archaeology, and marine ecology. In recent years, underwater image enhancement has been extensively studied to improve the visual quality of underwater images with varying degrees of color distortion and reduced contrast.

[0003] Compared to traditional underwater image enhancement methods based on algorithm engineering and physical models, deep learning-based underwater image enhancement methods can directly learn the mapping between degraded and enhanced images from the data, without the need for manual parameter tuning or prior knowledge of the physical processes involved in underwater image formation, making them highly versatile.

[0004] While the underwater image enhancement method disclosed in CN120013839A introduces multi-color space transformation and cross-color space Transformer modules, it lacks a dense connection strategy, resulting in low feature reuse efficiency. Furthermore, downsampling operations easily lead to detail loss and red artifacts. Therefore, the current technology still has room for improvement in areas such as multi-color space feature fusion, global consistency enhancement, and adaptive enhancement of degraded regions. Summary of the Invention

[0005] The technical problem this invention aims to solve is to overcome the shortcomings of existing technologies and provide an underwater image enhancement method based on dense reuse and skip connections for multi-layer color features. This method extracts features from different color spaces and combines an attention mechanism to highlight key features, thereby enriching the diversity of feature representations and enhancing the model's sensitivity to underwater image color features. Furthermore, when extracting features for each color space, this invention employs a strategy of dense connections and skip connections, achieving better extraction of potential high-level features in each color space with fewer parameters. This improves the model's ability to address degradation problems inherent in underwater environments, such as color distortion and reduced contrast.

[0006] This invention is achieved through the following technical solution: an underwater image enhancement method based on dense reuse and skip connections for multi-layer color features includes the following steps:

[0007] S1. The underwater image enhancement model MCF-Net receives the original image and obtains three color space images: RGB, HSV, and Lab. The underwater image enhancement model MCF-Net includes three parallel feature extraction branch networks CFCNN. The three color space images are respectively input into one feature extraction branch network CFCNN.

[0008] S2. The single feature extraction branch network CFCNN receives the image in the corresponding color space and transmits the image in the corresponding color space to each channel stitching layer through a skip connection strategy. Features are extracted through the densely connected convolutional module DCBlock.

[0009] S3. The features extracted by each feature extraction branch network CFCNN are concatenated by channels and then introduced into the attention mechanism module CBAM for adaptive feature selection and weighting. Finally, image enhancement is achieved through convolution operations, and the results are output.

[0010] S1 includes the following sub-steps:

[0011] S1-1: Receive the original image and perform normalization processing;

[0012] S1-2. Obtain HSV color space images and Lab color space images based on the RGB color space;

[0013] S1-3. Input the images of different color spaces into the corresponding feature extraction branch network CFCNN.

[0014] The original image received in S1-1 is cropped to 620×460 pixels, and the number of channels of the original image is 3.

[0015] S2 includes the following sub-steps:

[0016] S2-1. In the feature extraction branch network CFCNN, the image corresponding to the color space is connected to the convolution module DCBlock. The image corresponding to the color space is directly transmitted to each channel stitching layer through the skip connection strategy.

[0017] S2-2: The image corresponding to the color space is passed through 3 convolutional modules DCBlock. The output of the convolutional module DCBlock is the concatenation of the channels of the results of each convolutional layer after passing through the activation function.

[0018] S2-3. Map the number of channels in the feature map to 32, which will be used as the final output of the feature extraction branch network CFCNN.

[0019] In S2-2, a single convolutional module DCBlock performs convolution operations on the image sequentially using 7x7, 5x5, and 3x3 convolutional kernels. After each convolution operation, a non-linearity is introduced through the Leaky ReLU activation function. The output of the convolutional module DCBlock is the concatenation of the channels of the activation function outputs of each convolutional layer.

[0020] The images in the corresponding color space in S2-3 are concatenated by three convolutional modules DCBlock and their corresponding channels. Then, the number of channels in the feature map is mapped to 32 through a convolutional layer with a 1 x 1 convolutional kernel, which serves as the final output of the feature extraction branch network CFCNN.

[0021] S3 includes the following sub-steps:

[0022] S3-1: Integrate multi-dimensional feature information mined from different color spaces through a channel splicing layer;

[0023] S3-2. By introducing the attention mechanism module CBAM, adaptive feature selection and weighting are achieved;

[0024] S3-3. Further fuse the attention-weighted features using successive convolution operations, and map the number of feature map channels back to 3 to output the enhanced underwater image.

[0025] The attention mechanism module CBAM includes a cascaded channel attention CAM and a spatial attention SAM. S3-2 includes the following sub-steps:

[0026] S3-2-1. Channel Attention CAM uses global average pooling and global max pooling to capture the global context information of the feature map and inputs this information into a shared multilayer perceptron to learn the importance weights of each channel. Channel Attention CAM can assist the MCF-Net model of this invention in dynamically identifying and enhancing the feature channels that are most sensitive to underwater scenes and make the most significant contributions.

[0027] S3-2-2, Spatial Attention (SAM) receives the channel-weighted feature map output by Channel Attention (CAM), aggregates channel information by performing average pooling and max pooling in the channel dimension, and further generates a spatial attention map to help MCF-Net focus on the most degraded areas in underwater images.

[0028] S3-3 includes the following sub-steps:

[0029] S3-3-1. Further feature fusion is performed using 3x3 convolutional kernels to enable the extracted key feature information to interact and combine more non-linearly in the spatial dimension, allowing MCF-Net to learn more complex local patterns and contextual relationships.

[0030] S3-3-2. Features under different attention weights are smoothly integrated to ensure that the feature representations are highly refined and coordinated before being passed to the final output layer.

[0031] S3-3-3: 1x1 convolutional kernels are used to reduce the dimensionality of the feature map channels, mapping the channel dimension back to 3. The sigmoid activation function ensures that the pixel values ​​of the output image are between [0, 1]. The two final convolutional layers work together to ensure the generation of a high-quality, detailed, and naturally colored enhanced image.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] This invention proposes a feature extraction branch network CFCNN based on dense reuse and skip connections. By adopting a dense connection strategy, it realizes the direct reuse of deep and shallow features of the network, which significantly enhances the information flow transmission efficiency and effectively alleviates problems such as gradient vanishing. This ensures that the feature extraction branch network CFCNN can fully capture the complex textures and details of underwater images in various color spaces.

[0034] The feature extraction branch network CFCNN also introduces a cross-level skip connection strategy, fusing feature maps of different depths. This promotes the effective integration of global contextual information and local detail features, which is beneficial for recovering degradation problems such as reduced contrast and color distortion caused by uneven illumination and scattering effects in underwater environments. Through this unique synergy of dense reuse and multi-level skip connections, the feature extraction branch network CFCNN can adaptively learn and extract multi-level features of various color spaces in underwater images and efficiently feed them into the subsequent image reconstruction module.

[0035] This invention proposes an underwater image enhancement model, MCF-Net, based on multi-layer color features. It constructs an end-to-end enhancement network that deeply integrates multi-scale color priors and attention mechanisms to accurately recover color and detail information from underwater images. The MCF-Net model extracts color information at different levels in the RGB, HSV, and Lab color spaces of underwater images through three parallel feature extraction branches (CFCNN), laying the foundation for subsequent refined enhancement.

[0036] By introducing a dual attention mechanism module (CBAM), weights are dynamically assigned to different color channels and spatial regions. This allows the underwater image enhancement model MCF-Net to adaptively focus on color components and regions most affected by the underwater environment, thereby achieving targeted color restoration and contrast enhancement. Through a carefully designed feature fusion strategy, MCF-Net effectively integrates multi-layered color features with attention-weighted features, ensuring a synergistic improvement in global color consistency and local detail clarity. This effectively overcomes color distortion and detail blurring caused by water absorption and scattering.

[0037] In the overall design process of the underwater image enhancement model MCF-Net, this invention adopts convolutional kernels of various sizes, including 7x7, 5x5, 3x3 and 1x1. This design of multi-size convolutional kernels working together enables the underwater image enhancement model MCF-Net to capture multi-level features from large-scale contextual information to local fine textures, effectively addressing various complex degradation problems that may occur in underwater images.

[0038] The underwater image enhancement model MCF-Net does not employ downsampling throughout the feature extraction and enhancement process. Instead, it ensures that the feature map size remains constant across all stages by padding. This design maximizes the preservation of the original spatial information and high-frequency details of the image, fundamentally avoiding the red artifact problem introduced by changes in feature map size in underwater images. Through this strategy, the underwater image enhancement model MCF-Net improves feature representation capabilities while maintaining the structural fidelity and color naturalness of the enhancement results, achieving accurate and efficient enhancement of underwater images. Attached Figure Description

[0039] Figure 1 These are example images from the UIEB dataset of real underwater images;

[0040] Figure 2 This is a schematic diagram of the CFCNN feature extraction branch network architecture;

[0041] Figure 3 This is a schematic diagram of the DCBlock architecture for convolutional modules;

[0042] Figure 4 This is a comparison chart of the activation functions ReLU and Leaky ReLU;

[0043] Figure 5 This is a schematic diagram of the MCF-Net network architecture, an underwater image enhancement model.

[0044] Figure 6 This is a schematic diagram of the CBAM architecture for the attention mechanism module;

[0045] Figure 7 It is a trend chart of the model trained using the training module;

[0046] Figure 8 This is a qualitative comparison of a synthetic underwater image test dataset;

[0047] Figure 9 It is a qualitative comparison of a real underwater image test dataset;

[0048] Figure 10 These are example images from a synthetic underwater image dataset. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] Example 1

[0051] Reference Figures 1-10 An underwater image enhancement system for dense reuse and skip connections of multi-layer color features includes:

[0052] Datasets are used to store underwater image information;

[0053] The CFCNN network consists of three parallel feature extraction branches that process images in RGB, HSV, and Lab color spaces respectively. Each branch extracts features through a densely connected DCBlock module and transmits the original image to each channel stitching layer through skip connections, ultimately outputting a feature map.

[0054] The underwater image enhancement model MCF-Net integrates features extracted from various branches of the feature extraction branch network CFCNN, introduces the attention mechanism module CBAM for adaptive feature selection and weighting, and outputs an enhanced image after feature fusion and channel dimensionality reduction.

[0055] The training module uses MSE loss L MSE Structural similarity loss L SSIM And VGG perceived loss L VGG The weighted combination of values ​​is used as the total loss function to train the underwater image enhancement model MCF-Net.

[0056] The image dataset includes the real underwater image dataset UIEB and the synthetic underwater image dataset;

[0057] The real underwater image dataset UIEB contains multiple real underwater images taken under different lighting conditions;

[0058] The synthetic underwater image dataset uses an underwater degradation physics model to degrade clear land images, synthesize underwater images, and generate the dataset.

[0059] The image dataset stores images with a size of 620×460 pixels.

[0060] Due to the data-driven nature of these models, neural network models rely on a large number of underwater "degraded image-enhanced image" data pairs for training. However, real underwater image datasets with reference enhanced images are relatively scarce. Therefore, this invention combines the real underwater image dataset UIEB with a synthetic underwater image dataset for subsequent training and testing tasks.

[0061] Specifically, the real underwater image dataset UIEB in this embodiment consists of 890 real underwater images taken under different lighting conditions, exhibiting varying color gamuts and contrasts. The synthetic underwater image dataset utilizes an underwater degradation physics model to degrade clear land images, resulting in synthesized underwater images that present visual effects under different underwater conditions. (Refer to...) Figure 1 and Figure 10 The image shows some example images from two underwater image datasets. Figure 1 and Figure 10 Row I contains the original underwater images of the corresponding dataset, and row II contains the labels of the reference augmented images provided for the corresponding dataset.

[0062] As can be seen, the two datasets cover various degradation problems that may occur in underwater environments, such as color distortion and reduced contrast. To balance model training speed and computer hardware performance, the acquired underwater images were cropped to 620×460 pixels.

[0063] In this embodiment, the feature extraction branch network CFCNN is used to extract potential high-level features from various color spaces. Its network architecture is as follows: Figure 2 As shown, the feature extraction branch network CFCNN consists of a set of densely connected convolutional modules (DCBlocks). The input raw image is fed to each channel stitching layer using skip connections. The raw image is a 620×460 pixel underwater image. After convolution with a 3x3 kernel, the number of feature map channels is transformed from 3 to 16. After passing through an activation function, it is connected to three DCBlocks. The last convolutional layer uses a 1x1 kernel to map the number of feature map channels to 32 for subsequent image reconstruction.

[0064] Specifically, Figure 2In this context, Input represents the input, and the feature extraction branch network CFCNN includes multiple convolutional modules DCBlock. Each convolutional module DCBlock includes convolutional layers and activation function layers with convolutional kernels of different sizes.

[0065] Each convolutional module (DCBlock) is connected through a channel concatenation layer, i.e., Concat. The channel concatenation layer contains the output of the current convolutional module (DCBlock) and the output of the previous channel concatenation layer.

[0066] The last convolutional module, DCBlock, is connected to a convolutional layer with a 1 x 1 kernel through a channel splicing layer, namely CONV-1.

[0067] Reference Figure 3 The blue squares represent convolutional layers. A single convolutional module DCBlock includes convolutional layers with three kernel sizes: CONV-7, CONV-5, and CONV-3. These kernel sizes represent 7x7, 5x5, and 3x3, respectively, to achieve progressive receptive field expansion, thereby enabling hierarchical capture of the inherent multi-scale degradation features in underwater images.

[0068] Each convolutional layer is connected to an activation function layer, which uses the Leaky ReLU activation function, i.e., LReLU;

[0069] The output of the convolution module DCBlock is a concatenation of the channels of the results of each convolutional layer after passing through the activation function.

[0070] Compared to the widely used ReLU activation function, the Leaky ReLU activation function effectively alleviates the problem of neuron death. The ReLU activation function directly sets all negative input values ​​to zero, which causes some neurons to be permanently inactivated during training, unable to respond to any input, thus losing information and hindering gradient propagation.

[0071] The Leaky ReLU activation function, on the other hand, assigns a small, non-zero positive slope to negative input values. Leaky ReLU ensures that neurons can still generate gradients when the input is negative, thus ensuring a more stable gradient flow in the model. A comparison of the two activation functions is shown in the figure below. Figure 4 As shown.

[0072] This embodiment incorporates the idea of ​​a dense connection strategy into the design of the entire feature extraction branch network CFCNN, which is manifested as follows: Figure 2 and Figure 3The brown lines in the diagram represent the dense connection strategy for the DCBlock convolutional module, which constitutes the feature extraction branch network CFCNN. The dense connection strategy involves concatenating the channels of each convolutional layer's output after the activation function. This approach not only promotes deep feature extraction and efficient reuse but also strengthens local information flow, ensuring that effective details learned within a unit are transmitted without attenuation, thus generating a more refined feature representation of local color degradation patterns. In the macroscopic design of the CFCNN feature extraction branch network, the dense connection strategy involves the channel concatenation layer encompassing not only the output of the current convolutional module DCBlock but also the output of the previous channel concatenation layer. This design achieves progressive enhancement of global contextual features, improves gradient propagation, and enhances the stability and efficiency of network training.

[0073] The feature extraction branch network CFCNN consists of a large number of convolutional layers. To avoid the gradient vanishing problem as much as possible, this invention adopts a skip connection strategy, continuously transmitting the original input image to each channel concatenation layer, which manifests as follows: Figure 2 The green lines in the image represent skip connections that assign greater weights to channels associated with the original input image, ensuring that the model can learn new features from each processing step.

[0074] Reference Figure 5 and Figure 6 The attention mechanism module CBAM includes a cascaded channel attention CAM and a spatial attention SAM;

[0075] The attention mechanism module CBAM sequentially connects two sets of convolutional layer-activation function layers. One set of convolutional layer-activation function layers consists of convolutional layers with 3x3 kernels, corresponding to the LReLU activation function. The other set consists of convolutional layers with 1x1 kernels, corresponding to the Sigmoid activation function.

[0076] The attention mechanism module CBAM used in this invention aims to address the problems of information redundancy and insufficient discriminative power after multi-source feature fusion, enabling the model to adaptively focus on key information that enhances underwater images. Its network architecture is as follows: Figure 6 As shown. CBAM provides stronger feature selection and expressive power by concatenating channel attention (CAM) and spatial attention (SAM). The network architecture of the underwater image enhancement model MCF-Net is as follows. Figure 5 As shown, at the end of the underwater image enhancement model MCF-Net, two sets of convolutional layer-activation function layers are used to achieve the final feature integration and image reconstruction.

[0077] In this embodiment, to train the underwater image enhancement model MCF-Net, a weighted combination of three types of loss functions is used as the total loss function, namely MSE loss. Structural similarity loss and VGG perceived loss Total loss function The calculation formula is as follows:

[0078] ;

[0079] in, , , These are the weight coefficients for the three loss functions.

[0080] MSE loss Underwater enhanced image I and reference image I output by the computational model * The sum of the squared differences between pixels is used to ensure that the enhanced image output by the model retains edges and details. The calculation formula is as follows:

[0081] ;

[0082] Structural similarity loss It can effectively measure the perceptual similarity between the underwater augmented image output by the model and the reference image. The SSIM value is typically calculated on a grayscale image to focus on the image's brightness, contrast, and structural information, reducing the interference of color bias. Specifically, for each pixel in the image... The corresponding SSIM value is evaluated within a 13×13 pixel block in its neighborhood, and the calculation formula is as follows:

[0083] ;

[0084] in, and The augmented graph output by the model is based on The mean and standard deviation of the pixel block centered on it. Correspondingly, and For reference image The mean and standard deviation of the pixel block centered on the target. Let be the covariance between the two types of pixel blocks mentioned above. and To avoid small constants that could cause division by zero errors, we set the following value here: =0.02, =0.03. Ultimately, the SSIM loss is... The calculation formula is as follows:

[0085] ;

[0086] VGG perceived loss The aim is to evaluate the similarity of images by comparing their representations in a deep feature space, thereby better capturing the high-level semantic information perceived by the human visual system. Specifically, a VGG-19 network pre-trained on the ImageNet dataset is used as the feature extractor. When calculating the perceptual loss, the enhanced image I output by the model is compared with the reference image I. * The images are input into a VGG-19 network, and feature maps from specific convolutional layers are extracted as depth feature representations of these two images. The VGG perceptual loss is defined as the Euclidean distance between these two feature representations, calculated as follows: This is the j-th convolutional layer of the pre-trained network VGG-19.

[0087] .

[0088] In the three different types of loss functions, the subscript i represents the pixel value of the i-th pixel in the image.

[0089] Compared to terrestrial images, underwater images exhibit more widespread and complex color deviations, ranging from classic bluish or greenish tints to less common yellowish tints. This diverse color distortion caused by selective absorption and scattering in water makes it difficult for network architectures using a single color space to comprehensively capture and correct all types of degradation.

[0090] Inspired by the traditional approach of targeted processing in different color spaces in image enhancement algorithms, this invention employs three parallel color space feature extraction branches of the CFCNN network to perform deep feature mining in three representative color spaces: RGB, HSV, and Lab. The RGB color space, directly corresponding to human perception and display devices, possesses clear physical meaning, facilitating image storage and display. However, due to the high correlation between its R, G, and B components, its features are easily affected by factors such as lighting changes, occlusion, and shadows, making it difficult to separate pure color information. The HSV color space decouples color information into hue, saturation, and brightness, more intuitively reflecting the color essence, vividness, and overall brightness of an image, which is highly valuable for adjusting the color saturation and contrast of underwater images. The Lab color space exhibits perceptual uniformity, where the L component represents brightness, and the a and b components represent color-opposite dimensions (green-red and blue-yellow). This design results in a more uniform color distribution, covering the entire color range perceptible to the human eye, and its brightness is relatively independent of color information, facilitating brightness-independent color correction.

[0091] Given the unique advantages of these three color spaces and their ability to represent different levels of color information, the underwater image enhancement model MCF-Net designed in this invention integrates these complementary characteristics into a unified deep learning framework. This allows it to comprehensively capture rich information related to underwater image degradation from multiple dimensions, including color, hue, saturation, and brightness. Furthermore, considering the varying sensitivities of different color spaces to color differences—that is, a small color difference between two pixels in one color space may be very significant in another—MCF-Net's multi-color-space parallel processing mechanism can more effectively quantify and perceive subtle color deviations in underwater images.

[0092] Specifically, the underwater image enhancement system for dense reuse and skip connections of multi-layer color features includes the following steps during image augmentation:

[0093] S1. The underwater image enhancement model MCF-Net receives the original image and obtains three color space images: RGB, HSV, and Lab. The underwater image enhancement model MCF-Net includes three parallel feature extraction branch networks CFCNN. The three color space images are respectively input into a feature extraction branch network CFCNN to obtain the hidden high-level color features.

[0094] S1 includes the following sub-steps:

[0095] S1-1: Receive the original image and perform normalization processing; the original image received in S1-1 is cropped to 620×460 pixels, and the number of channels of the original image is 3 at this time.

[0096] S1-2. Obtain HSV color space images and Lab color space images based on the RGB color space;

[0097] S1-3. Input the images of different color spaces into the corresponding feature extraction branch network CFCNN.

[0098] S2. The single feature extraction branch network CFCNN receives the image in the corresponding color space and transmits the image in the corresponding color space to each channel stitching layer through a skip connection strategy. Features are extracted through the densely connected convolutional module DCBlock.

[0099] S2 includes the following sub-steps:

[0100] S2-1. In the feature extraction branch network CFCNN, the image corresponding to the color space is connected to the convolution module DCBlock. The image corresponding to the color space is directly transmitted to each channel stitching layer through the skip connection strategy.

[0101] S2-2: The image in the corresponding color space is processed by three convolutional modules (DCBlock). The output of each DCBlock is a concatenation of the channels of the activation functions of each convolutional layer. In S2-2, each DCBlock performs convolution operations on the image with 7x7, 5x5, and 3x3 kernels in sequence. After each convolution operation, a non-linearity is introduced through the Leaky ReLU activation function. The output of each DCBlock is a concatenation of the channels of the activation functions of each convolutional layer.

[0102] For a single convolutional module DCBlock, the image is sequentially convolved with kernels of 7x7, 5x5, and 3x3, achieving progressive expansion of the receptive field, thereby enabling the hierarchical capture of the inherent multi-scale degradation features in underwater images.

[0103] Furthermore, the initial large 7x7 convolutional kernel can capture a wide range of contextual information and global color distribution, which is beneficial for the model to understand the degradation process of the overall scene.

[0104] The subsequent 5x5 convolutional kernel effectively perceives regional color deviations and mesoscale structural features;

[0105] Finally, the 3x3 convolution kernel focuses on extracting fine local textures and edge details.

[0106] This embodiment uses the Leaky ReLU activation function to introduce nonlinearity after the convolution operation, ultimately improving the model's performance.

[0107] S2-3. Map the number of channels in the feature map to 32, which is used as the final output of the feature extraction branch network CFCNN. In S2-3, the image corresponding to the color space is processed by three convolutional modules DCBlock and channel concatenation, and then the number of channels in the feature map is mapped to 32 through a convolutional layer with a 1 x 1 convolutional kernel, which is used as the final output of the feature extraction branch network CFCNN.

[0108] S3. The features extracted by each feature extraction branch network CFCNN are concatenated by channels and then introduced into the attention mechanism module CBAM for adaptive feature selection and weighting. Finally, image enhancement is achieved through convolution operations, and the results are output.

[0109] S3 includes the following sub-steps:

[0110] S3-1. For the latent features mined by the feature extraction branch network CFCNN, the multi-dimensional feature information mined from different color spaces is integrated through the channel splicing layer.

[0111] S3-2. For the concatenated rich feature set, an attention mechanism module CBAM is introduced to achieve adaptive feature selection and weighting. The attention mechanism module CBAM includes a cascaded channel attention CAM and a spatial attention SAM. S3-2 includes the following sub-steps:

[0112] S3-2-1. Channel Attention CAM uses global average pooling and global max pooling to capture the global context information of the feature map and inputs this information into a shared multilayer perceptron to learn the importance weights of each channel. Channel Attention CAM can help the MCF-Net model of this invention to dynamically identify and enhance the feature channels that are most sensitive to underwater scenes (such as specific color shifts, saturation loss or uneven brightness) and make the most significant contributions.

[0113] S3-2-2, Spatial Attention (SAM) receives the channel-weighted feature map output by Channel Attention (CAM), aggregates channel information by performing average pooling and max pooling along the channel dimension, and further generates a spatial attention map to assist the underwater image enhancement model MCF-Net in focusing on the most severely degraded areas in the underwater image.

[0114] S3-3: Further fuse the attention-weighted features using successive convolutional operations, and map the feature map channel number back to 3 to output the enhanced underwater image. S3-3 includes the following sub-steps:

[0115] S3-3-1. For the feature maps refined from the dual attention mechanism module CBAM, further feature fusion is performed through 3x3 size convolution kernels to promote more full interaction and non-linear combination of the extracted key feature information in the spatial dimension, so that MCF-Net can learn more complex local patterns and contextual relationships.

[0116] S3-3-2. Features under different attention weights are smoothly integrated to ensure that the feature representations are highly refined and coordinated before being passed to the final output layer.

[0117] S3-3-3: 1x1 convolutional kernels are used to reduce the dimensionality of the feature map channels, mapping the channel dimension back to 3. The Sigmoid activation function ensures that the pixel values ​​of the output image are between [0, 1]. The two final convolutional layers work together to ensure the generation of a high-quality, detailed, and naturally colored enhanced image.

[0118] Example 2

[0119] Building upon Example 1, this embodiment, in order to train the underwater image enhancement model MCF-Net, randomly selected 800 pairs of underwater images from the real underwater image dataset UIEB. The UIEB dataset comprises a total of 890 real underwater images and corresponding reference images. Furthermore, this invention also integrates 1250 synthetic underwater images from a synthetic underwater image dataset, covering 10 water body types. Therefore, a total of 2050 "underwater image-reference image" pairs were selected as the training dataset for the underwater image enhancement model MCF-Net, and 990 underwater images with different water body characteristics were selected from the two datasets as test data. To balance hardware performance and model efficiency, the images in the training dataset were cropped to 620×460 pixels.

[0120] This embodiment implements the training and testing of the underwater image enhancement model MCF-Net on the PyTorch platform. During model training, the Adam optimizer is used for parameter optimization, with an initial learning rate set to 0.0001. The two momentum parameters of the Adam optimizer are set to 0.9 and 0.999, respectively. A cosine annealing learning rate adjustment strategy is adopted, where the maximum learning rate is 0.1, the minimum is 0.0001, and the learning rate adjustment period is 200. The weight coefficients of each loss function are all set to 1. Furthermore, the total number of training iterations is set to 100 throughout the entire training process. To ensure the traceability of the training process, the current training parameters of the model are saved every 5 rounds of training. In addition, to monitor the learning progress and enhancement effect of the model in real time, the enhancement results are visualized on the images in the test set every 500 iterations during training.

[0121] Figure 5 This paper presents the overall network architecture of the underwater image enhancement model MCF-Net proposed in this embodiment. The feature extraction branch network CFCNN, which extracts latent feature information from each color space, plays a crucial role in the entire underwater image enhancement model MCF-Net. To visualize the transformation process of the input image throughout the entire MCF-Net, Tables 1 and 2 provide detailed structural parameters of the feature extraction branch network CFCNN and the complete underwater image enhancement model MCF-Net, respectively.

[0122] Table 1. Structural parameters of the feature extraction branch network CFCNN

[0123]

[0124] Table 2. Structural parameters of the underwater image enhancement model MCF-Net

[0125]

[0126] Image enhancement evaluation metrics are typically based on the natural statistical properties of images. To quantitatively evaluate the performance of the proposed underwater image enhancement model MCF-Net, this embodiment employs two full-reference evaluation metrics and three non-reference evaluation metrics. For full-reference evaluation metrics, this embodiment uses Peak Signal-to-Noise Ratio (PSMR) and Structural Similarity (SSIM). For non-reference evaluation metrics, this embodiment uses the Underwater Image Quality Metric (UIQM), the Underwater Color Image Quality Evaluation Metric (UCIQE), and Entropy. Furthermore, to comprehensively evaluate the proposed underwater image enhancement model MCF-Net, this invention compares the proposed method with algorithms such as CLAHE, DCP, GC, ICM, UCM, UDCP, and ULAP. The underwater image enhancement methods used for comparison include those based on traditional algorithm engineering and those based on physical models, representing a significant portion of the underwater image enhancement field.

[0127] Figure 8 and Figure 9 The enhancement effects of this application and seven comparative methods on synthetic underwater test datasets and real underwater test datasets are shown respectively. (a) and (b) represent the original underwater image and the reference enhanced image labels provided by the dataset, respectively. (c), (d), (e), (f), (g), (h), and (i) represent the enhancement effects using CLAHE, DCP, GC, ICM, UCM, UDCP, and ULAP algorithms, respectively. (j) shows the enhancement effect using this application. The synthetic underwater image dataset achieves degradation processing of high-quality land images through an underwater degradation physical model. Its advantage lies in the ability to obtain targeted underwater degradation images by adjusting the corresponding parameters. Figure 8 The three test images shown represent common degradation problems in underwater environments, such as reduced brightness, bluish tint, and greenish tint. However, for these combined degradation problems, comparative methods struggle to address them simultaneously. Algorithms like CLAHE, GC, and ICM significantly improve image brightness and solve the brightness reduction problem, but they are ineffective against color distortion. Algorithms like DCP and UDCP, due to limitations in their application scenarios, even have the opposite effect, resulting in poorer visualization of the enhanced results. In contrast, the underwater image enhancement model MCF-Net proposed in this invention can solve the combined degradation problems in synthetic image datasets. It can be seen that the enhanced image eliminates color distortion while preserving its original color characteristics, and also effectively compensates for image brightness, achieving the best overall visual effect. Compared to synthetic underwater image datasets, Figure 9The dataset of real underwater images presented reflects the true state of the underwater environment. It can be seen that all three test images exhibit varying degrees of color distortion. However, the overall performance of the comparative methods remains poor; algorithms such as UCM and ULAP exhibit overcompensation, resulting in red artifacts in the enhanced images. In contrast, this invention performs well, producing highly visible enhanced underwater images with a high degree of similarity to the reference enhanced image labels.

[0128] To further verify the performance of the proposed underwater image enhancement model MCF-Net, the five evaluation metrics mentioned above were used to quantitatively evaluate 990 images in the test set. Furthermore, to better reflect the versatility of the enhancement method and reduce the impact of extreme data, the 10 best and 10 worst data sets were removed, and the remaining results were averaged. The test results are shown in Table 3. The upward arrows on the right side of the evaluation metrics indicate that the evaluation result is directly proportional to the numerical value; that is, the larger the value, the better the enhancement effect is considered.

[0129] Table 3. Comparison of objective metrics for each method on the test dataset.

[0130]

[0131] Table 3 shows the objective performance comparison results of the proposed method and the seven comparison algorithms mentioned above on the test dataset. PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) are used as reference evaluation metrics, requiring comparison with the reference image provided in the test dataset to obtain evaluation scores. PSNR scores based on the error and color difference between the compared images, while SSIM scores based on brightness, contrast, and overall structure. The data in the table shows that the proposed method achieved the highest scores in both PSNR and SSIM, indicating that the enhanced image is close to the reference image in terms of color distribution, brightness distribution, and contrast distribution, fully demonstrating the high versatility of the underwater image enhancement model MCF-Net in underwater image enhancement.

[0132] Furthermore, Entropy, UIQM (Underwater Image Quality Metric), and UCIQE (Underwater Color Image Quality Assessment), as no-reference evaluation metrics, can directly use mathematical formulas to evaluate the quality of restored underwater images. This also means that no-reference evaluation metrics can only evaluate certain features of the enhanced image. Specifically, Entropy measures the amount of information contained in the image, UIQM measures the contrast, color, and sharpness of the image, and UCIQE measures the chroma, saturation, and contrast of the image. According to the data in the table, the MCF-Net proposed in this invention achieved the highest scores in UCIQE and Entropy, but performed poorly in UIQM. However, this result is consistent with... Figure 8 , Figure 9 The significant discrepancies in the displayed visualization results fully demonstrate that relying solely on a single reference evaluation indicator cannot directly assess the quality of enhanced images. A comprehensive evaluation of various indicators and subjective assessments is necessary for a holistic evaluation of the enhanced image quality. Therefore, based on multi-faceted evaluations, it is evident that this invention effectively addresses the inherent problems of underwater images, exhibiting excellent enhancement effects and versatility.

[0133] The above description is merely an optional embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the content of the present invention under the concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. An underwater image enhancement method based on dense reuse and skip connections for multi-layer color features, characterized in that, Includes the following steps: S1. The underwater image enhancement model MCF-Net receives the original image and obtains three color space images: RGB, HSV, and Lab. The underwater image enhancement model MCF-Net includes three parallel feature extraction branch networks CFCNN. The three color space images are respectively input into one feature extraction branch network CFCNN. S2. The single feature extraction branch network CFCNN receives the image in the corresponding color space, and transmits the image in the corresponding color space to each channel concatenation layer through a skip connection strategy, and extracts features through the densely connected convolutional module DCBlock; S2 includes the following sub-steps: S2-1. In the feature extraction branch network CFCNN, the image corresponding to the color space is connected to the convolution module DCBlock. The image corresponding to the color space is directly transmitted to each channel stitching layer through the skip connection strategy. S2-2. The image corresponding to the color space is passed through 3 convolutional modules DCBlock. The output of the convolutional module DCBlock is the channel concatenation of the results of each convolutional layer after passing through the activation function. S2-3. Map the number of channels in the feature map to 32, and use it as the final output of the feature extraction branch network CFCNN; S3. The features extracted by each feature extraction branch network CFCNN are concatenated through channels and then introduced into the attention mechanism module CBAM for adaptive feature selection and weighting. Image enhancement is then achieved through convolution operations, and the result is output. The attention mechanism module CBAM includes a cascaded channel attention CAM and spatial attention SAM. S3 includes the following sub-steps: S3-1: Integrate multi-dimensional feature information mined from different color spaces through a channel splicing layer; S3-2. By introducing the attention mechanism module CBAM, adaptive feature selection and weighting are achieved. S3-3. Further fuse the attention-weighted features using continuous convolution operations, and map the number of feature map channels back to 3 to output the enhanced underwater image.

2. The underwater image enhancement method based on dense multiplexing and skip connections for multi-layer color features according to claim 1, characterized in that, S1 includes the following sub-steps: S1-1: Receive the original image and perform normalization processing; S1-2. Obtain HSV color space images and Lab color space images based on the RGB color space; S1-3. Input the images in different color spaces into the corresponding feature extraction branch network CFCNN.

3. The underwater image enhancement method based on dense multiplexing and skip connections for multi-layer color features according to claim 2, characterized in that, The original image received in S1-1 is cropped to 620×460 pixels.

4. The underwater image enhancement method based on dense multiplexing and skip connections for multi-layer color features according to claim 1, characterized in that, In S2-2, a single convolutional module DCBlock performs convolution operations on the image sequentially using 7x7, 5x5, and 3x3 convolutional kernels. After each convolution operation, a non-linearity is introduced through the Leaky ReLU activation function. The output of the convolutional module DCBlock is the concatenation of the channels of the activation function outputs of each convolutional layer.

5. The underwater image enhancement method based on dense multiplexing and skip connections for multi-layer color features according to claim 4, characterized in that, The images in the corresponding color space in S2-3 are processed by three convolutional modules (DCBlock) and channel concatenation. Then, the number of channels in the feature map is mapped to 32 through a convolutional layer with a 1 x 1 convolutional kernel, which serves as the final output of the feature extraction branch network CFCNN.

6. The underwater image enhancement method based on dense multiplexing and skip connections for multi-layer color features according to claim 1, characterized in that, S3-2 includes the following sub-steps: S3-2-1, Channel Attention CAM uses global average pooling and global max pooling to capture global contextual information of the feature map and inputs this information into a shared multilayer perceptron to learn the importance weights of each channel; S3-2-2. Spatial Attention (SAM) receives the channel-weighted feature map output by Channel Attention (CAM), and aggregates the channel information by performing average pooling and max pooling on the channel dimension to generate a spatial attention map.

7. The underwater image enhancement method based on dense multiplexing and skip connections for multi-layer color features according to claim 1, characterized in that, S3-3 includes the following sub-steps: S3-3-1. Further feature fusion is performed using 3x3 convolutional kernels to enable the extracted key feature information to interact and combine more non-linearly in the spatial dimension, allowing MCF-Net to learn more complex local patterns and contextual relationships. S3-3-2. Features under different attention weights are smoothly integrated to ensure that the feature representations are highly refined and coordinated before being passed to the final output layer. S3-3-3: Use a 1x1 convolution kernel to reduce the dimensionality of the feature map channels, mapping the channel dimension back to 3, and use the sigmoid activation function to ensure that the pixel values ​​of the output image are between [0, 1].

Images