Improved underwater image enhancement method
Through the underwater image enhancement method of lightweight convolution and three-stage attention mechanism, the problems of color distortion and blurred details in the prior art are solved, and efficient underwater image enhancement is achieved, suitable for a variety of underwater scenes.
Patent Information
- Application Number
- CN202510581743.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-07
- Publication Date
- 2025-08-05
AI Technical Summary
The existing underwater image enhancement methods have color distortion, blurred details, and large model parameters, which cannot meet the real-time requirements and are limited in adaptability.
The lightweight convolution module is used for feature extraction, combined with the three-stage attention mechanism and multi-path feature reconstruction, and a clear and natural enhanced image is generated through multi-stage output fusion.
The model parameters and calculation complexity are significantly reduced, and the problems of suspended particles, such as underwater images, such as blurring of edges, and color distortion, are accurately repaired, and the enhancement effect is more targeted, solving the problems of artifacts such as light spots and excessive smoothing.
Smart Images

Figure CN120430963A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image enhancement, and in particular to an improved underwater image enhancement method. Background Art
[0002] Underwater imaging analysis is crucial for marine resource exploration, marine ecological research, deep-sea facility monitoring, and naval military applications. However, facing relatively chaotic ocean scenes presents many serious disadvantages, including high water turbidity, uneven illumination, monotonous colors, and complex underwater backgrounds. These factors severely impact the integrity and usability of underwater images in practical applications. These quality issues significantly limit the effectiveness of underwater images in real-world scenarios.
[0003] In past studies, researchers have conducted many explorations into underwater image enhancement techniques, which can be roughly divided into traditional methods and deep learning-based methods. For traditional methods, various methods based on physical imaging models are the main representatives. However, the applicability of physical model-based methods to various underwater degradations is not satisfactory. In addition, directly modifying pixel values to improve the visual quality of images is another widely studied traditional method (such as color balancing, histogram stretching, and contrast improvement). However, overcorrection and oversaturation are the main problems of these methods, which may even damage the already fragile visual quality.
[0004] With advances in deep learning, its applications have significantly expanded to the field of underwater image enhancement (UIE). In recent years, scholars have proposed using deep learning-based methods to address the problem. Current methods for underwater image enhancement include convolutional neural networks (CNNs) and generative adversarial networks (GANs). CNNs have shortcomings in capturing global information and extracting features when processing images, which limits their performance to a certain extent. While the visual self-attention model (Vision Transformer, ViT) excels at extracting global information from images, it often neglects local information. Furthermore, deep convolutional neural networks have high computational and memory requirements, hindering their widespread adoption and practical applications. A range of ViTs using attention mechanisms often suffer from excessive computation and a large number of parameters. The generative adversarial network (WaterGAN) is used to generate realistic underwater images from pairs of aerial images and depth maps, enabling real-time color correction of underwater images. Although WaterGAN performs well in controlled environments and real-world datasets, its adaptability to specific scenarios is limited and may require separate training for different environments to improve the model's generalization ability. Summary of the Invention
[0005] In response to the problems of color distortion, blurred details and generally large number of model parameters in existing underwater image enhancement, which cannot meet the real-time requirements during underwater missions, the present invention proposes a lightweight and improved underwater image enhancement method that can restore image details.
[0006] An improved underwater image enhancement method comprises the following steps:
[0007] A. Lightweight downsampling and feature extraction
[0008] The underwater image is input into a lightweight convolution module, which extracts local details layer by layer into global semantic features. The spatial size is gradually compressed through maximum pooling, retaining key features and reducing computational complexity, providing an efficient multi-scale foundation for subsequent enhancement.
[0009] B. Multi-stage attention enhancement and feature optimization
[0010] Through the three-stage attention mechanism, features are enhanced layer by layer, and residual fusion is combined to retain the original low-frequency information.
[0011] C. Multipath feature reconstruction
[0012] A four-column nested encoder-decoder path is constructed through the dense skip connection architecture of the medical image segmentation model UNet++. Each decoding node integrates feature information from different paths to form progressive feature reconstruction.
[0013] D. Multi-level output fusion and image restoration
[0014] Enhanced predictions are generated from each of the four decoded depths, and depth supervision is achieved by averaging or selecting specific outputs. Finally, clear and natural enhanced underwater images are generated.
[0015] Furthermore, the steps of the lightweight convolution downsampling are as follows:
[0016] A1. Input the original underwater image to be enhanced into the first-layer lightweight convolution module. This module extracts spatial features from the input image using depthwise separable convolution. The depthwise convolution layer independently captures 3×3 local features for each channel. The point-by-point convolution layer further adjusts the channel dimensions and integrates cross-channel information. After processing by the first-layer lightweight convolution module, a first-layer feature map containing 64 channels is generated, where each channel corresponds to a feature response at a different level of abstraction. The underwater image is in RGB three-channel format.
[0017] A2. Perform a max pooling downsampling operation on the first-layer feature map. In this operation, a pooling window of size 2×2 is slid across the feature map with a step size of 2. Each time, the maximum value among the four pixels within the window is selected as the output. This operation reduces the spatial size of the feature map to 1 / 2 of the original image, resulting in a secondary feature map. The spatial size of the feature map includes height and width.
[0018] A3. Input the secondary feature map into the second-layer lightweight convolution module. This second-layer lightweight convolution module uses the depthwise separable convolution architecture. The depthwise convolution layer further exploits high-level semantic features, and the pointwise convolution layer expands the number of channels to 128. This step generates the second-layer feature map. The high-level semantic features include object outlines and area distribution.
[0019] A4: Repeat the max pooling operation to downsample the second-layer feature map. The pooling window traverses the second-layer feature map with the same parameters as step A2, and the output size is further reduced to 1 / 4 of the original image to obtain the third-layer feature map.
[0020] A5. Input the third-layer feature map into the third-layer lightweight convolution module, and extract 256-channel deep features through depth-wise separable convolution. The size of the feature map of this layer is reduced to 1 / 8 of the original image.
[0021] A6. The reduced feature map is fed into the fourth-layer lightweight convolution module. The input features and the convolution output are added pixel-wise via residual connections to generate 512-channel high-order features. After downsampling, the feature map size is reduced to 1 / 16 of the original image. Finally, the feature map is fed into the fifth-layer lightweight convolution module, which outputs a 1024-channel coding layer feature map.
[0022] Furthermore, the three stages of the three-stage attention mechanism in step B include a local noise suppression stage, a spatial-channel collaborative optimization stage, and a global color shift compensation stage. The three-stage attention mechanism enhances features through the three-stage attention operations of local noise suppression, spatial-channel collaborative optimization, and global color shift compensation. The specific steps are as follows:
[0023] B1. The output feature map of each layer of lightweight convolution module is input to the local noise suppression stage. Global statistical information of each channel is extracted through global average pooling. Adaptive channel weights are generated using one-dimensional convolution to dynamically suppress the noise channel. The channel weights are multiplied by the input feature map channel by channel to output the feature map of the local noise suppression stage.
[0024] B2. The output feature map of the local noise suppression stage is input into the space-channel collaborative optimization stage. The statistical distribution information of the spatial dimension is extracted through global average pooling and global maximum pooling. The two pooling results are fused through a dilated convolutional layer to generate spatial attention weights to locate the turbid areas and severely degraded spatial locations with blurred edges. The spatial weights are multiplied pixel by pixel with the feature map of the space-channel collaborative optimization to enhance the detail expression of the key areas and obtain the output feature map of the space-channel collaborative optimization stage.
[0025] B3. The output feature map of the spatial-channel collaborative optimization stage is input into the global color cast compensation stage. The global channel relationship is modeled, and the inter-channel dependencies are learned through the fully connected layer to generate channel excitation weights. Based on the color cast characteristics dominated by blue-green underwater, specific channels are dynamically enhanced or suppressed to compensate for color distortion. The channel excitation weights are multiplied by the input features of the global color cast compensation channel by channel to output the feature map of the global color cast compensation stage.
[0026] B4. Add the feature map output by the global color cast compensation stage and the input feature map by the local noise suppression stage at the pixel level to retain the original low-frequency information and avoid over-smoothing or artifact introduction, thus obtaining the enhanced feature map of the corresponding layer.
[0027] Repeat steps B1 to B4, performing a three-stage attention operation of local noise suppression, spatial-channel co-optimization, and global color cast compensation after the lightweight convolutions in the first to fifth layers. This optimizes the multi-scale feature representation from microscopic details to macroscopic semantics layer by layer until the output feature map of the encoder's fifth layer is obtained.
[0028] Furthermore, the local noise suppression module in step B1 quantizes the output feature map of the lightweight convolution module As input, where Represents a set of real numbers, H, W, and C represent the height, width, and number of channels, respectively. First, global average pooling is performed on each channel to generate a channel descriptor. The c-th element of Z is represented as follows:
[0029]
[0030] Among them, F c (i, j) represents the eigenvalue of the cth channel at the position of the horizontal coordinate i and the vertical coordinate j. After obtaining the channel descriptor, the dependency between local channels is further captured by one-dimensional convolution. Specifically, a one-dimensional convolution with a kernel size of k is used to process the channel descriptor Z to generate the channel attention weight
[0031] s=σ(C1D k (Z))
[0032] Among them, C1D k represents a one-dimensional convolution with a kernel size of k, σ is the Sigmoid activation function, and k is the convolution kernel size, which is adaptively determined according to the number of channels C. The calculation formula for the convolution kernel size k is:
[0033]
[0034] Where γ and b are both hyperparameters, the default value of γ is 2, and the default value of b is 1. |·| odd Finally, the generated channel attention weight s is multiplied by the input feature F channel by channel to obtain the output feature
[0035]
[0036] in, Represents channel-by-channel multiplication.
[0037] In step B2, the spatial-channel collaborative optimization stage converts the output features of the local noise suppression module into As input, the spatial-channel co-optimization stage calculates a one-dimensional channel attention map through the channel branch Compute a 2D spatial attention map via the spatial branch The calculation method is as follows:
[0038] B21. Calculate one-dimensional channel attention map
[0039] The mean and maximum values of the channel dimensions are calculated in parallel, and then fused after compression-expansion by a shared multi-layer perceptron MLP to generate the channel weight W. channel :
[0040] First, the spatial information of the features is aggregated by using average pooling and maximum pooling operations to generate two different spatial context descriptors. and in and Represent the average pooling features and the maximum pooling features respectively:
[0041]
[0042] The two descriptors are then forwarded to the shared network to generate channel attention weights The shared network consists of a multi-layer perceptron (MLP). After applying the shared network to each descriptor, the output feature vectors are merged using element-wise summation. The channel attention map is calculated as follows:
[0043]
[0044] B22. Calculate the two-dimensional spatial attention map
[0045] The spatial relationship of features is used to generate a spatial attention map. First, average pooling and maximum pooling operations are applied along the channel axis and they are concatenated to generate a valid feature descriptor. On the concatenated feature descriptors, a convolutional layer is applied to generate a spatial attention map.
[0046] By using two pooling operations to aggregate the channel information of the feature map, two two-dimensional maps are generated:
[0047]
[0048] and Represent the average pooled features and maximum pooled features across channels respectively. Further, a two-dimensional spatial attention map is generated through standard convolutional layer connection and convolution. Spatial Attention Map The calculation is as follows:
[0049]
[0050] The final output is the element-wise product of the channel and the spatial weight:
[0051]
[0052] in Represents space-wise multiplication.
[0053] The global color shift compensation described in step B3 uses an adaptive compression-excitation strategy: first, the input features are compressed in the channel dimension, and then the channel weights are reconstructed through nonlinear mapping. The compression and excitation methods are as follows:
[0054] B31, Compression
[0055] In order to alleviate the problem of not being able to utilize contextual information in the lower layers of the network with smaller receptive fields, the global spatial information is compressed into the channel descriptor by using global average pooling to generate channel statistics. Formally, the channel descriptor It shrinks the input features by the spatial dimension H×W Generated by , where the cth element of Z is calculated as follows:
[0056]
[0057] B32, Motivation: In order to achieve the goal of fully capturing channel dependencies, channel weights are generated through two fully connected layers and Sigmoid function as follows:
[0058] s=σ(W2·δ(W1·z))
[0059] in, is the weight matrix of the fully connected layer, δ is the ReLU activation function, and σ is the Sigmoid function.
[0060] Finally, the generated channel weight s is multiplied by the input feature map F channel by channel to obtain the output feature map
[0061]
[0062] Furthermore, the multi-scale feature fusion and image reconstruction method described in step C generates the final enhanced image through nested skip connections, multi-level feature fusion and weighted average output. The specific steps are as follows:
[0063] C1. The output feature map of the encoder's fifth layer is upsampled using bilinear interpolation, expanding its spatial size to 1 / 8 of the original image. It is then channel-wise concatenated with the feature map output from the encoder's fourth layer using skip connections. The concatenated feature map is fed into the fourth residual enhancement module, which uses convolution to fuse the deep semantic information of the object's overall contour with the mid-level semantic features of the regional texture distribution to generate a shallow fused feature map.
[0064] C2: The shallow fusion feature map is upsampled twice to restore its size to 1 / 4 of the original image and concatenated with the feature map output by the third layer of the encoder. The concatenated feature map is input into the third residual enhancement module, which fuses the mid-level semantic features with the shallow-level details to generate a mid-level fusion feature map.
[0065] C3: Upsample the mid-level fused feature map to half the original image size and concatenate it with the feature map output by the encoder's second layer. The mixed receptive field convolution operation in the second residual enhancement module combines high-frequency details with mid-frequency structure to generate a high-level fused feature map.
[0066] C4: Upsample the high-level fused feature map one final time to restore it to the original image size and concatenate it with the shallow feature map output by the first encoder layer. Through dense skip connections in the first residual enhancement module, the high-resolution color distribution information is fused with the deep-level correction features to generate the final fused feature map.
[0067] C5: Enhancement results are output from the decoder's shallow, mid, high, and final fusion layers. Feature maps from each layer are mapped to a three-channel image using 1×1 convolution. The output feature maps from all layers are then upsampled to the original image size using bilinear interpolation. Finally, a pixel-level weighted averaging strategy is used to combine the local detail enhancement effects of each layer with the global color correction results to produce an enhanced image with suppressed artifacts and clear details.
[0068] Furthermore, the mathematical expression for achieving size expansion by weighting neighboring pixel values in the upsampling process is:
[0069]
[0070] Among them, w i,j is a bilinear weight to ensure smooth transition of interpolation results. The final output image is a balanced fusion of multi-level results:
[0071]
[0072] in, Represents the upsampling result of the k-th layer output, suppressing the single-layer error accumulation by averaging.
[0073] Furthermore, the steps of multi-level output fusion and image restoration in step D are as follows:
[0074] D1. At the end of the decoder's shallow, mid, high, and final fusion layers, 1×1 convolutions are used to map the feature maps of each layer into a three-channel preliminary enhanced image. Subsequently, bilinear interpolation is performed on the sub-scale preliminary enhanced image to expand its spatial size to the resolution of the original image. This ensures that the enhancements of different layers are spatially aligned, avoiding overlapping artifacts or misaligned details caused by size mismatch.
[0075] D2. Perform pixel-level weighted averaging fusion on the four aligned enhanced images.
[0076] D3. During the training process, we only rely on the L2 loss function to supervise the multi-level output, constrain the pixel-level difference between the multi-level output and the true label, and optimize the parameters to minimize the mean square error.
[0077] Furthermore, the bilinear interpolation upsampling achieves smooth expansion of low-resolution images to high-resolution by weighting adjacent pixel values, and the formula is:
[0078]
[0079] Among them, x and y are the horizontal and vertical coordinates of the target high-resolution image respectively. Indicates rounding down. The weight coefficient is determined by the distance between the target position and the nearest four integer pixels. The closer the distance, the higher the weight, ensuring a smooth transition of the interpolation result. Weighted average fusion generates the final image by balancing the enhancement results of each layer. The formula is:
[0080]
[0081] in, represents the interpolation result of the kth level, The full-size output of the final layer is used. The average operation avoids the dominant enhancement effect of a single layer and suppresses overfitting or local artifacts. The L2 loss function constrains the consistency of the multi-level output and the true label through the mean square error. The formula is:
[0082]
[0083] Among them, L2 Loss is the loss value, N is the total number of pixels, I gt For the real label image, the loss function ensures the pixel-level accuracy of the enhanced result by minimizing the squared difference between the prediction and the true value.
[0084] Compared with the prior art, the present invention has the following beneficial effects:
[0085] This paper introduces the UNet++ model to the field of underwater image enhancement for the first time. It replaces the standard convolution in the downsampling layer with lightweight convolution. After each lightweight convolution layer, a three-stage attention mechanism is cascaded: local noise suppression, spatial-channel collaborative optimization, and global color cast compensation. This generates multi-level enhancement results, which are ultimately fused and output using a weighted averaging strategy. This lightweight convolution replacement significantly reduces the number of model parameters and computational complexity, improving computational efficiency.
[0086] 2. This invention uses a three-stage attention mechanism to accurately repair typical degradation problems of underwater images, such as suspended particle noise, edge blur, and color distortion, and the enhancement effect is more targeted.
[0087] 3. The present invention balances global correction and local detail retention through multi-level feature fusion and weighted averaging strategy, solving the common artifact problems such as light spots and over-smoothing problems in existing methods.
[0088] 4. Compared with the existing technology, the present invention shows significant advantages in computational efficiency, detail preservation, color restoration and artifact suppression. It is suitable for various underwater scenes such as low light and high turbidity, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0089] Figure 1 It is a flow chart of the present invention.
[0090] Figure 2 It is a schematic diagram of the overall architecture of the present invention.
[0091] Figure 3 Schematic diagram of the lightweight convolution module of the present invention.
[0092] Figure 4 Schematic diagram of the three-stage attention mechanism of the present invention. DETAILED DESCRIPTION
[0093] The present invention will be further described below with reference to the accompanying drawings.
[0094] like Figure 1 As shown, the present invention is a lightweight underwater image enhancement algorithm that can restore image details, and the specific steps are the same as the invention content.
[0095] like Figure 2 As shown, the overall architecture of the present invention consists of three parts: encoder, skip connection, and decoder.
[0096] The encoder adds a lightweight convolution module to obtain global semantic features and an ECS attention module to obtain enhanced feature maps. The details are respectively in Figure 3 、 Figure 4 The specific steps are the same as those in the invention content.
[0097] like Figure 3 As shown in the figure, the preset lightweight convolution module significantly reduces the number of model parameters and computational complexity by decomposing the traditional convolution operation into three steps: 1×1 convolution → 3×3 depth convolution → 1×1 convolution, while maintaining strong feature extraction capabilities.
[0098] The present invention is not limited to this embodiment, and any equivalent concepts or modifications within the technical scope disclosed by the present invention are included in the protection scope of the present invention.
Claims
1. An improved underwater image enhancement method, characterized in that: The following steps are involved: A. Lightweight downsampling and feature extraction The underwater image is input into a lightweight convolution module, which extracts local details layer by layer into global semantic features. The spatial size is gradually compressed through maximum pooling, retaining key features and reducing computational complexity, providing an efficient multi-scale foundation for subsequent enhancement. B. Multi-stage attention enhancement and feature optimization Through the three-stage attention mechanism, features are enhanced layer by layer, and residual fusion is combined to retain the original low-frequency information; C. Multipath feature reconstruction A four-column nested encoder-decoder path is constructed using the dense skip connection architecture of the medical image segmentation model UNet++. Each decoding node integrates feature information from different paths to form a progressive feature reconstruction. D. Multi-level output fusion and image restoration Enhanced predictions are generated from each of the four decoded depths, and depth supervision is achieved by averaging fusion or selecting specific outputs; ultimately, clear and natural enhanced underwater images are generated.
2. The improved underwater image enhancement method according to claim 1, characterized in that: The steps of the lightweight convolution downsampling are as follows: A1. Input the original underwater image to be enhanced into the first-layer lightweight convolution module. The first-layer lightweight convolution module extracts spatial features from the input image through depthwise separable convolution. The depthwise convolution layer independently captures 3×3 local features for each channel. The point-by-point convolution layer further adjusts the channel dimension and integrates cross-channel information. After processing by the first-layer lightweight convolution module, a first-layer feature map containing 64 channels is generated, where each channel corresponds to a feature response at a different level of abstraction. The underwater image is in RGB three-channel format. A2. Perform a maximum pooling downsampling operation on the first-layer feature map. In this operation, a pooling window is slid on the feature map with a size of 2×2 and a step size of 2. Each time, the maximum value of the four pixels in the window is selected as the output. Through this operation, the spatial size of the feature map is reduced to 1 / 2 of the original image, and a secondary feature map is obtained. The spatial size of the feature map includes height and width. A3. Input the secondary feature map into the second-layer lightweight convolution module. The second-layer lightweight convolution module continues to use the depthwise separable convolution structure. The depthwise convolution layer further mines high-order semantic features, and the point-by-point convolution layer expands the number of channels to 128. Through this step, the second-layer feature map is generated. The high-order semantic features include object contours and area distribution. A4. Repeat the maximum pooling operation to downsample the second-layer feature map. The pooling window traverses the second-layer feature map with the same parameters as step A2, and the output size is further reduced to 1 / 4 of the original image to obtain the third-layer feature map. A5. Input the third-layer feature map into the third-layer lightweight convolution module, and extract the deep features of 256 channels through depth-wise separable convolution. The size of the feature map of this layer is reduced to 1 / 8 of the original image. A6. The reduced feature map is input into the fourth-layer lightweight convolution module. The input features and the convolution output are added pixel-wise through residual connections to generate 512-channel high-order features. After downsampling, the feature map size is reduced to 1 / 16 of the original image. Finally, the feature map is input into the fifth-layer lightweight convolution module, which outputs a 1024-channel coding layer feature map.
3. The improved underwater image enhancement method according to claim 1, characterized in that: The three stages of the three-stage attention mechanism in step B include the local noise suppression stage, the spatial-channel collaborative optimization stage, and the global color shift compensation stage. The three-stage attention mechanism enhances features through the three-stage attention operations of local noise suppression, spatial-channel collaborative optimization, and global color shift compensation. The specific steps are as follows: B1. The output feature map of each layer's lightweight convolution module is input to the local noise suppression stage. Global statistical information of each channel is extracted through global average pooling. Adaptive channel weights are generated using one-dimensional convolution to dynamically suppress the noise channel. The channel weights are multiplied by the input feature map channel by channel to output the feature map of the local noise suppression stage. B2. The output feature map of the local noise suppression stage is input to the spatial-channel collaborative optimization stage. Global average pooling and global maximum pooling are used to extract the statistical distribution information of the spatial dimension. The two pooling results are fused through a dilated convolutional layer to generate spatial attention weights to locate the spatial locations with turbidity and severe degradation of blurred edges. The spatial weights are multiplied pixel by pixel with the feature map of the spatial-channel collaborative optimization to enhance the detail expression of key areas, resulting in the output feature map of the spatial-channel collaborative optimization stage. B3. The output feature map from the spatial-channel collaborative optimization phase is input into the global color cast compensation phase. Global channel relationships are modeled, and inter-channel dependencies are learned through a fully connected layer to generate channel excitation weights. Based on the dominant blue-green color cast characteristics of underwater images, specific channels are dynamically enhanced or suppressed to compensate for color distortion. The channel excitation weights are multiplied channel by channel by the input features of the global color cast compensation phase to output a feature map from the global color cast compensation phase. B4. Perform pixel-wise addition of the feature map output from the global color cast compensation stage and the input feature map from the local noise suppression stage to retain the original low-frequency information and avoid over-smoothing or artifacts, thereby obtaining an enhanced feature map for the corresponding layer. B5. Repeat steps B1 to B4, performing a three-stage attention operation of local noise suppression, spatial-channel co-optimization, and global color cast compensation after the lightweight convolution of the first to fifth layers, optimizing the multi-scale feature expression from micro details to macro semantics layer by layer. Until the output feature map of the fifth layer of the encoder is obtained.
4. The improved underwater image enhancement method according to claim 3, characterized in that: The local noise suppression module in step B1 quantizes the output feature map of the lightweight convolution module As input, where Represents a set of real numbers, H, W, and C represent the height, width, and number of channels respectively; first, global average pooling is performed on each channel to generate a channel descriptor The c-th element of Z is represented as follows: Among them, F c (i, j) represents the eigenvalue of the cth channel at the position of the horizontal coordinate i and the vertical coordinate j. After obtaining the channel descriptor, the dependency between local channels is further captured by one-dimensional convolution. Specifically, a one-dimensional convolution with a kernel size of k is used to process the channel descriptor Z to generate the channel attention weight. s=σ(C1D k (WITH)) Among them, C1D k represents a one-dimensional convolution with a kernel size of k, σ is the Sigmoid activation function, k is the convolution kernel size, which is adaptively determined according to the number of channels C; and the calculation formula for the convolution kernel size k is: Where γ and b are both hyperparameters, the default value of γ is 2, and the default value of b is 1; |·| odd Indicates taking the nearest odd number; Finally, the generated channel attention weight s is multiplied by the input feature F channel by channel to obtain the output feature in, Indicates channel-by-channel multiplication; In step B2, the spatial-channel collaborative optimization stage converts the output features of the local noise suppression module into As input, the spatial-channel co-optimization stage calculates a one-dimensional channel attention map through the channel branch Compute a 2D spatial attention map via the spatial branch The calculation method is as follows: B21. Calculate one-dimensional channel attention map The mean and maximum values of the channel dimensions are calculated in parallel, and then fused after compression-expansion by a shared multi-layer perceptron MLP to generate the channel weight W. channel : First, the spatial information of the features is aggregated by using average pooling and maximum pooling operations to generate two different spatial context descriptors. and in and Represent the average pooling features and the maximum pooling features respectively: The two descriptors are then forwarded to the shared network to generate channel attention weights The shared network consists of a multi-layer perceptron (MLP). After applying the shared network to each descriptor, the output feature vectors are merged using element-wise summation. The channel attention map is calculated as follows: B22. Calculate the two-dimensional spatial attention map The spatial relationship of features is used to generate a spatial attention map; first, average pooling and maximum pooling operations are applied along the channel axis and they are concatenated to generate a valid feature descriptor; on the concatenated feature descriptors, a convolutional layer is applied to generate a spatial attention map By using two pooling operations to aggregate the channel information of the feature map, two two-dimensional maps are generated: and Represent the average pooled features and maximum pooled features across channels respectively; further, a two-dimensional spatial attention map is generated through standard convolutional layer connection and convolution; spatial attention map The calculation is as follows: The final output is the element-wise product of the channel and the spatial weight: in represents space-wise multiplication; The global color cast compensation in step B3 adopts an adaptive compression-excitation strategy: first, the input features are compressed in the channel dimension, and then the channel weights are reconstructed through nonlinear mapping. The compression and excitation methods are as follows: B31, Compression In order to alleviate the problem of not being able to utilize contextual information in the lower layers of the network with a small receptive field, the global spatial information is compressed into the channel descriptor by using global average pooling to generate channel statistics; formally, the channel descriptor It shrinks the input features by the spatial dimension H×W Generated by , where the cth element of Z is calculated as follows: B32, Motivation: In order to achieve the goal of fully capturing channel dependencies, channel weights are generated through two fully connected layers and Sigmoid function as follows: s=σ(W2·δ(W1·z)) in, is the weight matrix of the fully connected layer, δ is the ReLU activation function, and σ is the Sigmoid function; Finally, the generated channel weight s is multiplied by the input feature map F channel by channel to obtain the output feature map 5. The improved underwater image enhancement method according to claim 1, characterized in that: The multi-scale feature fusion and image reconstruction method described in step C generates the final enhanced image through nested skip connections, multi-level feature fusion and weighted average output. The specific steps are as follows: C1. Upsample the output feature map of the fifth layer of the encoder using bilinear interpolation to expand its spatial size to 1 / 8 of the original image. Then, it is channel-wise concatenated with the feature map output by the fourth layer of the encoder using a skip connection. The concatenated feature map is input to the fourth residual enhancement module, which fuses the deep semantic information of the object's overall contour with the mid-level semantic features of the regional texture distribution through a convolution operation to generate a shallow fused feature map. C2, perform secondary upsampling on the shallow fusion feature map to restore its size to 1 / 4 of the original image and concatenate it with the feature map output by the third layer of the encoder; the concatenated feature map is input into the third residual enhancement module to fuse the middle-layer semantic features with the shallow-layer details to generate a middle-layer fusion feature map; C3: Upsample the middle-layer fusion feature map to half of the original image and concatenate it with the feature map output by the second layer of the encoder. Through the mixed receptive field convolution operation of the second residual enhancement module, high-frequency details and mid-frequency structure are combined to generate a high-layer fusion feature map. C4, perform a final upsampling on the high-level fusion feature map to restore it to the original image size and concatenate it with the shallow feature map output by the first layer of the encoder; Through the dense skip connection of the first residual enhancement module, the high-resolution color distribution information and the deep correction features are fused to generate the final fused feature map; C5: Output the enhancement results from the shallow, middle, high and final fusion layers of the decoder respectively, map the feature maps of each layer into a 3-channel image through 1×1 convolution, and perform bilinear interpolation upsampling on the output feature maps of all layers to the original image size; finally, adopt a pixel-level weighted averaging strategy to combine the local detail enhancement effects of each layer with the global color correction results to generate an enhanced image with suppressed artifacts and clear details.
6. The improved underwater image enhancement method according to claim 5, characterized in that: The mathematical expression for the upsampling process to achieve size expansion by weighting neighboring pixel values is: Among them, w i,j is a bilinear weight to ensure a smooth transition of the interpolation result; the final output image is a balanced fusion of multi-level results: in, Represents the upsampling result of the k-th layer output, suppressing the single-layer error accumulation by averaging.
7. The improved underwater image enhancement method according to claim 1, characterized in that: The steps of multi-level output fusion and image restoration in step D are as follows: D1. At the end of the shallow, middle, high and final fusion layers of the decoder, the feature maps of each layer are mapped into a preliminary enhanced image of 3 channels through 1×1 convolution operations; Subsequently, the non-full-size preliminary enhanced image is upsampled using bilinear interpolation to expand its spatial size to the resolution of the original image, ensuring that the enhancement effects of different levels are aligned in the spatial dimension and avoiding overlapping artifacts or detail misalignment caused by size mismatch; D2, perform pixel-level weighted averaging fusion on the four aligned enhanced images; D3. During the training process, we only rely on the L2 loss function to supervise the multi-level output, constrain the pixel-level difference between the multi-level output and the true label, and optimize the parameters to minimize the mean square error.
8. The improved underwater image enhancement method according to claim 7, characterized in that: The bilinear interpolation upsampling achieves smooth expansion of low-resolution images to high-resolution by weighting adjacent pixel values. The formula is: Among them, x and y are the horizontal and vertical coordinates of the target high-resolution image respectively. Indicates rounding down. The weight coefficient is determined by the distance between the target position and the nearest four integer pixels. The closer the distance, the higher the weight, ensuring a smooth transition of the interpolation result. Weighted average fusion generates the final image by balancing the enhancement results of each layer. The formula is: in, represents the interpolation result of the kth level, The full-size output of the final layer is used. The averaging operation avoids the dominant enhancement effect of a single layer and suppresses overfitting or local artifacts. The L2 loss function constrains the consistency of the multi-level output and the true label through the mean square error. The formula is: Among them, L2 Loss is the loss value, N is the total number of pixels, I gt For the real label image, the loss function ensures the pixel-level accuracy of the enhanced result by minimizing the squared difference between the prediction and the true value.
Citation Information
Cited By
Digitalized batch repair method and system for ancient books
CN120765513A
A method and system for batch restoration of digitized ancient books
CN120765513B
Visible light image and infrared image fusion method and device, medium and equipment
CN120782668A
Zero-reference retina image enhancement method, system and device and medium
CN121120441A
Real-time underwater image enhancement method based on attention fusion and histogram stretching
CN121707834A
