A dual-guided underwater image multiscale enhancement method

Abstract

The application belongs to the technical field of computer vision and digital image processing, and particularly relates to a dual-guided underwater image multi-scale enhancement method, especially relates to an underwater image color cast correction, contrast enhancement and detail recovery method combining a dynamic multi-scale attention mechanism, depth perception and brightness inversion fusion technology, and is suitable for image preprocessing and quality optimization in scenes such as underwater robot navigation, ocean resource survey and underwater monitoring.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and digital image processing, and particularly relates to a dual-guided underwater image multi-scale enhancement method. BACKGROUND

[0002] Underwater images are generally affected by factors such as water absorption, scattering effect and uneven illumination, and have problems such as color cast, blurring, low contrast and loss of local details, which seriously restrict the application effect of underwater vision systems.

[0003] In the prior art, underwater image enhancement methods based on deep learning (such as UWCNN, WaterNet, UIE-DAL, etc.) improve the image quality to a certain extent by designing specific network structures or loss functions. UWCNN improves the visual effect through color correction and contrast stretching, significantly improves the overall visual effect of underwater images, and performs outstandingly in color cast correction, but is difficult to adapt to dynamic degradation in complex underwater scenes; WaterNet uses a physical model to guide network training, enhances the modeling ability of complex degradation processes, and improves the generalization of the algorithm, but the use of depth information is insufficient, resulting in limited structure recovery capability; UIE-DAL introduces an attention mechanism to enhance local features, strengthens the extraction ability of local features, and can optimize and enhance specific areas, but lacks a multi-scale dynamic weight distribution mechanism, and the collaborative optimization effect of global brightness equalization and detail recovery is not good. SUMMARY

[0004] Based on the above problems, the application introduces a dynamic multi-scale module to adaptively allocate attention weights and better compensate for the shortcomings of static models. The technical solution is as follows:

[0005] A dual-guided underwater image multi-scale enhancement method, the steps are as follows:

[0006] S1. Obtain an underwater image and perform preprocessing;

[0007] S2. Construct a depth estimation subnetwork DBEDNet based on the Encoder-Decoder framework, which introduces spatial structure information and can estimate scene depth information from the underwater image;

[0008] S3. Use the backbone network to further extract image high-level features and combine them with depth information for enhancement processing;

[0009] S4. Use the brightness inversion map R generated in the early stage to assist in enhancing the detail recovery capability of the low brightness area of the image;

[0010] S5. Using the dynamic multi-scale attention model DMAM, the importance weights of different regions and different scales are dynamically allocated after the multi-scale features and depth guidance information are fused;

[0011] S6. Reconstruction of the fused feature map;

[0012] S7. Designing a loss function to output the predicted underwater image.

[0013] Preferably, the preprocessing in step S1 includes:

[0014] Normalization: normalize the original pixel value of the input underwater image I to the interval [0, 1];

[0015] Brightness inversion generation: for the dark region in the image, a brightness inversion mapping function is designed :

[0016] ;

[0017] The brightness inversion image R is obtained by taking the inverse of the original image, so that the network can focus on the detail recovery of the low brightness region in the subsequent process;

[0018] Convolution preprocessing layer: after the input image is normalized and brightness inverted, an initial feature extraction layer is set, which can be represented as:

[0019] ;

[0020] wherein, is the convolution kernel parameter, is the bias term, and at this time represents the preliminary extracted low-level features, and the size remains consistent with the input.

[0021] Preferably, the depth estimation sub-network DBEDNet in step S2 includes an encoder and a decoder;

[0022] The encoder adopts a multi-level convolution stacking structure. Assuming that the encoder has multiple layers, the processing process of each layer is described as follows:

[0023] ;

[0024] ;

[0025] PReLU is the activation function, represents the convolution kernel weight matrix of the i-th layer of the encoder, represents the bias term of the i-th layer of the encoder;

[0026] A dual attention module (DAM) is introduced in the encoder, which is used to capture the inter-channel and spatial dependencies. The structure of the DAM includes a self-attention mechanism and a position attention mechanism:

[0027] In a specific implementation, the channel attention is obtained by global average pooling, and the spatial attention is obtained by local convolution.

[0028] The output of the DAM is fused with the standard convolutional features, and the formula is as follows:

[0029] ;

[0030] wherein, and represent the channel attention and spatial attention functions, respectively, is the fusion weight.

[0031] The decoder uses an upsampling operation to gradually restore the feature map to the original size. Let the decoder be denoted as:

[0032] ;

[0033] wherein, denotes an upsampling convolution operation, denotes the convolution kernel weight matrix of the jth layer of the decoder, denotes the bias term of the jth layer of the decoder. In each decoding stage, in order to maintain the encoder information, a skip connection is used to directly fuse the corresponding encoding layer features and decoding layer features.

[0034] Preferably, the last layer of the decoder is mapped to output a single-channel depth map D through convolution, which can approximately describe the spatial distance information of each region in the image, and the formula is as follows:

[0035] ;

[0036] wherein, denotes a Sigmoid activation function, which restricts the value to [0, 1], denotes the convolution kernel weight matrix of the depth estimation, denotes the output feature map of the last layer of the decoder, denotes the bias term of the depth estimation; the final depth map D will be used as an auxiliary input for the subsequent dynamic multi-scale attention model (DMAM) to guide the multi-scale attention mechanism.

[0037] Preferably, the backbone network in step S3 includes primary feature extraction, multi-scale feature fusion, and feature and depth information fusion.

[0038] The basic feature extraction formula is as follows:

[0039] ;

[0040] in, For the original feature map, and The first and second convolutional layers are respectively used, and the feature map size of the primary feature extraction output is consistent with the input.

[0041] Multi-scale feature fusion: Convolutional kernels of different scales (e.g., 3×3, 5×5, etc.) are applied to the same input. Features from different receptive fields are extracted through multi-branch parallel convolution and then fused. Multiple scale branches are included. The calculation formula is as follows:

[0042] ;

[0043] Indicates the first The bias term of the scaling branch is a vector used to offset the output of each neuron after the convolution operation, increasing the model's flexibility and fitting ability. Indicates the first The scale branch's convolution kernel weight matrix; it is used to scale the input feature map. Convolutional operations are performed to extract features at this scale. (The size and number of convolutional kernels depend on the network design). The fusion operation involves channel concatenation followed by 1×1 convolutional dimensionality reduction, and then activation processing to generate a fused multi-scale feature map. ;

[0044] Feature and depth information fusion: Let the processed depth features be... The feature fusion operation is then:

[0045] ;

[0046] in, This indicates that the feature maps are summed or concatenated channel by channel, and then adjusted through a fusion convolution.

[0047] Preferably, step S4 is implemented as follows:

[0048] S41. The input brightness inverted image R is simultaneously processed through a separate convolution branch, which is designed as follows:

[0049] ;

[0050] in, This represents the kernel weight matrix in the independent convolutional branches that process the brightness inversion image R. denotes the bias term in the convolution branch, after multiple layers of convolution, the output feature map is fused with the backbone feature layer by layer;

[0051] brightness inversion feature extracted by convolution and the backbone feature Layer-by-layer fusion: Specifically, at key positions such as the residual block, the end of the encoding layer, and the entrance of the DMAM module, and are element-wise summed, and the formula is:

[0052] ;

[0053] wherein, is the fusion coefficient.

[0054] Preferably, the dynamic multi-scale attention DMAM model includes multi-branch attention calculation, dynamic learning of attention weights, and feature re-weighting and fusion;

[0055] Multi-branch attention calculation:

[0056] The input feature is processed by multiple convolution branches of different scales, each branch uses a different size of convolution kernel to capture context information in different ranges; let the output of each scale branch be , and the calculation formula is:

[0057] ;

[0058] denotes the convolution kernel weight matrix of the i-th scale branch in the dynamic multi-scale attention module (DMAM). It is used to perform convolution operation on the input feature to capture features of different scales, denotes the bias term in the scale branch, and through the calculation of different scale features, the respective response map is obtained;

[0059] Dynamic learning of attention weights:

[0060] After the multi-scale response map is spliced, the learning of the attention weights is realized through global average pooling and a fully connected layer, let the spliced feature map be , and the calculation steps of its attention vector are as follows:

[0061] ;

[0062] wherein, is the Sigmoid activation function, and the output weight vector is in the interval [0,1]; (This represents the weight matrix in attention weight learning. It is used to perform a linear transformation on the features after global average pooling (GAP) to generate the attention vector.) (This represents the feature map resulting from the concatenation of multi-scale response maps. It contains feature information from different scale branches and is used to calculate attention weights.) (This represents the bias term in attention weight learning).

[0063] Feature reweighting and fusion:

[0064] Each branch feature is weighted by attention After weighting, the feature maps are then generated through channel concatenation and 1×1 convolution fusion to produce dynamically adjusted feature maps. The fusion formula is expressed as:

[0065] ;

[0066] in, For the attention weights of the corresponding branches, (Convolutional kernel weight matrix used for fusion operation) (Feature map of the i-th scale branch) (Bias term of fusion operation).

[0067] Preferably, the image output by the dynamic multi-scale attention DMAM model is subjected to smoothing processing, including reshape, reverse, and region smoothing.

[0068] Reshape processing:

[0069] The feature map output by the Dynamic Multi-Scale Attention (DMAM) model is re-blocked, with regions of fixed size divided to ensure local consistency within each region. The resulting sub-blocks are denoted as follows: ;

[0070] Reverse handling:

[0071] The features within each sub-block are reverse-processed, that is, the brightness inversion features generated in the previous stage are utilized. Compensate this area by applying a reverse mapping to each pixel within the sub-block:

[0072] ;

[0073] in, To adjust the parameters, This is a compensation term reconstructed based on the regional brightness inversion relationship;

[0074] Region Smoothing Processing:

[0075] After processing, each sub-block is smoothed using a region smoothing process (such as mean filtering or bilateral filtering). Through multiple iterations, abrupt changes between blocks are eliminated, and finally, the sub-blocks are reassembled into a unified feature map. .

[0076] Preferably, the reconstruction of the fused feature map in step S6 is as follows:

[0077] A multi-level residual unit is designed to perform nonlinear mapping and detail recovery on the fused features, using the following formula:

[0078] ;

[0079] (This represents the input feature map of the k-th layer. It contains feature information passed from the previous layer.) (This represents the weight matrix of the first convolutional operation in the k-th layer. It is used to weight the input feature map.) (perform convolution operation) (This represents the bias term for the first convolution operation in the k-th layer. It is used to adjust the output after the convolution operation.)

[0080] Then, the result is obtained through a second convolutional layer:

[0081] ;

[0082] (This represents the feature map of the k-th layer after the first convolution operation and activation function. It is obtained by converting the input feature map...) (The result is obtained by adding the residual connection) (This represents the weight matrix of the second convolutional operation in the k-th layer. It is used to weight the feature map.) (to perform further convolution operations) This represents the bias term of the second convolution operation in the k-th layer, used to adjust the output after the convolution operation;

[0083] Multi-level residual design can preserve the initial image feature information and refine and enhance the effect layer by layer;

[0084] The reconstruction module finally uses a 3×3 convolutional mapping layer to map the feature map output by the residual unit into a three-channel image, as shown in the following formula:

[0085] ;

[0086] The Sigmoid activation function constrains the output to within [0,1], resulting in the final output being... For the final fused feature map, (This represents the weight matrix of the final convolutional mapping layer. It is used to map the feature map output by the residual unit into a three-channel image.) (Represents the final feature map after all processing) (This represents the bias term of the final convolutional mapping layer).

[0087] Preferably, in step S7, a loss function is designed to output the predicted underwater image. The specific method is as follows:

[0088] Using mean square error loss To measure the output image Compared to the original clear image The gap between them

[0089] High-level features of images are extracted using a pre-trained VGG network, and a perceptual loss is constructed in the feature space:

[0090] ;

[0091] Where N (represents the number of feature layers used to calculate the perceptual loss in the VGG network) This represents the feature map of the j-th layer of the VGG network. This loss helps to improve the high-level semantic consistency and visual realism of the image.

[0092] Design Structural Similarity Index (SSIM) Loss:

[0093] ;

[0094] This loss encourages the output image to maintain a high degree of consistency with the original image in terms of local structure and texture;

[0095] Attention-constrained loss: Let the weight vector To satisfy certain sparsity and smoothness constraints, it is possible to design Loss or KL divergence:

[0096] ;

[0097] in (This represents the preset target attention vector. It is an ideal attention weight distribution used to guide the model to learn a more reasonable attention allocation.) It can be set according to the preset target distribution;

[0098] Total Loss Function: The total loss function for each module is defined as follows:

[0099] ;

[0100] or,

[0101] ;

[0102] in, The weights for each type of loss.

[0103] Compared with the prior art, the beneficial effects of this application are as follows:

[0104] 1. By using a depth estimation subnetwork and a multi-scale attention mechanism, the system can capture global structural information while dynamically adjusting and enhancing local details, effectively solving the problems of blurred details and unclear edges in underwater images.

[0105] 2. By utilizing inverted brightness image information, a special brightness inversion fusion was performed to address the typical problems of uneven underwater lighting and insufficient detail in dark areas, ensuring a more balanced overall brightness distribution in the image without introducing overexposure or underexposure.

[0106] 3. Each module employs residual learning and skip connections to ensure gradient propagation and prevent network degradation. Furthermore, the design of multiple loss functions ensures stable performance on various test datasets. Ablation experiments verify the performance improvements brought by the DMAM, RBM, and R³S modules, demonstrating that this invention is applicable to different underwater scenarios. Attached Figure Description

[0107] Figure 1 The overall architecture of DBED-Net;

[0108] Figure 2 Here is a diagram of the DMAM structure;

[0109] Figure 3 This is a comparison chart showing the effects. Detailed Implementation

[0110] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following describes in detail, with reference to the accompanying drawings and specific embodiments, a method and system for frequency domain detection of colored noise environmental signals based on spectral envelope extraction proposed in accordance with the present invention.

[0111] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0112] like Figure 1As shown, the overall framework of this system can be divided into an input preprocessing module, a depth estimation sub-network (DBEDNet), a backbone feature extraction module, an RBM (Reverse Brightness Mask) enhancement module, a DMAM (Dynamic Multi-scale Attention Module) module, an R³S (Reshape-Reverse-Region Smoothing) module, and a final reconstruction module. The entire network adopts an end-to-end training method to achieve direct mapping from input to output of underwater images. The system has the following characteristics:

[0113] 1. Depth perception guidance: The depth estimation sub-network is used to obtain scene structure information to guide the main network in feature enhancement.

[0114] 2. Multi-scale attention: Dynamically captures regional features at different scales through DAM, so that local details and global information are processed simultaneously.

[0115] 3. Brightness Inversion Blending: The RBM module is used to enhance and restore dark areas, making up for the shortcomings of traditional methods in addressing uneven brightness.

[0116] 4. Residual and Multi-Layer Fusion: Multi-level residual modules and R³S modules are used to ensure feature transfer and smooth details, achieving accurate reconstruction.

[0117] A dual-guided underwater image multi-scale enhancement method, comprising the following steps:

[0118] S1. Acquire underwater images and perform preprocessing;

[0119] S2. Construct the depth estimation subnetwork DBEDNet, which is based on the Encoder-Decoder framework and incorporates spatial structure information, enabling it to estimate scene depth information from underwater images;

[0120] S3. Utilize the backbone network to further extract high-level image features and combine them with depth information for enhancement processing;

[0121] S4. Utilize the previously generated brightness inversion map R to assist in enhancing the detail recovery capability of low-brightness areas in the image;

[0122] S5. By using the Dynamic Multi-Scale Attention Model (DMAM), which integrates multi-scale features and deep guidance information, importance weights for different regions at different scales are dynamically assigned.

[0123] S6. Reconstruction of fused feature maps;

[0124] S7. Design a loss function to output the predicted underwater image.

[0125] The preprocessing method in step S1 is as follows:

[0126] Underwater images often suffer from color cast and low contrast due to their unique light absorption and scattering characteristics. To fully preserve the original information and facilitate feature extraction by the network, this implementation performs the following preprocessing during the input stage:

[0127] 1. Image normalization: The original pixel values ​​of the input underwater image I are normalized to the [0,1] interval to reduce the impact of numerical fluctuations on subsequent training.

[0128] 2. Brightness Inversion Generation: A brightness inversion mapping function is designed for the dark areas in the image. :

[0129] ;

[0130] By inverting the original image to obtain a brightness inversion map R, the network can focus on detail recovery in low-brightness areas during subsequent processes. In the actual implementation, this operation is performed on each color channel, and the original image and its inverted image are combined as joint input.

[0131] 3. Convolutional Preprocessing Layer: After the input image is normalized and brightness inverted, an initial feature extraction layer is set up, using a 3×3 convolution kernel with a stride of 1 and padding of 1, followed by the PReLU activation function and batch normalization (BN). This layer can be represented as:

[0132] ;

[0133] in, For convolution kernel parameters, This is the bias term. At this point... This represents the initial low-level features extracted, with the same size as the input.

[0134] In step S2, the depth estimation subnetwork (DBEDNet) is used.

[0135] To incorporate spatial structure information during image enhancement, this implementation design includes a sub-network specifically for depth estimation, namely DBEDNet. This network, based on the Encoder-Decoder framework, is capable of estimating scene depth information from underwater images, thus providing effective guidance for subsequent modules.

[0136] 1. Encoder section:

[0137] The encoder employs a multi-level convolutional stacked structure, with each layer consisting of convolution, batch normalization (BN), and pre-return (PReLU) operations, progressively reducing the feature map size and increasing the receptive field. Assuming the encoder has N layers, the processing procedure for each layer is described below:

[0138] ;

[0139] ;

[0140] In this design, each convolutional kernel is set to a size of 3×3 or 5×5, with a stride typically of 2, to achieve image downsampling. In addition to standard convolutions, a Dual-Attention Module (DAM) is introduced into the encoder. This module captures inter-channel and spatial dependencies, and its structure basically includes self-attention and positional attention mechanisms. Specifically, channel attention is first obtained through global average pooling, and then spatial attention is obtained through local convolutions. The output of DAM is fused with standard convolutional features, as shown in the following formula:

[0141] ;

[0142] in, and Let these represent the channel attention and spatial attention functions, respectively. The fusion weights were determined experimentally.

[0143] 2. Decoder section:

[0144] The multi-scale features extracted by the encoder are processed through several coding layers and then reconstructed in the decoder. The decoder primarily employs upsampling operations to progressively restore the feature maps to their original size. Let each layer of the decoder be:

[0145] ;

[0146] in This indicates an upsampling convolution operation. In each decoding stage, to preserve encoder information, skip connections are used to directly fuse the corresponding encoder layer features with the decoder layer features. The fusion employs simple addition or concatenation operations, and is then fused into a fixed number of channels via a 1×1 convolution.

[0147] 3. Depth map output:

[0148] The final layer of the decoder outputs a single-channel depth map D after convolutional mapping. This depth map can approximately describe the spatial distance information of various regions in the image, as shown in the following formula:

[0149] ;

[0150] in, This represents the Sigmoid activation function, with values ​​constrained between [0,1]. The final depth map D will serve as auxiliary input for the subsequent DMAM module, guiding the multi-scale attention mechanism.

[0151] The backbone feature extraction module in step S3:

[0152] After preprocessing and depth estimation, the main task of the backbone network is to further extract high-level features of the image and combine them with depth information for enhancement. This module mainly includes the following parts:

[0153] 1. Primary Feature Extraction:

[0154] In addition to the aforementioned input convolutional layers, the primary feature extraction of the backbone network again utilizes multiple convolutional layers (3×3 or 5×5 kernels can be used) and a residual block (ResBlock) structure. The ResBlock employs an identity mapping to avoid gradient vanishing, and its structure is as follows:

[0155] ;

[0156] in, and The first and second convolutional layers are respectively used, and residual connections ensure that information is fully transferred. The feature map size output by the primary feature extraction module is consistent with the input, and the number of channels is set to 64 or 128 according to actual needs.

[0157] 2. Multi-scale feature fusion:

[0158] To fully capture information at various scales in the image, this invention introduces a pyramid structure into the backbone network, applying convolutional kernels of different scales and receptive fields to the same input. Features at different scales are obtained through multi-branch parallel convolution, and then fused. Multiple scale branches are included. The calculation formula is as follows:

[0159] ;

[0160] The fusion operation involves channel concatenation followed by 1×1 convolution for dimensionality reduction, and then activation processing to generate a fused multi-scale feature map. .

[0161] 3. Feature and deep information fusion:

[0162] To incorporate depth information for guidance, this implementation upsamples the depth map D obtained from DBEDNet to the same size as the backbone network feature map using bilinear interpolation, and then processes it with a separate 3×3 convolution and PReLU. Let the processed depth feature be... The feature fusion operation is then:

[0163] ;

[0164] in, The feature maps are summed or concatenated channel by channel, and then adjusted through a fusion convolution. This design allows the network to focus on both local texture and overall structural information, helping to suppress noise and blur in underwater images.

[0165] In step S4, the RBM module (luminance inversion auxiliary enhancement) is used.

[0166] The main purpose of the RBM module is to utilize the previously generated luminance inversion map R to enhance the detail recovery capability of low-brightness areas in the image. The specific implementation steps are as follows:

[0167] 1. Branch creation:

[0168] The input luminance inverted image R is processed simultaneously through a separate convolution branch, and... Similarly, this branch is designed as follows:

[0169] ;

[0170] The convolutional kernel size is also set to 3×3, maintaining a similar number of channels as the main branch. After multiple convolutions, the output feature map of this branch is fused with the main features layer by layer.

[0171] 2. Feature complementarity fusion:

[0172] Brightness inversion feature extracted after convolution With main characteristics Layered fusion. Specifically, at each key location (e.g., residual block edges, coding layer ends, and DMAM module entry points), [the process is as follows:] [The ... and For element-wise summation, the formula is:

[0173] ;

[0174] in, The fusion coefficient can be determined through preliminary experiments. This design ensures that even areas where dark details are lost in underwater images can be effectively supplemented by inverting brightness information.

[0175] Step S5: DMAM module (Dynamic Multi-Scale Attention Module):

[0176] The DMAM module is the core of this invention. Its function is to dynamically allocate importance weights for different regions and scales after fusing multi-scale features and depth-guided information, thereby improving the recovery capability of key structural regions and texture details. The implementation process of this module includes the following steps:

[0177] 1. Multi-branch attention calculation:

[0178] Input features The process involves multiple convolutional branches at different scales, each using a different kernel size (e.g., 3×3, 5×5, 7×7, etc.) to capture contextual information within different ranges. Let the output of each scale branch be denoted as... The calculation formula is:

[0179] ;

[0180] By calculating features at different scales, their respective response maps are obtained.

[0181] 2. Dynamic learning of attention weights:

[0182] After concatenating the multi-scale response maps, attention weights are learned through Global Average Pooling (GAP) and a Fully Connected (FC) layer. Let the concatenated feature map be denoted as . Its attention vector The calculation steps are as follows:

[0183] ;

[0184] in, The sigmoid activation function outputs a weight vector in the interval [0,1]. This vector is then used to weight features at different scales.

[0185] 3. Feature reweighting and fusion:

[0186] After the features from each branch are weighted by the attention vector, they are then fused through channel concatenation and 1×1 convolution to generate a dynamically adjusted feature map. The fusion formula is expressed as:

[0187] ;

[0188] in, These correspond to the attention weights of the respective branches. This design allows the system to adaptively adjust the weight of information at each scale according to the importance of different regions, thereby more accurately restoring image details.

[0189] R³S module (Reshape-Reverse-Region Smoothing module):

[0190] To further improve the smoothness and local consistency of the reconstructed image, this implementation design includes an R³S module, which mainly consists of three operations: Reshape, Reverse, and Region Smoothing.

[0191] 1.Reshape operation:

[0192] The feature map output by the DMAM module is re-divided into blocks. The feature map is divided into regions of a fixed size (e.g., 8×8 or 16×16) to ensure a certain degree of local consistency within each region. The resulting sub-blocks are denoted as follows: .

[0193] 2. Reverse operation:

[0194] The features within each sub-block are reverse-processed, that is, the brightness inversion features generated in the previous stage are utilized. Compensate this area by applying a reverse mapping to each pixel within the sub-block:

[0195] ;

[0196] in, To adjust the parameters, This is a compensation term reconstructed based on the regional brightness inversion relationship. This operation can enhance the details of low-brightness areas while avoiding local distortion caused by over-enhancement.

[0197] 3. Region Smoothing:

[0198] Region smoothing is applied to each processed sub-block using mean filtering or bilateral filtering to ensure smooth transitions between different sub-blocks and avoid artifacts caused by abrupt transitions. After multiple iterations of smoothing, each sub-block is finally reconstructed into a unified feature map. .

[0199] Reconstructing the module in step S6:

[0200] After processing by the above modules, a fused feature map is obtained, which has undergone dynamic multi-scale attention and depth guidance, brightness inversion compensation, and local smoothing. The reconstruction module's task is to convert this feature map into the final enhanced image output. Its structural design is as follows:

[0201] 1. Residual Reconstruction Unit:

[0202] A multi-level residual unit is designed to perform nonlinear mapping and detail recovery on the fused features. Each residual unit contains two convolutional layers, using the following formula:

[0203] ;

[0204] Then, the result is obtained through a second convolutional layer:

[0205] ;

[0206] Multi-level residual design can preserve the initial image information and refine and enhance the effect layer by layer.

[0207] 2. Final convolution mapping:

[0208] The reconstruction module finally uses a 3×3 convolutional mapping layer to map the feature map output by the residual unit into a three-channel image, as shown in the following formula:

[0209] ;

[0210] The Sigmoid activation function constrains the output to within [0,1]. The final output... The enhanced underwater image, after comprehensive processing by all modules, achieves color correction, contrast enhancement, and detail restoration.

[0211] Step S7, Training Strategy and Loss Function Design:

[0212] This system adopts an end-to-end joint training strategy. To ensure that the modules can work together, the following multiple loss functions are designed during the training process.

[0213] 1. Reconstruction losses:

[0214] Mean Squared Error (MSE) is used to measure the output image. Compared to the original clear image The difference between them:

[0215] ;

[0216] This loss is primarily used to guide the accuracy of global reconstruction.

[0217] 2. Perceived loss:

[0218] High-level features of images are extracted using a pre-trained VGG network, and a perceptual loss is constructed in the feature space:

[0219] ;

[0220] in, Let represent the feature map of the j-th layer of the VGG network. This loss helps improve the high-level semantic consistency and visual realism of the image.

[0221] 3. Structural retention loss:

[0222] To address the problem of preserving structural information in underwater images, a structural similarity index (SSIM) loss is designed:

[0223] ;

[0224] This loss encourages the output image to maintain a high degree of consistency with the original image in terms of local structure and texture.

[0225] 4. Loss of attentional constraint:

[0226] To address the dynamic learning process of attention weights in the DMAM module, a regularization term is added to make the distribution of attention weights at each scale more reasonable. Let the weight vector... To satisfy certain sparsity and smoothness constraints, L1 loss or KL divergence can be designed:

[0227] ;

[0228] in It can be set according to the preset target distribution.

[0229] 5. Total loss function:

[0230] Taking into account the above losses, the total loss function for each module is defined as follows:

[0231] ;

[0232] ;

[0233] in, We assign weights to each loss term and determine the optimal combination through cross-validation and experiments.

[0234] 6. Training details:

[0235] Dataset: During training, publicly available underwater image datasets such as UIEB and EUVP are used. Data can also be augmented manually, and noise, blur, color cast, etc. can be artificially synthesized.

[0236] Optimizer: The Adam optimizer is used, with an initial learning rate set to 1e-4. A learning rate decay strategy is employed to gradually reduce the learning rate when the validation set loss stops decreasing.

[0237] Batch size: Depending on the GPU memory, the batch size is generally set to 8 to 16.

[0238] Number of iterations: Typically, the model is trained around 100,000 times, and its performance on the validation set is recorded at regular intervals. System advantages and implementation results.

[0239] Through the above design, this implementation method has the following obvious advantages:

[0240] 1. Balancing global and local information:

[0241] By using a depth estimation subnetwork and a multi-scale attention mechanism, the system can capture global structural information while dynamically adjusting and enhancing local details, effectively solving the problems of blurred details and unclear edges in underwater images.

[0242] 2. Adaptive brightness compensation:

[0243] By utilizing inverted brightness image information, a special brightness inversion fusion was performed to address the typical problems of uneven underwater lighting and insufficient detail in dark areas, ensuring a more balanced overall brightness distribution in the image without introducing overexposure or underexposure.

[0244] 3. Robustness and generalization ability:

[0245] Each module employs residual learning and skip connections to ensure gradient propagation and prevent network degradation. Furthermore, the design of multiple loss functions ensures stable model performance across different test datasets. Ablation experiments verify the performance improvements brought by the DMAM, RBM, and R³S modules, demonstrating that this invention is applicable to various underwater scenarios.

[0246] 4. Advantages of end-to-end training:

[0247] The entire network adopts an end-to-end joint training method, which eliminates the need for complex post-training corrections and can output augmentation results in real time. It has extremely high practical value in applications such as underwater robots, remote monitoring, and marine engineering exploration.

[0248] Hardware and software implementation environment:

[0249] To verify the feasibility of this implementation plan, the following training and deployment environment is recommended:

[0250] 1. Software environment:

[0251] The system is based on the PyTorch framework and uses Python 3.8.18 and other environments.

[0252] Dependencies include NumPy, OpenCV Python, and Scikit-image.

[0253] The training code is modularly designed to support distributed training across multiple GPUs.

[0254] 2. Hardware configuration:

[0255] It is recommended to use a GPU (such as an NVIDIA RTX 3080 or higher) for model training to improve training efficiency.

[0256] The system runs on the Windows operating system and also supports embedded system deployment to meet the real-time inference requirements of industrial applications. It enables fast inference after being converted to the TensorRT model format.

[0257] Summary of the specific implementation process:

[0258] Based on the above modules, the specific implementation process of this solution is as follows:

[0259] 1. Normalize and invert the brightness of the input underwater image to generate the initial feature F0 and the inverted image R.

[0260] 2. Obtain low-level features through the primary feature extraction layer, and simultaneously input the image into the depth estimation subnetwork DBEDNet to obtain the depth map D.

[0261] 3. The primary features are processed using a multi-level convolutional residual module and a multi-scale pyramid structure to obtain the multi-scale fused feature F_ms, which is then fused with the upsampled deep feature F_D to obtain F_fused.

[0262] 4. Using the RBM module, the features F_R extracted from the luminance inversion branch are added to F_fused for local detail enhancement, generating F_enhanced.

[0263] 5. Input F_enhanced into the DMAM module, learn through multi-scale convolutional branches and global attention weights, dynamically reweight the features at each scale, and output F_{DMAM}.

[0264] 6. For F_{DMAM}, the R³S module is used to perform reshape, reverse compensation and region smoothing within the region to obtain the smoothed comprehensive feature F_{R^3S}.

[0265] 7. Finally, F_{R^3S} is fed into the residual reconstruction module and the final convolutional mapping layer to output the restored underwater enhanced image I_out.

[0266] 8. During training, MSE, perceptual loss, SSIM loss, and attentional constraint loss are used together to optimize network parameters, ensuring the best balance between global restoration and detail recovery.

[0267] Experiment and Results Presentation:

[0268] In the experimental section, the method proposed in this invention needs to be compared with existing mainstream methods (such as UWCNN, WaterNet, UIE-DAL, etc.). Public datasets such as UIEB and EUVP are used in the experiments, and the following tests are performed.

[0269] 1. Quantitative indicators:

[0270] Indicators such as PSNR and SSIM are used to measure the enhanced image quality. Numerical results demonstrate that this method is superior to traditional methods in detail restoration and color correction.

[0271] 2. Ablation experiment:

[0272] Remove the DAM, RBM, and R³S modules respectively, observe the changes in image details, and verify the improvement effect of each module on the final performance.

[0273] 3. Qualitative observation:

[0274] Comparing the images under different lighting and underwater environments demonstrates that the present invention can work stably in various underwater scenarios.

[0275] Experimental results show that this method outperforms existing methods in terms of detail restoration, color balance, and edge sharpness, demonstrating strong robustness and broad application prospects.

[0276] This implementation proposes a novel underwater image enhancement method based on depth estimation and dynamic multi-scale attention fusion. By preprocessing the input image and introducing a DBEDNet sub-network for depth guidance, combined with the brightness inversion assistance of the RBM module and the multi-scale dynamic attention adjustment of the DMAM module, and finally using the R³S module for smoothing local regions, the method effectively improves the performance of underwater images in terms of detail, contrast, and color. Furthermore, an end-to-end joint training strategy is employed to ensure that the entire model gradually converges during training and exhibits excellent enhancement results in various underwater scenarios.

[0277] This invention has high industrial application value, especially in the fields of underwater robot navigation, marine resource surveying, and environmental monitoring. It can play a significant role in image enhancement, providing more reliable input information for underwater vision systems, thereby improving the overall system's efficiency and safety.

Claims

1. A dual-guided underwater image multi-scale enhancement method, characterized in that, The steps are as follows: S1. Acquire underwater images and perform preprocessing; S2. Construct the depth estimation subnetwork DBEDNet, which is based on the Encoder-Decoder framework and incorporates spatial structure information, enabling it to estimate scene depth information from underwater images; S3. Utilize the backbone network to further extract high-level image features and combine them with depth information for enhancement processing; S4. Utilize the previously generated brightness inversion map R to assist in enhancing the detail recovery capability of low-brightness areas in the image; S5. By using the Dynamic Multi-Scale Attention Model (DMAM), which integrates multi-scale features and deep guidance information, importance weights for different regions at different scales are dynamically assigned. S6. Reconstruction of fused feature maps; S7. Design a loss function to output the predicted underwater image.

2. The dual-guided underwater image multi-scale enhancement method according to claim 1, characterized in that, The preprocessing in step S1 includes: Normalization: The original pixel values ​​of the input underwater image I are normalized to the [0,1] interval; Brightness Inversion Generation: Design a brightness inversion mapping function for dark areas in an image. : ； By inverting the original image to obtain the brightness inversion map R, the network can focus on the recovery of details in low-brightness areas in subsequent processes. Convolutional preprocessing layer: After the input image is normalized and brightness inverted, an initial feature extraction layer is set up, which is represented as: ； in, For convolution kernel parameters, For the bias term, at this time This represents the initial low-level features extracted, with the same size as the input.

3. The underwater image multi-scale enhancement method with dual guidance according to claim 1, characterized in that, In step S2, the depth estimation subnetwork DBEDNet includes an encoder and a decoder; The encoder employs a multi-level convolutional stacking structure, with multiple layers. The processing procedure for each layer is described below: ； ； PReLU is an activation function. The weight matrix of the convolution kernel in the i-th layer of the encoder is represented by... This represents the bias term of the encoder's i-th layer; A dual attention module is introduced into the encoder to capture inter-channel and spatial dependencies. Its structure includes self-attention and positional attention mechanisms. First, channel attention is obtained through global average pooling, and then spatial attention is obtained through local convolution. The output of DAM is fused with standard convolutional features, as shown in the following formula: ； in, and Let these represent the channel attention and spatial attention functions, respectively. For fusion weights; The decoder uses upsampling to gradually restore the feature map to its original size. Let each layer of the decoder be: ； in This indicates an upsampling convolution operation. The kernel weight matrix of the j-th layer of the decoder is represented by... This represents the bias term of the j-th layer of the decoder; in each decoding stage, in order to preserve encoder information, skip connections are used to directly fuse the corresponding coding layer features with the decoding layer features.

4. The dual-guided underwater image multi-scale enhancement method according to claim 3, characterized in that, The final layer of the decoder outputs a single-channel depth map D after convolutional mapping. This depth map describes the spatial distance information of each region in the image, as shown in the following formula: ； in, This represents the Sigmoid activation function, which constrains values ​​to the range [0,1]. The convolution kernel weight matrix represents the depth estimation. This represents the output feature map of the last layer of the decoder. This represents the bias term for depth estimation; the final depth map D will serve as an auxiliary input for the subsequent dynamic multi-scale attention model, guiding the multi-scale attention mechanism.

5. The underwater image multi-scale enhancement method with dual guidance according to claim 1, characterized in that, In step S3, the backbone network includes primary feature extraction, multi-scale feature fusion, and feature-deep information fusion. The basic feature extraction formula is as follows: ； in, For the original feature map, and The first and second convolutional layers are respectively used, and the feature map size of the primary feature extraction output is consistent with the input. Multi-scale feature fusion: Convolutional kernels of different scales are applied to the same input, and features from different receptive fields are extracted and fused through multi-branch parallel convolution; multiple scale branches are included. The calculation formula is as follows: ； Indicates the first The bias term of the scale branch. Indicates the first The scale branch's convolution kernel weight matrix; it is used to scale the input feature map. Convolutional operations are performed to extract features at this scale; the fusion operation involves concatenating channels, reducing the dimensionality using 1×1 convolutions, and then performing activation processing to generate a fused multi-scale feature map. ; Feature and depth information fusion: Let the processed depth features be... The feature fusion operation is then: ； in, This indicates that the feature maps are summed or concatenated channel by channel, and then adjusted through a fusion convolution.

6. The underwater image multi-scale enhancement method with dual guidance according to claim 1, characterized in that, The specific implementation steps of step S4 are as follows: S41. The input brightness inverted image R is simultaneously processed through a separate convolution branch, which is designed as follows: ; in, This represents the kernel weight matrix in the independent convolutional branches that process the brightness inversion image R. This represents the bias term in the convolutional branch. After multiple convolutions, the output feature map of this branch is fused with the main features layer by layer. Brightness inversion feature extracted after convolution With main characteristics Layered integration, and The formula for summing element by element is: ； in, This is the fusion coefficient.

7. The dual-guided underwater image multi-scale enhancement method according to claim 6, characterized in that, The Dynamic Multi-Scale Attention (DMAM) model includes multi-branch attention computation, dynamic learning of attention weights, and feature reweighting and fusion. Multi-branch attention calculation: The input features are processed through multiple convolutional branches at different scales, each using a different sized kernel to capture contextual information within different ranges; the output of each scale branch is denoted as... The calculation formula is: ； This represents the convolution kernel weight matrix of the i-th scale branch in the dynamic multi-scale attention module. This represents the bias term in this scale branch. By calculating the features at different scales, their respective response maps are obtained. Dynamic learning of attention weights: After concatenating the multi-scale response maps, attention weights are learned through global average pooling and fully connected layers. Let the concatenated feature map be denoted as... Its attention vector The calculation steps are as follows: ； in, The sigmoid activation function outputs a weight vector in the interval [0,1]. This represents the weight matrix in attention weight learning. This represents the feature map obtained by stitching together multi-scale response maps. This represents the bias term in attention weight learning; Feature reweighting and fusion: Each branch feature is weighted by attention After weighting, the feature maps are then fused through channel concatenation and 1×1 convolution to generate dynamically adjusted feature maps. The fusion formula is expressed as: ； in, For the attention weights corresponding to the i-th branch, The convolutional kernel weight matrix used for the fusion operation. The feature map representing the i-th scale branch. This represents the bias term for the fusion operation.

8. The underwater image multi-scale enhancement method with dual guidance according to claim 1, characterized in that, The images output by the Dynamic Multi-Scale Attention DMAM model are smoothed using techniques including Reshape, Reverse, and RegionSmoothing. Reshape processing: The feature map output by the Dynamic Multi-Scale Attention (DMAM) model is re-blocked, with regions of fixed size divided to ensure local consistency within each region. The resulting sub-blocks are denoted as follows: ; Reverse handling: The features within each sub-block are reverse-processed, that is, the brightness inversion features generated in the previous stage are utilized. Compensate this area by applying a reverse mapping to each pixel within the sub-block: ； =1,2,...,n； in, To adjust the parameters, This is a compensation term reconstructed based on the regional brightness inversion relationship; Region Smoothing Processing: Region smoothing is applied to each processed sub-block, and inter-block mutations are eliminated through multiple iterations. Finally, the sub-blocks are recombined into a whole feature map. .

9. The dual-guided underwater image multi-scale enhancement method according to claim 1, characterized in that, The reconstruction of the fused feature map in step S6 is as follows: A multi-level residual unit is designed to perform nonlinear mapping and detail recovery on the fused features, using the following formula: ； This represents the input feature map of the k-th layer, which contains feature information passed from the previous layer; This represents the weight matrix of the first convolutional operation in the k-th layer, used to weight the input feature map. Perform convolution operations; This represents the bias term of the first convolution operation in the k-th layer; Then, the result is obtained through a second convolutional layer: ； This represents the feature map of the k-th layer after the first convolution operation and activation function. It is obtained by converting the input feature map... Obtained by adding the residual connection; This represents the weight matrix for the second convolutional operation in the k-th layer, used to weight the feature map. Perform further convolution operations; This represents the bias term of the second convolution operation in the k-th layer; Multi-level residual design can preserve the initial image feature information and refine and enhance the effect layer by layer; The reconstruction module finally uses a 3×3 convolutional mapping layer to map the feature map output by the residual unit into a three-channel image, as shown in the following formula: ； The Sigmoid activation function constrains the output to within [0,1], resulting in the final output being... For the final fused feature map, This represents the weight matrix of the final convolutional mapping layer, used to map the feature map output by the residual unit into a three-channel image; This represents the final feature map after all processing. This represents the bias term of the final convolutional mapping layer.

10. The dual-guided underwater image multi-scale enhancement method according to claim 1, characterized in that, Step S7: Design the loss function and output the predicted underwater image. The specific method is as follows: Using mean square error loss To measure the output image Compared to the original clear image The gap between them High-level features of images are extracted using a pre-trained VGG network, and a perceptual loss is constructed in the feature space: ； Where N represents the number of feature layers used to calculate the perceptual loss in the VGG network. This represents the feature map of the j-th layer of the VGG network; Design SSIM loss: ； The loss The output image is encouraged to maintain a high degree of consistency with the original image in terms of local structure and texture. Attention-constrained loss: Let the weight vector To satisfy the set sparsity and smoothness constraints, design Loss or KL divergence: ; in This represents the preset target attention vector, which is an ideal attention weight distribution used to guide the model to learn a more reasonable attention allocation. It is set according to the preset target distribution. The total loss function for each module is defined as follows: ； or ； in, The weights for each type of loss.

Images