Real-time underwater image enhancement method based on attention fusion and histogram stretching

Abstract

The application discloses a real-time underwater image enhancement method based on attention fusion and histogram stretching. The method sequentially performs the following steps: extracting the basic features of the original image through convolution; deepening the features and fusing multi-scale information by using a depth separable convolution module; optimizing the channel and spatial features in parallel through a cross-domain feature fusion and enhancement module, and supplementing semantic information in combination with a residual connection; finally, after the convolution of the aggregated features, global histogram stretching and detail optimization are performed by a dynamic feature correction module, and an enhanced image is output. Through the collaborative design of lightweight modules, the application can effectively improve the image enhancement quality while reducing the computational complexity, achieving a balance between enhancement performance, lightweight and real-time, and is especially suitable for image enhancement tasks in complex underwater scenes such as marine resource exploration, underwater edge mechanical equipment navigation, underwater archaeology and the like.

Description

Technical Field

[0001] This invention pertains to image data processing, specifically to a real-time underwater image enhancement method based on attention fusion and histogram stretching. Background Technology

[0002] Underwater image enhancement is a key technology in computer vision and image processing, widely used in marine exploration, fisheries monitoring, seabed detection, and underwater robotics. However, the complexity of underwater scenes easily leads to image quality degradation, manifested in three main problems: color cast, blurring, and low light. Firstly, different wavelengths of light attenuate significantly in water; red, orange, yellow, and green light disappear sequentially with depth, resulting in a bluish-green hue and color distortion. Secondly, the scattering and absorption of light by suspended particles and dissolved substances contribute to a hazy and noisy image. Furthermore, ambient light weakens beyond a certain depth, requiring auxiliary light sources and further reducing brightness. Therefore, improving the quality of underwater images, especially achieving reliable enhancement in real and complex underwater environments, has become a crucial problem that urgently needs to be solved.

[0003] Existing methods are mainly divided into non-deep learning-based underwater image enhancement and deep learning-based underwater image enhancement. Non-deep learning-based underwater image enhancement relies heavily on various prior assumptions and physical models, attempting to simulate or reverse-engineer the degradation process through algorithms. However, due to the complex and variable nature of the underwater optical environment, the models or assumptions upon which these methods rely are often difficult to apply universally, leading to unstable or distorted enhancement results in specific scenarios, and a lack of robustness and scene adaptability.

[0004] With the development of deep learning technology, underwater image enhancement methods based on models such as convolutional neural networks and generative adversarial networks have made significant progress, and their enhancement performance is generally superior to traditional non-deep learning methods. However, in pursuit of higher enhancement quality, these models typically have deep and complex designs, resulting in a large number of parameters and high computational complexity. This poses a severe challenge in terms of processing speed (real-time performance) and computational resource consumption, making it difficult to achieve real-time deployment and application on underwater mobile devices (such as autonomous underwater vehicles and onboard vision systems) where computing power, storage space, and power consumption are limited. Therefore, current underwater image enhancement technology faces a prominent contradiction: deep learning-based models struggle to maintain excellent enhancement performance while meeting the practical deployment requirements of lightweight design and high real-time performance.

[0005] In conclusion, developing a real-time underwater image enhancement technology that achieves a high-efficiency balance between performance enhancement, lightweight models, and processing speed is not only of significant theoretical importance but also holds broad application prospects. For example, in marine engineering and underwater operations, it can provide high-quality visual feedback for underwater robots, divers, and underwater inspection equipment, improving operational efficiency and safety. Real-time underwater visual enhancement technology can provide powerful technical support for humanity to better understand and utilize marine resources. Summary of the Invention

[0006] To address the technical challenge of existing deep learning models failing to maintain excellent enhancement performance while meeting the practical deployment requirements of lightweight and high real-time capabilities, this invention proposes a real-time underwater image enhancement method (CFLnet) based on attention fusion and histogram stretching. This method reduces the computational cost and parameter count of deep learning models, achieving an efficient balance between enhancement performance, model lightweightness, and processing speed. A depthwise separable convolutional module (DSC) is designed as the main building block of CFLnet to ensure computational efficiency. However, DSC also has significant drawbacks, often sacrificing computational accuracy in pursuit of speed. Therefore, a cross-domain feature fusion and enhancement module (CFE) is designed after DSC. This module uses an attention mechanism to separately select key features in the channel and spatial domains, optimizing the dual-domain features before weighted fusion to enhance CFLnet's feature enhancement capabilities. However, the key value selection mechanism and dual-domain differences in the CFE module can easily lead to the loss of global details. Therefore, a dynamic feature correction module (DFC) is added, using multi-scale pooling and global histogram stretching to supplement lost global details and achieve dynamic feature correction.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A real-time underwater image enhancement method based on attention fusion and histogram stretching includes the following steps:

[0009] S1. Use 3×3 convolution on the 3-channel raw underwater image to extract basic visual features and output the first feature map with 15 channels.

[0010] S2. Perform a 3×3 depthwise separable convolution (DSC) on the first feature map to deepen spatial association and output a 30-channel second feature map. Perform 3×3 and 1×1 depthwise separable convolutions on the second feature map in parallel. The former enhances the contextual semantics, while the latter preserves spatial feature details, resulting in two 45-channel third and fourth feature maps. Then, perform element-wise addition and fusion on the two to output a 45-channel fifth feature map.

[0011] S3. Perform a 3×3 depthwise separable convolution on the fifth feature map to shrink the channels and smoothly transmit high-dimensional semantics, outputting a 30-channel sixth feature map. Use the cross-domain feature fusion and enhancement module (CFE) to enhance the sixth feature map, obtaining the first enhanced feature map. Add the first enhanced feature map and the second feature map element-wise to obtain a 30-channel seventh feature map. Perform a 3×3 depthwise separable convolution on the seventh feature map to shrink the channels, obtaining a 15-channel eighth feature map. Use the cross-domain feature fusion and enhancement module (CFE) to enhance the eighth feature map, obtaining the second enhanced feature map. Add the second enhanced feature map and the first feature map element-wise to obtain a 15-channel ninth feature map.

[0012] S4. Perform a 3×3 convolution on the ninth feature map to aggregate features and shrink channels, outputting a 3-channel initial enhanced image; use the Dynamic Feature Correction Module (DFC) to correct feature deviations and optimize global details in the initial enhanced image, outputting the final enhanced image.

[0013] Furthermore, the depthwise separable convolutional module (DSC) performs the following processing steps:

[0014] The depthwise convolution step is to independently perform k×k convolution on each channel of the input feature map to extract spatial features, while keeping the number of output channels unchanged.

[0015] Pointwise convolution step: Perform 1×1 convolution on the output of depthwise convolution to achieve cross-channel feature fusion and dimension adjustment;

[0016] Normalization and activation steps: The output of pointwise convolution is processed by batch normalization and ReLU activation function in sequence to obtain the final output of the module.

[0017] Furthermore, the Cross-Domain Feature Fusion and Enhancement (CFE) module's processing includes parallel execution of a channel domain enhancement branch and a spatial domain enhancement branch. The channel domain enhancement branch performs global average pooling on the input features, compressing them into a C×1×1 channel description vector. This vector is then input into a fully connected layer structure that includes dimensionality reduction and expansion, and channel weight coefficients are generated using the Sigmoid function. The channel weight coefficients are multiplied element-wise with the input features to obtain the channel domain enhanced features. The spatial domain enhancement branch performs mean aggregation on the input features along the channel dimension, obtaining a 1×H×W spatial description map. This spatial description map is then subjected to an s×s convolution, and spatial weight coefficients are generated using the Sigmoid function. The spatial weight coefficients are multiplied element-wise with the input features to obtain the spatial domain enhanced features. Finally, the channel domain enhanced features and the spatial domain enhanced features are weighted and fused to obtain the module's output.

[0018] Furthermore, the weighted fusion method involves averaging the channel domain enhancement features and the spatial domain enhancement features.

[0019] Furthermore, the Dynamic Feature Correction (DFC) module's processing includes:

[0020] Multi-scale pooling fusion step: Average pooling and max pooling are performed on the input feature map in parallel, and the results of the two are weighted and fused with a preset weight coefficient α to obtain a fused feature map; Global histogram stretching step: The fused feature map is flattened in the spatial dimension, the global maximum and global minimum values ​​of each channel are calculated, and the original input feature map is normalized and stretched accordingly; Output constraint step: The numerical range of the stretched feature map is restricted to the interval [0,1] to obtain the final output of this module.

[0021] Furthermore, the specific process and formula for step S1 are as follows:

[0022] Underwater image enhancement is defined as: the original underwater image ( This represents an RGB image, where 3 indicates the number of channels. (representing image size), through the model mapping function ( (For model learnable parameters) to generate the final enhanced image It can approximate clear reference images in the dataset in terms of visual effects (detail, color, contrast) and quantitative metrics. The enhancement process is as follows:

[0023] Formula 1: ,

[0024] in, This represents the final enhanced image. Represents the model mapping function. For the model's learnable parameters, Represents the original underwater image. , This represents an RGB image, where 3 indicates the number of channels. Indicates the image size.

[0025] The model enhancement performance target is defined as: and The closer the two are in terms of detail, color, and contrast, the better. The Structural Similarity Index (SSIM) is used; the closer the SSIM is to 1, the higher the similarity and the better the enhancement quality. The formula is as follows:

[0026] Formula 2: ,

[0027] SSIM stands for Structural Similarity Index. To indicate a clear reference image, , Represents the pixel mean. Represents the standard deviation of pixels. , It is a very small constant to avoid the denominator being zero.

[0028] The lightweight model objective is defined as follows: the model should have as few parameters, as little computational cost, and as little memory usage as possible. The total number of floating-point calculations (FLOPs) is the smallest. The smaller the FLOPs, the smaller the computational cost of the model.

[0029] The real-time performance target of the model is defined as follows: the faster the model processes images, the better. Frame rate (FPS) is used; the higher the FPS, the more images the model can process per second, and the faster the processing speed.

[0030] For the original image The basic visual features were extracted using 3×3 convolution, resulting in a 15-channel first feature map. The calculation process is as follows:

[0031] Formula 3: ,

[0032] in, This represents a 3×3 convolution. Represents the first feature map. .

[0033] Furthermore, the specific process and formula for step S2 are as follows:

[0034] Depthwise separable convolution consists of depthwise convolution (DC) and pointwise convolution (PC), as designed below:

[0035] DC is the spatial feature extraction unit of DSC, which extracts the input feature map through grouped convolution. Each channel undergoes convolution independently, accurately capturing spatial correlation information within a single channel (such as local features like image edges and textures), while avoiding interference between features from different channels during the initial extraction stage, ensuring the specificity and purity of spatial feature extraction. Compared to the ordinary convolution Conv model, which performs overall convolution on all channels, DC avoids repeating complex spatial dimension calculations across multiple channels, reducing the computational burden on the module from the outset. The formula is as follows:

[0036] Formula 4: ,

[0037] in, Characteristic map representing DC output, Represents a k×k convolution. Indicates the input feature map, , Indicates the number of channels. The learnable convolutional parameter weights of DC are represented. This indicates element-wise multiplication. This indicates the kernel size.

[0038] PC is a key step in channel feature fusion and dimensionality adjustment, which affects the feature map output by DC. This method integrates channel dimensions and applies to all channels at the same spatial location in the feature map. Cross-channel fusion of feature information from different channels is achieved through weight interaction, and the entire process requires no additional spatial dimension calculations, further improving the module's operational efficiency. The formula is as follows:

[0039] Formula 5: ,

[0040] in, The learnable convolutional parameter weights of the PC are represented. This represents the feature map output by the PC. This represents a 1×1 convolution.

[0041] To enhance the feature representation capabilities and training stability of DSC, after DC and PC, BatchNormal normalization and the ReLU activation function are introduced. BatchNormal can alleviate the "internal covariate shift" problem during training by stabilizing the distribution of feature outputs, helping the model converge faster. ReLU injects non-linear feature transformation capabilities into the module, allowing the model to better fit non-linear feature relationships in complex scenarios. Finally, the output of DSC is obtained. The formula is as follows:

[0042] Formula 6: ,

[0043] in, This represents the feature map output by DSC, BatchNormal represents the normalization function, and ReLU represents the activation function.

[0044] For the first feature map First, use 3×3DSC to obtain a second feature map with 30 channels. Then, by using 3×3DSC and 1×1DSC in parallel, two 45-channel third and fourth feature maps are obtained, respectively. , Then and Element-by-element addition and fusion yields a fifth feature map with 45 channels. The calculation process is as follows:

[0045] Formula 7: ,

[0046] Formula 8: ,

[0047] Formula 9: ,

[0048] Formula 10: ,

[0049] in, Indicates the second characteristic, Indicates the third characteristic, Indicates the fourth characteristic, Indicates the fifth characteristic, This represents a 3×3 depthwise separable convolution. This represents a 1×1 depthwise separable convolution. This indicates element-wise addition.

[0050] Furthermore, the specific process and formula for step S3 are as follows:

[0051] Due to the depthwise separable convolution characteristic of DSC, compared to the general convolution mode that performs convolution on all channels, there is a certain compromise in the integrity of feature interaction. Although the computation speed is improved, there is also a loss of computational accuracy. The Cross-Domain Feature Fusion and Enhancement (CFE) module is designed to fuse channel attention and spatial attention in parallel, rather than sequentially, to prevent adjustments to the feature map in the channel domain from affecting subsequent adjustments in the feature map's spatial domain. The specific design is as follows:

[0052] In the channel domain feature enhancement branch, the input features of the CFE are first compressed using average pooling (AvgPool). The process involves obtaining global channel information and converting the two-dimensional feature map into a one-dimensional vector that retains only channel-dimensional information to capture global statistical features at the channel level. Then, the average pooling result is input into the channel convolution block (ChannelConvolutionBlock). The formula is as follows:

[0053] Formula 11: ,

[0054] Formula 12: ,

[0055] in, This represents the calculation result of average pooling (AvgPool). This represents the calculation result of ChannelConvolutionBlock. This indicates the number of channels in a ChannelConvolutionBlock. This indicates the channel compression factor, which defaults to 3.

[0056] After processing, channel features Channel-dimensional weight coefficients are generated using the Sigmoid function, and finally, the input features of the CFE are... The channel domain features are enhanced by multiplying element-wise with the channel weights to obtain the enhanced channel domain features. The formula is as follows:

[0057] Formula 13: ,

[0058] in, Indicates channel domain enhancement features, This represents the input features of the CFE. The channel-dimensional weight coefficients are generated using the Sigmoid function.

[0059] In the spatial domain feature enhancement branch, the input features of CFE are first aggregated along the channel dimension mean. This generates a single-channel spatial average feature map. Then, the single-channel spatial average feature map is input into a spatial convolution block (SpatialConvolutionBlock) to mine correlation information in the spatial neighborhood, generating a more comprehensive spatial feature map. The formula is as follows:

[0060] Formula 14: ,

[0061] Formula 15: ,

[0062] in, This represents the single-channel spatial average feature map after mean aggregation. Indicates channel dimension, Indicates spatial characteristics, Represents an s×s convolution. This indicates the window size of SpatialConvolutionBlock.

[0063] After processing, spatial features The spatial dimension weight coefficients are generated using the Sigmoid function, and finally, the input features of the CFE are... By multiplying the spatial weights element-wise, feature enhancement in the spatial domain is achieved, resulting in enhanced spatial domain features. The formula is as follows:

[0064] Formula 16: ,

[0065] in, Indicates spatial domain enhancement features;

[0066] Finally, fusion and To obtain the output of CFE The formula is as follows:

[0067] Formula 17: ,

[0068] For the fifth feature map First, a 3×3DSC symmetrical channel shrinkage method is used to smoothly transmit high-dimensional semantics, resulting in the sixth feature map. Then, CFE is used to enhance the feature representation, resulting in the first enhanced feature map. ,Will and By adding elements one by one and supplementing semantic relationships, the seventh feature map is obtained. Continue using 3×3DSC to shrink the channels to obtain the eighth feature map. Then, CFE is used to further optimize the feature representation to obtain the second enhanced feature map. Then and By adding elements one by one to supplement shallow feature information, the ninth feature map is obtained. The calculation process is as follows:

[0069] Formula 18: ,

[0070] Formula 19: ,

[0071] Formula 20: ,

[0072] Formula 21: ,

[0073] Formula 22: ,

[0074] Formula 23: ,

[0075] in, Represents the sixth feature map. This represents the first enhanced feature map. Represents the seventh feature map. This represents the eighth feature map. This represents the second enhanced feature map. Represents the ninth feature map

[0076] Furthermore, the specific process and formula for step S4 are as follows:

[0077] Due to the selective nature of the attention mechanism in the CFE module and the differences between the channel domain and the spatial domain, global detail loss in the feature map is often encountered. A Dynamic Feature Correction (DFC) module is designed to supplement global details. The DFC consists of two parts: multi-scale pooling fusion and global histogram stretching. The specific design is as follows:

[0078] In the multi-scale pooling fusion stage, average pooling (AvgPool) and max pooling (MaxPool) are first used to process the input feature map of DFC in parallel. AvgPool preserves the overall distribution and smoothing region information of the feature map, while MaxPool focuses on local salient features and edge details. Then, the results of these two pooling methods are weighted and fused to balance global consistency and local importance. The formula is as follows:

[0079] Formula 24: ,

[0080] Formula 25: ,

[0081] Formula 26: ,

[0082] in, This represents the calculation result of average pooling (AvgPool). This represents the calculation result of max pooling (MaxPool). This represents the result of multi-scale pooling fusion. This represents the weighted fusion coefficient, which defaults to 0.8.

[0083] During the global histogram stretching phase, the flattening function is first used to stretch the histogram. Spatial dimension from two dimensions Compressed into a one-dimensional vector To eliminate the influence of spatial location on global extremum calculation, the global maximum and minimum features are then calculated, and the dimensions are expanded to determine the global dynamic range. Based on this range, the input features of the DFC are... Perform a global histogram stretching operation. The formula is as follows:

[0084] Formula 27: ,

[0085] Formula 28: ,

[0086] Formula 29: ,

[0087] Formula 30: ,

[0088] in, The result of the Flatten function is that it compresses the dimension range from the height dimension (dim=2) to the width dimension (dim=3) into the same height dimension (dim=2), while keeping other dimensions unchanged. Represents the largest global feature; The minimum feature is represented by , which calculates the minimum value on the flattened height dimension (dim=2); Unsqueeze represents the dimension increase operation, which expands the width dimension (dim=3); This represents the result of global histogram stretching. To avoid a denominator of zero, the default value is 0.04.

[0089] Finally, By limiting the feature output to the range [0,1], the output of DFC is further stabilized. The formula is as follows:

[0090] Formula 31: ,

[0091] in, The feature map represents the output of DFC, Clip represents the constraint function, and [0,1] represents the constraint interval.

[0092] For the ninth feature map First, a 3×3 convolution is used to further shrink the channels and aggregate features to obtain the initial enhanced image. Finally, DFC is used to correct feature bias, completing the final feature optimization and obtaining the final enhanced image. The calculation process is as follows:

[0093] Formula 32: ,

[0094] Formula 33: .

[0095] The beneficial effects of this invention are as follows:

[0096] (1) This invention, through the cascaded and complementary design of a depth-separable convolutional module, a cross-domain feature fusion and enhancement module and a dynamic feature correction module, significantly reduces the computational complexity and number of parameters of the model while maintaining excellent visual enhancement effects, enabling it to process underwater images of common resolutions such as 720P, 1080P, 2K and 4K in real time, effectively overcoming the inherent contradiction between enhancement performance and real-time performance in existing deep learning models.

[0097] (2) The depth-separable convolution module introduced in this invention serves as the basic building block of the network. It decomposes the standard convolution into two steps: depth convolution and pointwise convolution. This reduces the computational load and parameter load of the model from the source, significantly improving computational efficiency and providing a feasible lightweight solution for real-time image enhancement on underwater mobile devices with limited computing resources.

[0098] (3) To address the potential feature interaction deficiencies and accuracy loss caused by depthwise separable convolution, the cross-domain feature fusion and enhancement module introduced in this invention optimizes the channel dimension and spatial dimension response of the feature map by parallel fusion of channel attention and spatial attention mechanisms, and performs weighted fusion to enhance the network's ability to extract and reconstruct color and structural information of underwater images, thereby significantly improving the overall enhancement performance.

[0099] (4) To address the issue of global feature weakening that may be caused by the attention mechanism, the dynamic feature correction module introduced in this invention combines multi-scale pooling fusion and global histogram stretching technology to adaptively adjust the dynamic range of the feature map, effectively supplementing the overall contrast and detail consistency of the image, making the enhanced image more natural and realistic, and also providing a higher quality input basis for subsequent underwater visual perception tasks (such as target detection). Attached Figure Description

[0100] Figure 1 Overall framework diagram of the invention;

[0101] Figure 2 The depthwise separable convolution module (DSC) in step S2 of the present invention;

[0102] Figure 3 The cross-domain feature fusion and enhancement module CFE in step S3 of the present invention;

[0103] Figure 4 The Dynamic Feature Correction Module (DFC) in step S4 of the present invention. Detailed Implementation

[0104] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the present invention is not limited thereto.

[0105] The overall framework diagram of the present invention is as follows: Figure 1 As shown. The real-time underwater image enhancement method based on attention fusion and histogram stretching of the present invention specifically includes the following steps:

[0106] Step S1: Apply a 3×3 convolution to the 3-channel original image to extract basic visual features and output a 15-channel feature map, specifically:

[0107] Underwater image enhancement can be defined as: transforming the original input image (low-quality underwater image) into a higher quality one. (3-channel RGB image, image size) ), through the model mapping function ( (For model learnable parameters) to generate enhanced images It can approximate clear reference images in the dataset in terms of visual effects (detail, color, contrast) and quantitative metrics. The enhancement process is as follows:

[0108] Formula 1: ,

[0109] The model enhancement performance target is defined as: and The closer the two are in terms of detail, color, and contrast, the better. The Structural Similarity Index (SSIM) is used; the closer the SSIM is to 1, the higher the similarity and the better the enhancement quality. The formula is as follows:

[0110] Formula 2: ,

[0111] in, Represents the pixel mean. Represents the standard deviation of pixels. , It is a very small constant to avoid the denominator being zero.

[0112] The lightweight objective of a model can be defined as follows: the model should have as few parameters, as little computational cost, and as little memory usage as possible. The total number of floating-point calculations (FLOPs) should be as small as possible. The smaller the FLOPs, the smaller the computational cost of the model.

[0113] The real-time performance target of the model is defined as follows: the faster the model processes images, the better. Frame rate (FPS) is used; the higher the FPS, the more images the model can process per second, and the faster the processing speed.

[0114] For the original input image The basic visual features are extracted using 3×3 convolution to obtain the first feature map. The calculation process is as follows:

[0115] Formula 3: ,

[0116] Step S2: Use 3×3DSC to deepen spatial associations and output a 30-channel feature map. Then, use 3×3DSC and 1×1DSC in parallel. The former enhances the contextual semantics, while the latter preserves spatial feature details. Finally, the two feature maps are added and fused element-wise to output a 45-channel feature map. Specifically:

[0117] To reduce the computational cost and number of parameters in the model, a depthwise separable convolution module (DSC) is designed. The depthwise separable convolution consists of a depthwise convolution (DC) and a pointwise convolution (PC), as detailed below:

[0118] DC is the spatial feature extraction unit of DSC, which extracts the input feature map through grouped convolution. Each channel undergoes convolution independently, accurately capturing spatial correlation information within a single channel (such as local features like image edges and textures), while avoiding interference between features from different channels during the initial extraction stage, ensuring the specificity and purity of spatial feature extraction. Compared to the ordinary convolution Conv model, which performs overall convolution on all channels, DC avoids repeating complex spatial dimension calculations across multiple channels, reducing the computational burden on the module from the outset. The formula is as follows:

[0119] Formula 4: ,

[0120] in, The learnable convolutional parameter weights of DC are represented. This indicates element-wise multiplication. This indicates the kernel size.

[0121] PC is a key step in channel feature fusion and dimensionality adjustment, which affects the feature map output by DC. This method integrates channel dimensions and applies to all channels at the same spatial location in the feature map. Cross-channel fusion of feature information from different channels is achieved through weight interaction, and the entire process requires no additional spatial dimension calculations, further improving the module's operational efficiency. The formula is as follows:

[0122] Formula 5: ,

[0123] in, The learnable convolutional parameter weights of the PC are represented. This represents the feature map output by the PC.

[0124] To enhance the feature representation capabilities and training stability of DSC, after DC and PC, BatchNormal normalization and the ReLU activation function are introduced. BatchNormal can alleviate the "internal covariate shift" problem during training by stabilizing the distribution of feature outputs, helping the model converge faster. ReLU injects non-linear feature transformation capabilities into the module, allowing the model to better fit non-linear feature relationships in complex scenarios. Finally, the output of DSC is obtained. The formula is as follows:

[0125] Formula 6: ,

[0126] For the first feature map First, use 3×3DSC to obtain the second feature map. Then, by using 3×3DSC and 1×1DSC in parallel, the third and fourth feature maps are obtained respectively. , Then and The fifth feature map is obtained by adding and fusing elements one by one. The calculation process is as follows:

[0127] Formula 7: ,

[0128] Formula 8: ,

[0129] Formula 9: ,

[0130] Formula 10: ,

[0131] Step S3: Use 3×3DSC symmetrical channel shrinking to smoothly transmit high-dimensional semantics, outputting a 30-channel feature map. Then, use CFE to enhance the feature expression, keeping the number of channels unchanged. Add the output features of CFE to the 30-channel features from step S2 element-wise to supplement semantic associations, keeping the number of channels unchanged. Continue using 3×3DSC to shrink the channels, outputting a 15-channel feature map. Then, use CFE to further optimize the feature expression, keeping the number of channels unchanged. Add the output features of the second CFE to the output features from step S1 element-wise to supplement shallow feature information, keeping the number of channels unchanged. Specifically:

[0132] Due to the depthwise separable convolution characteristic of DSC, compared to the general convolution mode that performs convolution on all channels, there is a certain compromise in the integrity of feature interaction. Although the computation speed is improved, there is also a loss of computational accuracy. The Cross-Domain Feature Fusion and Enhancement (CFE) module is designed to fuse channel attention and spatial attention in parallel, rather than sequentially, to prevent adjustments to the feature map in the channel domain from affecting subsequent adjustments in the feature map's spatial domain. The specific design is as follows:

[0133] In the channel domain feature enhancement branch, the input features of the CFE are first compressed using average pooling (AvgPool). The process involves obtaining global channel information and converting the two-dimensional feature map into a one-dimensional vector that retains only channel-dimensional information to capture global statistical features at the channel level. Then, the average pooling result is input into the channel convolution block (ChannelConvolutionBlock). The formula is as follows:

[0134] Formula 11: ,

[0135] Formula 12: ,

[0136] in, This represents the calculation result of AvgPool. This represents the calculation result of ChannelConvolutionBlock. This indicates the number of channels in a ChannelConvolutionBlock. (Default is 3) indicates the channel compression factor.

[0137] After processing, channel features Channel-dimensional weight coefficients are generated using the Sigmoid function, and finally, the input features of the CFE are... The channel weights are multiplied element-wise to enhance the channel domain features, resulting in an enhanced channel domain feature map. The formula is as follows:

[0138] Formula 13: ,

[0139] In the spatial domain feature enhancement branch, the input features of CFE are first aggregated along the channel dimension mean. This generates a single-channel spatial average feature map. Then, the single-channel spatial average feature map is input into a spatial convolution block (SpatialConvolutionBlock) to mine correlation information in the spatial neighborhood, generating a more comprehensive spatial feature map. The formula is as follows:

[0140] Formula 14: ,

[0141] Formula 15: ,

[0142] in, This represents the single-channel spatial average feature map after mean aggregation. Indicates the channel dimension. This indicates the window size of SpatialConvolutionBlock.

[0143] After processing, spatial features The spatial dimension weight coefficients are generated using the Sigmoid function, and finally, the input features of the CFE are... By multiplying the spatial weights element-wise, feature enhancement in the spatial domain is achieved, resulting in an enhanced feature map in the spatial domain. The formula is as follows:

[0144] Formula 16: ,

[0145] Finally, fusion and To obtain the output of CFE The formula is as follows:

[0146] Formula 17: ,

[0147] For the fifth feature map First, a 3×3DSC symmetrical channel shrinkage method is used to smoothly transmit high-dimensional semantics, resulting in the sixth feature map. Then, CFE is used to enhance the feature representation, resulting in the first enhanced feature map. ,Will and By adding elements one by one and supplementing semantic relationships, the seventh feature map is obtained. Continue using 3×3DSC to shrink the channels to obtain the eighth feature map. Then, CFE is used to further optimize the feature representation to obtain the second enhanced feature map. Then and By adding elements one by one to supplement shallow feature information, the ninth feature map is obtained. The calculation process is as follows:

[0148] Formula 18: ,

[0149] Formula 19: ,

[0150] Formula 20: ,

[0151] Formula 21: ,

[0152] Formula 22: ,

[0153] Formula 23: ,

[0154] Step S4: Use 3×3 convolution to further shrink the channels, aggregate features, and output a 3-channel initial enhanced image. Finally, use DFC to correct feature bias, complete the final feature optimization, and output the final enhanced image. Specifically:

[0155] Due to the selective nature of the attention mechanism in the CFE module and the differences between the channel domain and the spatial domain, global detail loss in the feature map is often encountered. A Dynamic Feature Correction (DFC) module is designed to supplement global details. The DFC consists of two parts: multi-scale pooling fusion and global histogram stretching. The specific design is as follows:

[0156] In the multi-scale pooling fusion stage, average pooling (AvgPool) and max pooling (MaxPool) are first used to process the input feature map of DFC in parallel. AvgPool preserves the overall distribution and smoothing region information of the feature map, while MaxPool focuses on local salient features and edge details. Then, the results of these two pooling methods are weighted and fused to balance global consistency and local importance. The formula is as follows:

[0157] Formula 24: ,

[0158] Formula 25: ,

[0159] Formula 26: ,

[0160] in, This represents the calculation result of AvgPool. This represents the calculation result of MaxPool. This represents the result of multi-scale pooling fusion. (Default is 0.8) represents the weighted fusion coefficient.

[0161] In the global histogram stretching stage, the first step is to flatten the flattened histogram. Spatial dimension from two dimensions Compressed into a one-dimensional vector To eliminate the influence of spatial location on global extremum calculation, the global maximum and minimum features are then calculated, and the dimensions are expanded to determine the global dynamic range. Based on this range, the input features of the DFC are... Perform a global histogram stretching operation. The formula is as follows:

[0162] Formula 27: ,

[0163] Formula 28: ,

[0164] Formula 29: ,

[0165] Formula 30: ,

[0166] in, This represents the result of Flatten's calculation. Represents the global maximum feature. Represents the global minimum feature. This represents the result of global histogram stretching. (Default is 0.04) Avoid denominators of zero.

[0167] Finally, By limiting the feature output to the range [0,1], the output of DFC is further stabilized. The formula is as follows:

[0168] Formula 31: ,

[0169] For the ninth feature map First, a 3×3 convolution is used to further shrink the channels and aggregate features to obtain the initial enhanced image. Finally, DFC is used to correct feature bias, completing the final feature optimization and obtaining the final enhanced image. The calculation process is as follows:

[0170] Formula 32: ,

[0171] Formula 33: .

[0172] The above description illustrates the basic principles and specific implementation of this invention. This invention significantly reduces the computational cost and parameter count of the model by introducing a depthwise separable convolution module; effectively enhances the model's feature representation and enhancement capabilities by introducing a cross-domain feature fusion and enhancement module; and further optimizes the global consistency and detail representation of the enhancement effect by introducing a dynamic feature correction module. This invention can achieve real-time enhancement at multiple resolutions, including 720P, 1080P, 2K, and 4K, while ensuring the objectivity and realism of the enhancement results. It is particularly suitable for image enhancement tasks in complex underwater scenarios such as marine resource exploration, underwater edge machinery navigation, and underwater archaeology. It not only provides an efficient and feasible solution for real-time underwater image enhancement but also lays a high-quality image foundation for downstream visual tasks such as target detection.

Claims

1. A real-time underwater image enhancement method based on attention fusion and histogram stretching, characterized in that, Includes the following steps: S1. Use 3×3 convolution on the 3-channel raw underwater image to extract basic visual features and output the first feature map with 15 channels. S2. Perform a 3×3 depthwise separable convolution (DSC) on the first feature map to deepen spatial association and output a 30-channel second feature map. Perform 3×3 and 1×1 depthwise separable convolutions on the second feature map in parallel. The former enhances the contextual semantics, while the latter preserves spatial feature details, resulting in a 45-channel third and fourth feature maps. Then, perform element-wise addition and fusion on the two to output a 45-channel fifth feature map. S3. Perform a 3×3 depthwise separable convolution on the fifth feature map to shrink the channels and smoothly transmit high-dimensional semantics, outputting a 30-channel sixth feature map. Use the cross-domain feature fusion and enhancement module CFE to enhance the features of the sixth feature map to obtain the first enhanced feature map. The first enhanced feature map and the second feature map are added element by element to obtain the seventh feature map with 30 channels; The seventh feature map is subjected to a 3×3 depthwise separable convolution to shrink the channels, resulting in a 15-channel eighth feature map; the eighth feature map is then enhanced using a cross-domain feature fusion and enhancement module to obtain a second enhanced feature map. The second enhanced feature map is added element-wise to the first feature map to obtain the ninth feature map with 15 channels; S4. Perform a 3×3 convolution on the ninth feature map to aggregate features and shrink channels, outputting an initial enhanced image with 3 channels; The Dynamic Feature Correction (DFC) module is used to correct feature deviations and optimize global details in the initial enhanced image, and the final enhanced image is output.

2. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 1, characterized in that, The depthwise separable convolutional module (DSC) processes the following steps: The depthwise convolution step is to independently perform k×k convolution on each channel of the input feature map to extract spatial features, while keeping the number of output channels unchanged. Pointwise convolution step: Perform 1×1 convolution on the output of depthwise convolution to achieve cross-channel feature fusion and dimension adjustment; Normalization and activation steps: The output of pointwise convolution is processed by batch normalization and ReLU activation function in sequence to obtain the final output of the module.

3. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 1, characterized in that, The Cross-Domain Feature Fusion and Enhancement (CFE) module's processing includes parallel execution of channel domain enhancement branches and spatial domain enhancement branches; The channel domain enhancement branch performs global average pooling on the input features, compressing them into a C×1×1 channel description vector; this vector is then input into a fully connected layer structure that includes dimensionality reduction and dimensionality increase, and channel weight coefficients are generated using the Sigmoid function; the channel weight coefficients are then multiplied element-wise with the input features to obtain the channel domain enhanced features; The spatial domain enhancement branch performs mean aggregation on the input features along the channel dimension to obtain a 1×H×W spatial description map; the spatial description map is then subjected to an s×s convolution, and spatial weight coefficients are generated by the Sigmoid function. The spatial weight coefficients are multiplied element-wise with the input features to obtain the spatial domain enhanced features; The channel domain enhancement features and spatial domain enhancement features are weighted and fused to obtain the output of the CFE module.

4. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 3, characterized in that, The weighted fusion method involves averaging the channel domain enhancement features and the spatial domain enhancement features.

5. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 1, characterized in that, The Dynamic Feature Correction Module (DFC) processes the following steps: Multi-scale pooling fusion step: Average pooling and max pooling are performed on the input feature map in parallel, and the results of the two are weighted and fused with a preset weight coefficient α to obtain a fused feature map; Global histogram stretching step: The fused feature map is flattened in the spatial dimension, the global maximum and global minimum values ​​of each channel are calculated, and the original input feature map is normalized and stretched accordingly; Output constraint step: The numerical range of the stretched feature map is restricted to the interval [0,1] to obtain the final output of the DFC module.

6. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 1, characterized in that, In step S1, for the original image The basic visual features were extracted using 3×3 convolution, resulting in a 15-channel first feature map. The calculation process is as follows: Official 3: , in, This represents a 3×3 convolution. Represents the first feature map. .

7. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 1, characterized in that, The specific process and formula for step S2 are as follows: The general definition of a depthwise separable convolutional module (DSC) is: Official 4: , Official 5: , Official 6: ; Based on the above general definition of DSC, the specific execution of step S2 is as follows: Official 7: , Official 8: , Official 9: , Official 10: , in, Characteristic map representing DC output, Represents a k×k convolution. Indicates the input feature map, , Indicates the number of channels. The learnable convolutional parameter weights of DC are represented. This indicates element-wise multiplication. Indicates the kernel size; The learnable convolutional parameter weights of the PC are represented. This represents the feature map output by the PC. Represents a 1×1 convolution; The feature map represents the output of DSC, BatchNormal represents the normalization function, and ReLU represents the activation function. This represents the second feature map. Represents the third feature map. This represents the fourth feature map. This represents the fifth feature map. This represents a 3×3 depthwise separable convolution. This represents a 1×1 depthwise separable convolution. This indicates element-wise addition; For the first feature map First, use 3×3DSC to obtain a second feature map with 30 channels. Then, by using 3×3DSC and 1×1DSC in parallel, the third and fourth feature maps with 45 channels were obtained, respectively. , Then and Element-by-element addition and fusion yields a fifth feature map with 45 channels. .

8. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 1, characterized in that, The specific process and formula for step S3 are as follows: The general definition of the Cross-Domain Feature Fusion and Enhancement (CFE) module is: Official 11: , Official 12: , Official 13: , Official 14: , Official 15: , Official 16: , Official 17: ; Based on the above general definition of CFE, the specific execution of step S3 is as follows: Official 18: , Official 19: , Official 20: , Official 21: , Official 22: , Official 23: , in, This represents the calculation result of average pooling (AvgPool). This represents the calculation result of ChannelConvolutionBlock. This indicates the number of channels in a ChannelConvolutionBlock. This indicates the channel compression factor, which defaults to 3. Indicates channel domain enhancement features, This represents the input features of the CFE. The channel-dimensional weight coefficients are generated using the Sigmoid function; This represents the single-channel spatial average feature map after mean aggregation. Indicates channel dimension, Indicates spatial characteristics, Represents an s×s convolution. Indicates the window size of SpatialConvolutionBlock; Indicates the output characteristics of CFE; Indicates spatial domain enhancement features; Represents the sixth feature map. This represents the second feature map. This represents the first enhanced feature map. Represents the seventh feature map. This represents the eighth feature map. This represents the second enhanced feature map. Represents the ninth feature map; For the fifth feature map First, use 3×3DSC symmetrical channel shrinking to obtain the sixth feature map. Then, CFE is used to enhance the feature representation, resulting in the first enhanced feature map. ,Will and By adding element by element, we obtain the seventh feature map. Continue using 3×3DSC to shrink the channels to obtain the eighth feature map. Then, CFE is used to further optimize the feature representation to obtain the second enhanced feature map. Then and By adding elements one by one to supplement shallow feature information, the ninth feature map is obtained. .

9. The real-time underwater image enhancement method based on attention fusion and histogram stretching according to claim 1, characterized in that, The specific process and formula for step S4 are as follows: The general definition of the Dynamic Feature Correction Module (DFC) is: Official 24: , Official 25: , Official 26: , Official 27: , Official 28: , Official 29: , Official 30: , Official 31: ; Based on the above general DFC definition, step S4 is executed as follows: Official 32: , Official 33: , in, This represents the input feature map of DFC. This represents the calculation result of average pooling (AvgPool). This represents the calculation result of max pooling (MaxPool). This represents the result of multi-scale pooling fusion. This represents the weighted fusion coefficient, which defaults to 0.

8. The result of the Flatten function is that it compresses the dimension range from the height dimension to the width dimension into the same height dimension, while keeping other dimensions unchanged. Represents the largest global feature; The minimum feature is represented by , which calculates the minimum value in the height dimension after flattening; Unsqueeze represents a dimensionality increase operation, which expands the width dimension. This represents the result of global histogram stretching. To avoid a denominator of zero, the default value is 0.04; The feature map represents the output of DFC; Clip represents the constraint function, and [0,1] represents the constraint interval; For the ninth feature map First, a 3×3 convolution is used to further shrink the channels and aggregate features to obtain the initial enhanced image. Finally, DFC is used to correct feature bias, completing the final feature optimization and obtaining the final enhanced image. .

Images