A multi-stage underwater image enhancement method based on transformer
By adopting a multi-stage underwater image enhancement method based on Transformer, the problems of limited receptive field and weak adaptability of existing networks in underwater image enhancement are solved, achieving efficient enhancement of underwater images and improving image quality and visual effects.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2026-03-27
AI Technical Summary
Existing deep learning networks for underwater image enhancement suffer from limitations such as the limited receptive field of convolution operators, inability to capture long-range dependencies in images, inability to fully extract features at different scales, and weak adaptability of static weights, making it difficult to cope with the diversity of water body types and complex degradation levels.
A multi-stage underwater image enhancement method based on Transformer is adopted. By constructing a multi-stage network, including feature extraction, global feature enhancement and local feature refinement stages, the long-term dependencies between similar blocks in the image are captured by the Transformer backbone network, and the reconstruction and refinement are performed by an adaptive neural network structure. The method is trained by combining mean squared error and structural similarity loss functions.
It effectively improves the color shift and low contrast issues caused by backscattering in underwater images, enhances image details and textures, improves the visual perception of underwater images, and adapts to the complexity and diversity of different underwater scenes.
Smart Images
Figure CN116596788B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of digital image processing, and in particular to a multi-stage underwater image enhancement method based on a Transformer. BACKGROUND
[0002] With the increasing shortage of land resources, people have eagerly explored the ocean which contains rich oil and mineral resources. Obtaining high-quality, clear underwater images plays a crucial role in the development of the marine environment and the utilization of resources. However, due to the attenuation and scattering of underwater light, underwater images will have serious color deviation, image blurring and low contrast.
[0003] In recent years, more and more researchers have paid attention to underwater image enhancement to improve the visual quality of underwater images and have achieved good results in improving the quality of underwater images. These methods can be roughly divided into three categories: physical model-based methods, non-physical model-based methods and learning-based methods. The physical model-based method aims to analyze the physical process of image degradation, focusing on constructing a degradation model suitable for underwater images, and estimating the parameters in the model to reverse the entire degradation process to obtain the restored underwater image. The non-physical model-based underwater image enhancement algorithm starts from the pixel intensity of the image and enhances the image by redistributing the pixel intensity values. With the rise of convolutional neural networks and generative adversarial networks, deep learning methods have achieved great success in the field of underwater image enhancement.
[0004] Although some existing deep learning networks can complete the task of processing degraded underwater images to a certain extent, they still have limitations: on the one hand, the convolutional operators used by these networks have limited receptive fields and cannot capture long-range dependencies in images, and cannot fully extract different scale features. On the other hand, they have static weights during inference, and this strategy has weak adaptability to visual input, making it more difficult to face the diversity of water types and the complex degradation of underwater images. SUMMARY
[0005] To solve the above technical problems: some deep learning networks have the following problems in image enhancement processing: on the one hand, the convolutional operators used by these networks have limited receptive fields and cannot capture long-range dependencies in images, and cannot fully extract different scale features. On the other hand, they have static weights during inference, and this strategy has weak adaptability to visual input, making it more difficult to face the diversity of water types and the complex degradation of underwater images.
[0006] The present application provides a multi-stage underwater image enhancement method based on a Transformer, comprising the following steps:
[0007] An image training set is established, and a reference image and an underwater image to be processed are obtained.
[0008] constructing a multi-stage underwater image enhancement network based on a Transformer backbone;
[0009] constructing a loss function to constrain the training of the multi-stage underwater image enhancement network based on the Transformer backbone, and obtaining a trained multi-stage underwater image enhancement network;
[0010] inputting the reference image and the underwater image to be processed into the trained multi-stage underwater image enhancement network in sequence to realize the enhancement processing of the underwater image to be processed.
[0011] Further, the process of constructing the multi-stage underwater image enhancement network based on the Transformer backbone is as follows:
[0012] inputting the underwater image to be processed into the feature extraction network to extract the features of the underwater image at different scales and generate a basic feature map;
[0013] inputting the basic feature map into the first detail preservation module to preserve the details and reduce the information loss when extracting the features of the underwater image at different scales;
[0014] constructing a Transformer backbone to capture the long-term dependency relationship between the similar blocks of the image of the basic feature map and perform global feature enhancement processing on the basic feature map;
[0015] inputting the image after the global feature enhancement processing into the second detail preservation module to preserve the details and reduce the information loss when capturing the long-term dependency relationship between the similar blocks of the image of the basic feature map;
[0016] constructing an adaptive neural network structure to reconstruct and refine the enhanced global structural features, and completing the construction of the multi-stage underwater image enhancement network based on the Transformer backbone.
[0017] Further, the steps of establishing the image training set are as follows: randomly extracting underwater images with different scattering degrees, different color deviation phenomena, and various underwater scenes from the UIEB dataset and the EUVP dataset to establish the image training set.
[0018] Further, the specific steps of inputting the underwater image to be processed into the feature extraction network to extract the features of the underwater image at different scales and generate a basic feature map are as follows:
[0019] The target image to be processed is first extracted by a 3x3 convolution to obtain a low-level feature map;
[0020] The features of the target image at different scales are fully extracted by the feature extraction network to obtain a high-level feature map.
[0021] Further, the feature extraction network comprises a multi-level learning module based on wavelet transform, which fully extracts the features of the target image at different scales through the feature extraction network, and obtains a high-level feature map. The specific steps are as follows:
[0022] First, the discrete wavelet transform is used to obtain the features of the underwater image at three different scales, and the convolution method is used to adjust the feature channels to:
[0023]
[0024] Wherein: represents the input of the multi-level learning module;
[0025] The multi-level learning module uses an attention residual block (ARB) to learn the background information of the underwater image at each scale feature:
[0026]
[0027] The residual attention module comprises three SE modules and a 3x3 convolution, and each residual block adds its input and output element by element through a jump connection to realize information multiplexing;
[0028] Finally, the multi-level learning module uses convolution to adjust the size of the feature channel, and uses IWT to upsample it, and the result after upsampling is added to the output of the multi-level learning module.
[0029]
[0030] Further, the Transformer backbone comprises an improved multi-head self-attention and a multi-layer perception; at the same time, layer normalization is used before each block of the improved multi-head self-attention and a multi-layer perception, and residual connection is also applied after each block.
[0031] Further, the adaptive neural network structure comprises an operation layer and an attention layer,
[0032] The operation layer comprises a plurality of parallel operations, including depth separable convolution with convolution kernel size of 1x1, 3x3, 5x5 and 7x7; dilated convolution with convolution kernel size of 3x3, 5x5 and 7x7; maximum pooling and average pooling operations with expansion rate of 2 and size of 3x3.
[0033] The attention layer structure is based on SENet, and corresponding operations are adaptively selected according to input features, then different operations are given attention weights, and finally the final output enhanced features are obtained by weighting the attention weights and outputs from different operations.
[0034] Further, the first detail retention module and the first detail retention module structure are the same; the first detail retention module comprises a difference channel prior module and a guidance fusion module, and the difference channel prior module and the guidance fusion module are connected in parallel;
[0035] The difference channel prior is the difference between the brightest channel value and the darkest channel value, and the difference channel prior is used to eliminate the influence of back scattering, and the calculation process of the difference channel prior module is represented as:
[0036]
[0037] Wherein, represents the result obtained after the feature map output by each stage is subjected to the difference channel prior, respectively, the brightest channel value and the darkest channel value;
[0038] The calculation process of the guidance fusion module is represented as:
[0039]
[0040] u represents the feature map output from the previous stage, represents the result obtained after the feature map output by the previous stage is subjected to the difference channel prior, represents pixel value multiplication, represents pixel value addition, and v represents the feature map obtained after the guidance fusion module is processed.
[0041] Further, the specific steps of S6 are: the loss function is constructed, and the method combining mean square error loss and structural similarity loss is used to train the multi-stage underwater image enhancement network based on the Transformer backbone in an end-to-end manner;
[0042] L=L MSE +L SSIM (6)
[0043] L MSE is a mean square error loss function, and L SSIM is a structural similarity loss function;
[0044] The comprehensive loss function is represented as:
[0045]
[0046] Wherein, J represents a reference image, and G(I)i Output results representing different stages.
[0047] A multi-stage underwater image enhancement device based on a Transformer, comprising:
[0048] A building module: for building an image training set, obtaining a reference image and an underwater image to be processed;
[0049] A building module: for building a multi-stage underwater image enhancement network based on a Transformer backbone;
[0050] A training module: for building a loss function, constraining the training of the multi-stage underwater image enhancement network based on the Transformer backbone, and obtaining a trained multi-stage underwater image enhancement network;
[0051] A processing module: for sequentially inputting the reference image and the underwater image to be processed into the trained multi-stage underwater image enhancement network, and realizing underwater image enhancement processing of the underwater image to be processed.
[0052] Compared with the existing underwater image enhancement method, the present application proposes an underwater image enhancement method based on a Transformer, constructs a multi-stage overall architecture, and gradually learns the mapping process from the degraded underwater image to the high-quality underwater image, thereby dividing the degraded underwater image processing process into easier sub-processes. The multi-stage network based on the Transformer of the present application comprises a feature extraction stage, a global feature enhancement stage and a local feature refinement stage. The multi-stage strategy can integrate important information in the previous stage, gradually learn the mapping process from the degraded underwater image to the high-quality underwater image, selectively process more important information, discard redundant information, and better guide the task of the next stage. The present application can effectively improve the color deviation and low contrast problem of underwater images caused by backscattering under different underwater scenes (scattered underwater images, dark scene underwater images, and partially blurred underwater images, etc.), while enhancing the details and texture of the images, and improving the visual perception effect of the underwater degraded images. BRIEF DESCRIPTION OF DRAWINGS
[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.
[0054] Figure 1 is a flowchart of the present application;
[0055] Figure 2 is a schematic diagram of a multi-stage learning module structure;
[0056] Figure 3 is a schematic diagram of an efficient Transformer backbone structure;
[0057] Figure 4 is a schematic diagram of an adaptive neural network structure;
[0058] Figure 5 (a) is a comparison diagram based on a model method on the UIEB dataset; (b) is a comparison diagram based on a deep learning method on the UIEB dataset. DETAILED DESCRIPTION
[0059] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.
[0060] Figure 1 is a flowchart of the present application;
[0061] The present application discloses a multi-stage underwater image enhancement method based on Transformer, and comprises the following steps:
[0062] S1: Establish an image training set, obtain a reference image and an underwater image to be processed;
[0063] S2: Construct a multi-stage underwater image enhancement network based on Transformer backbone;
[0064] S3: Construct a loss function, constrain the training of the multi-stage underwater image enhancement network based on Transformer backbone, and obtain a trained multi-stage underwater image enhancement network;
[0065] S4: Input the reference image and the underwater image to be processed into the trained multi-stage underwater image enhancement network in turn, and realize the enhancement processing of the underwater image to be processed.
[0066] The steps S1 / S2 / S3 / S4 are executed in sequence, or the steps S2 / S1 / S3 / S4 are executed in sequence;
[0067] Further, the process of constructing a multi-stage underwater image enhancement network based on Transformer backbone is as follows:
[0068] S21: input the underwater image to be processed into a feature extraction network, extract the features of the underwater image at different scales, and generate a basic feature map;
[0069] S22: input the basic feature map into a first detail preservation module, preserve the details and reduce information loss when extracting the features of the underwater image at different scales;
[0070] S23: build a Transformer backbone to capture the long-term dependency between similar blocks of the image of the basic feature map, and perform global feature enhancement processing on the basic feature map;
[0071] S24: input the image after global feature enhancement processing into a second detail preservation module, preserve the details and reduce information loss when capturing the long-term dependency between similar blocks of the image of the basic feature map;
[0072] S25: build an adaptive neural network structure to reconstruct and refine the enhanced global structural features, and complete the construction of the multi-stage underwater image enhancement network based on the Transformer backbone.
[0073] The steps S21 / S22 / S23 / S24 / S25 are sequentially executed;
[0074] Further, the image training set is established, and the reference image and the underwater image to be processed are obtained as follows:
[0075] In order to be universal and fair, the present application selects two public underwater data sets: UIEB data set and EUVP data set, randomly extracts underwater images with different scattering degrees, different color deviation phenomena and various underwater scenes to establish an image training set, obtain a reference image and an underwater image to be processed.
[0076] Further, the underwater image to be processed is input into a feature extraction network to extract the features of the underwater image at different scales, and a basic feature map is generated as follows:
[0077] S2.1: the target image to be processed is first extracted by a 3x3 convolution to obtain a low-level feature map;
[0078] S2.2: then, the features of the target image at different scales are fully extracted by a feature extraction network to obtain a high-level feature map; the feature extraction network comprises a multi-level learning module based on wavelet transform, which adopts discrete wavelet transform (DWT) and inverse wavelet transform (IWT) to replace down-sampling and deconvolution operations, and specifically comprises the following steps:
[0079] Figure 2 is a multi-level learning module structure diagram;
[0080] The multi-level learning module first obtains three different scale underwater image features using discrete wavelet transform, and uses a convolution method to adjust the feature channel to:
[0081]
[0082] wherein represents the input of the multi-level learning module. Next, the multi-level learning module uses an Attention Residual Block (ARB) to learn the background information of the underwater image at each scale feature:
[0083]
[0084] The Attention Residual Block contains three Squeeze and Excitation (SE) modules and a 3x3 convolution, and each residual block adds its input and output element by element through a skip connection to achieve information multiplexing.
[0085] Finally, the multi-level learning module uses convolution to adjust the size of the feature channel, and uses IWT for upsampling. At the same time, they are added to the output of the upper level:
[0086]
[0087] The multi-level learning module can extract the global structural features in the original image and balance the color information of the image, obtaining a high-level feature map with global structural information in the feature extraction stage. The generated high-level feature map is used as the input of the subsequent global enhancement process to guide the enhancement process in the next stage.
[0088] Figure 3 is a schematic diagram of an efficient Transformer backbone structure; an efficient Transformer backbone is constructed to capture long-term dependencies between similar blocks of images, and the obtained high-level feature image is input into the efficient Transformer backbone for global feature enhancement processing of the basic feature map; the efficient Transformer backbone contains an improved Efficient Multi-Head Attention (EMHA) and a Multiple Perceptron (MLP). At the same time, layer normalization is used before the improved Efficient Multi-Head Attention and a Multiple Perceptron, and residual connection is also applied after each block. The global feature enhanced image
[0089] Further, although the Transformer has excellent global modeling capability, it lacks the ability to preserve local detail information in the early and late stages of the underwater image enhancement task, thus requiring further refinement of the local features of the image obtained in the previous stage.
[0090] Therefore, an adaptive neural network structure is constructed in the local feature reconstruction stage, containing an operation layer and an attention layer, to reconstruct and refine the enhanced global structural features; Figure 4 is a structural diagram of the adaptive neural network structure;
[0091] The adaptive neural network structure contains an operation layer and an attention layer. The operation layer contains multiple parallel operations, including depth separable convolution with convolution kernel size of 1×1, 3×3, 5×5, and 7×7; dilated convolution with convolution kernel size of 3×3, 5×5, and 7×7; maximum pooling and average pooling operations with dilation rate of 2 and size of 3×3.
[0092] The attention layer structure is based on SENet, which adaptively selects the corresponding operation according to the input features, then assigns attention weights to different operations, and finally the final output enhanced features are obtained by weighting the attention weights and the outputs from different operations.
[0093] Further, the difference channel prior is the difference between the brightest channel value and the darkest channel value, and the difference channel prior is used to eliminate the influence of backscatter, and the difference channel prior calculation process is represented as:
[0094]
[0095] wherein, represents the result obtained after the difference channel prior is applied to the feature map output by each stage, are the brightest channel value and the darkest channel value, respectively.
[0096] The calculation process of the guidance fusion module is represented as:
[0097]
[0098] u represents the feature map output from the previous stage, represents the result obtained after the difference channel prior is applied to the feature map output by the previous stage, represents pixel value multiplication, represents pixel value addition, and v represents the feature map obtained after processing by the guidance fusion module.
[0099] Between adjacent stages, the network uses the difference channel prior module and the guidance fusion module to connect, while preserving the detail information, reducing the loss of information during transmission between different stages, and making the network optimization more stable.
[0100] Further, the application constructs a loss function, and uses a method combining mean square error (MSE) loss and structural similarity (SSIM) loss to train the network in an end-to-end manner.
[0101] L=L MSE +L SSIM , (6)
[0102] Under the multi-stage enhancement strategy, the model outputs a result at each stage.
[0103]
[0104] Wherein, J represents a reference image, G(I) i represents the output result of different stages.
[0105] Compared with the existing underwater image enhancement method, the application adopts a multi-stage strategy to integrate important information at the previous stage, gradually learn the mapping process from the degraded underwater image to the high-quality underwater image, selectively process more important information, discard redundant information, and better guide the task of the next stage.
[0106] All experiments of the application are completed in a Python 3.7 (Ubuntu 18.04) environment, the CPU of the experimental server is 48-core Intel(R) Xeon(R) Platinum 8255C CPU, 64GB RAM, GPU acceleration training is used, and the GPU model is NVIDIA Tesla V100 SXM232GB. The network is trained by using a PyTorch deep learning framework. And the Adam optimizer is used for training, wherein the learning rate is 0.0001, the patch size is 128, the batch size is 6, and the training period is 100. In order to reasonably evaluate the running time of the algorithm, the size of all input images is adjusted to 500*375 pixels to enhance the stability of the output.
[0107] Ten underwater image enhancement methods are selected as comparative methods for comparative experiments, and the comparative methods include traditional enhancement methods (MIP, GDCP, UDCP, Retinex-based, Ts) and learning methods (FUnIEGAN, UGAN, UWCNN, CWR, PUIE-Net).
[0108] Figure 5 The enhancement effect figures of the application and 10 comparative methods on the UIEB dataset are shown, compared with the comparative methods, the application can obtain good results for processing underwater dark scene images, degraded images and images from real underwater scenes.
[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A multi-stage underwater image enhancement method based on Transformer, characterized in that, The method comprises the following steps: An image training set is established, reference images and underwater images to be processed are obtained; A multi-stage underwater image enhancement network based on a Transformer backbone is constructed; A loss function is constructed to constrain the training of the multi-stage underwater image enhancement network based on the Transformer backbone, and a trained multi-stage underwater image enhancement network is obtained; The reference images and the underwater images to be processed are sequentially input into the trained multi-stage underwater image enhancement network to realize underwater image enhancement processing of the underwater images to be processed; The process of constructing the multi-stage underwater image enhancement network based on the Transformer backbone is as follows: The underwater image to be processed is input into a feature extraction network to extract features of the underwater image at different scales and generate a basic feature map; The basic feature map is input into a first detail preservation module to preserve the detail information when the features of the underwater image at different scales are extracted and to reduce information loss; A Transformer backbone is constructed to capture the long-term dependency between similar blocks of the basic feature map and perform global feature enhancement processing on the basic feature map; The image after the global feature enhancement processing is input into a second detail preservation module to preserve the detail information when the long-term dependency between similar blocks of the basic feature map is captured and to reduce information loss; An adaptive neural network structure is constructed to reconstruct and refine the enhanced global features, and the construction of the multi-stage underwater image enhancement network based on the Transformer backbone is completed; The specific steps of inputting the underwater image to be processed into the feature extraction network to extract features of the underwater image at different scales and generate a basic feature map are as follows: The target image to be processed is first extracted by a 3×3 convolution to obtain a low-level feature map; The features of the target image at different scales are fully extracted by the feature extraction network to obtain a high-level feature map; The specific steps of fully extracting the features of the target image at different scales by the feature extraction network to obtain a high-level feature map are as follows: Firstly, the discrete wavelet transform is used to obtain the features of underwater images at three different scales, and the convolution method is used to combine the feature channels adjusted to: (1) wherein: represents the input of the multi-level learning module; The multi-level learning module uses a residual attention module ARB to learn the background information of the underwater image at each scale feature: （2） The residual attention module ARB includes three SE modules and a 3×3 convolution, and each residual block adds its input and output element by element through a jump connection to realize information multiplexing; Finally, the multi-level learning module uses convolution to adjust the size of the feature channel, and uses IWT for up-sampling, and the up-sampled result is added to the output of the multi-level learning module: （3）。 2. The multi-stage underwater image enhancement method based on Transformer according to claim 1, wherein, The specific steps of establishing the image training set are as follows: underwater images under different scattering degrees, different color deviation phenomena and various underwater scenes are randomly extracted from the UIEB dataset and the EUVP dataset to establish the image training set.
3. The multi-stage underwater image enhancement method based on Transformer according to claim 1, wherein, The Transformer backbone includes an improved multi-head self-attention and a multi-layer perceptron; at the same time, layer normalization is used before each block of the improved multi-head self-attention and the multi-layer perceptron, and residual connection is also applied after each block.
4. The multi-stage underwater image enhancement method based on Transformer according to claim 1, wherein, The adaptive neural network structure comprises an operation layer and an attention layer, The operation layer comprises a plurality of parallel operations, including depth separable convolution with a convolution kernel size of 1*1, 3*3, 5*5 and 7*7; dilated convolution with a convolution kernel size of 3*3, 5*5 and 7*7; maximum pooling and average pooling operations with a dilated rate of 2 and a size of 3*3; The attention layer structure is based on SENet, and corresponding operations are adaptively selected according to input features, then attention weights are given to different operations, and finally the enhanced features are obtained by weighting the attention weights and the outputs from different operations.
5. The multi-stage underwater image enhancement method based on Transformer according to claim 1, wherein, The first detail retention module and the second detail retention module are of the same structure; the first detail retention module comprises a difference channel prior module and a guidance fusion module, and the difference channel prior module and the guidance fusion module are connected in parallel; The difference channel prior is the difference between the brightest channel value and the darkest channel value, and the difference channel prior is used to eliminate the influence of back scattering, and the calculation process of the difference channel prior module is represented as: （4） wherein, represents the result obtained after the feature map output by each stage passes through the difference channel prior, , respectively the brightest channel value and the darkest channel value; The calculation process of the guidance fusion module is represented as: （5） denotes a feature map from the output of the previous stage, denotes a result obtained after the feature map output by the previous stage is subjected to the difference channel prior, denotes pixel value multiplication, denotes pixel value addition, denotes a feature map obtained after the guiding fusion module is processed.
6. The multi-stage underwater image enhancement method based on Transformer according to claim 1, wherein, The loss function is constructed, and the method combining the mean square error loss and the structural similarity loss is used to train the multi-stage underwater image enhancement network based on the Transformer backbone in an end-to-end manner; （6） is a mean squared error loss function, is a structural similarity loss function; The comprehensive loss function is represented as: （7） wherein J denotes a reference image, denotes the output result of different stages.
7. A multi-stage underwater image enhancement device based on Transformer, characterized in that, It comprises: The establishing module is used to establish an image training set, obtain a reference image and an underwater image to be processed; The constructing module is used to construct the multi-stage underwater image enhancement network based on the Transformer backbone; The training module is used to construct a loss function, constrain the training of the multi-stage underwater image enhancement network based on the Transformer backbone, and obtain the trained multi-stage underwater image enhancement network; The processing module is used to input the reference image and the underwater image to be processed into the trained multi-stage underwater image enhancement network in sequence, and realize the enhancement processing of the underwater image to be processed; The process of constructing the multi-stage underwater image enhancement network based on the Transformer backbone is as follows: The underwater image to be processed is input into a feature extraction network to extract features of the underwater image at different scales and generate a basic feature map; The basic feature map is input into a first detail retention module to retain the detail information when the features of the underwater image at different scales are extracted and reduce information loss; A Transformer backbone is constructed to capture the long-term dependency between image similar blocks of the basic feature map, and the basic feature map is subjected to global feature enhancement processing; The image subjected to the global feature enhancement processing is input into a second detail retention module to retain the detail information when the long-term dependency between the image similar blocks of the basic feature map is captured and reduce information loss; An adaptive neural network structure is constructed to reconstruct and refine the enhanced global features, and the construction of the multi-stage underwater image enhancement network based on the Transformer backbone is completed; The specific steps of inputting the underwater image to be processed into the feature extraction network to extract features of the underwater image at different scales and generate a basic feature map are as follows: The target image to be processed is first extracted low-level feature maps via a 3x3 convolution; The features of the target image at different scales are fully extracted via the feature extraction network to obtain high-level feature maps; The feature extraction network comprises a multi-level learning module based on wavelet transform, and the features of the target image at different scales are fully extracted via the feature extraction network to obtain high-level feature maps, and the specific steps are as follows: Firstly, the discrete wavelet transform is used to obtain the underwater image features at three different scales, and the convolution method is used to combine the feature channels adjusted to: (1) wherein: represents the input of the multi-level learning module; The multi-level learning module adopts a residual attention module ARB to learn the background information of the underwater image at each scale feature: （2） The residual attention module ARB comprises three SE modules and a 3x3 convolution, and each residual block adds its input and output element by element through a jump connection to realize information multiplexing; Finally, the multi-level learning module uses convolution to adjust the size of the feature channel, and uses IWT to upsample it, and at the same time, the result after upsampling is added to the output of the multi-level learning module: （3）。
Citation Information
Patent Citations
Multi-stage progressive underwater image enhancement method
CN114445292A