Multi-scale fusion image enhancement method based on discrete wavelet transform and deep network

Abstract

The present invention discloses a multi-scale fusion image enhancement method based on discrete wavelet transform and deep network, comprising: step 1, using DWT to decompose the image to be enhanced to obtain high-frequency components and low-frequency components; step 2, using a feature extraction module to extract features of the low-frequency component to obtain a first component feature map, and at the same time performing DWT on the feature information obtained in the middle layer, retaining the obtained high-frequency information as the second component feature map; step 3, using a deconvolution network decoding module, upsampling and multi-scale fusion and decoding the first and second feature component feature maps to obtain an enhanced low-frequency component feature map; step 4, performing IDWT on the high-frequency component feature map and the enhanced low-frequency component feature map to obtain a reconstructed image; step 5, using a refinement module to process the image to be enhanced and the reconstructed image, and fusing them to obtain the final enhanced image. The present invention can significantly improve the visual effect and information quality of the image under the condition of low computing resource consumption.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a multi-scale fusion image enhancement method based on discrete wavelet transform and deep network. Background Art

[0002] With the increasing demand for the development of marine resources, underwater robots are increasingly used in underwater data monitoring, seabed detection, marine target acquisition and identification, and other fields. The acquisition of underwater images and video data provides valuable research resources for the study of underwater image processing technology. However, due to the absorption and scattering of water bodies, as well as the influence of suspended particles and underwater light conditions, images captured underwater usually have color distortion, reduced contrast, blurred details, and uneven brightness. These problems not only affect the visual effect of the image, but also increase the difficulty of subsequent image processing and analysis, such as target detection, recognition, and underwater navigation.

[0003] In order to better solve the problems existing in underwater images, researchers and engineers are working on developing a series of technologies and methods such as underwater image enhancement and restoration. By applying these technologies, the quality and efficiency of underwater image processing will be significantly improved. Currently, the commonly used underwater image enhancement technologies mainly include methods based on non-physical models, methods based on physical models, and data-driven methods. Data-driven methods, that is, underwater image enhancement methods based on deep learning, have good effects in color correction and detail retention, but they have strong data dependence, and the enhancement effect is heavily dependent on training data. In addition, the images obtained by methods based on physical models and methods based on non-physical models are not satisfactory. Summary of the invention

[0004] The present invention provides a multi-scale fusion image enhancement method based on discrete wavelet transform and deep network, which can significantly improve the visual effect and information quality of the image with lower computing resource consumption.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0006] A multi-scale fusion image enhancement method based on discrete wavelet transform and deep network, comprising:

[0007] Step 1, using discrete wavelet transform to decompose the image to be enhanced to obtain a high-frequency component feature map and a low-frequency component feature map;

[0008] Step 2, using a feature extraction module to extract features from the low-frequency component feature map obtained in step 1 to obtain a first component feature map; at the same time, the feature information obtained by several intermediate layers of the feature extraction module is subjected to discrete wavelet transformation, and the obtained high-frequency information is retained, which is recorded as a second component feature map;

[0009] Step 3, using a deconvolution network decoding module, upsampling, multi-scale fusion, and decoding the first and second component feature maps obtained in step 2 to obtain an enhanced low-frequency component feature map;

[0010] Step 4, performing inverse discrete wavelet transform on the high-frequency component feature map obtained in step 1 and the enhanced low-frequency component feature map obtained in step 3 to obtain a reconstructed image;

[0011] In step 5, the image to be enhanced and the reconstructed image obtained in step 4 are refined using their respective refine modules, and then fused to obtain the final enhanced image.

[0012] Furthermore, the feature extraction module adopts a main encoding network with ResNet50 as the backbone; the ResNet50 consists of a convolutional layer layer0, four residual blocks layer1~4 and a fully connected layer, and is improved as follows:

[0013] (1) Remove the fully connected layer;

[0014] (2) Convolutional layer layer0: Use a 3x3 convolutional layer and replace the original maximum pooling layer with LeakyReLU;

[0015] (3) Introduce dilated convolution into the conv2 layer in the third residual block layer3 and the fourth residual block layer4, and modify the downsample layer so that its stride is 1;

[0016] (4) A style-based recalibration module, denoted as SRM, is added after layer0, layer1, layer2, layer3, and layer4.

[0017] Furthermore, the deconvolution network decoding module uses two deconvolution layers for upsampling operations, and each deconvolution layer is followed by a LeakyReLU activation function, and each LeakyReLU activation function is followed by an SRM layer; a convolution layer is set at the end of the deconvolution network decoding module to convert the feature map into a final enhanced low-frequency component feature map.

[0018] Further, the several intermediate layers of the feature extraction module described in step 2 refer to the SRM layer behind layer1 and the SRM layer behind layer2, which are respectively recorded as the first SRM layer and the second SRM layer; the feature maps output by the first SRM layer and the second SRM layer are respectively subjected to discrete wavelet transform, and then the high-frequency information therein is processed by using convolution layers respectively, so as to obtain the first part and the second part of the second component feature map respectively;

[0019] In the deconvolution network decoding module, the second part of the second component feature map is first integrated with the first component feature map, and then sent to the first deconvolution layer for processing; after being processed by the LeakyReLU activation function and the SRM layer after the first deconvolution layer, it is integrated with the first part of the second component feature map, and then sent to the second deconvolution layer for processing; after being processed by the LeakyReLU activation function and the SRM layer after the second deconvolution layer, it is sent to the final convolution layer for processing, and the final enhanced low-frequency component feature map is converted.

[0020] Furthermore, the style-based recalibration module, i.e., the SRM layer, includes style pooling and style integration;

[0021] The style pooling is applied to the input features Extract style features, the calculation formula is as follows:

[0022]

[0023]

[0024]

[0025] In the formula, represents the input features, N represents the number of input features, C represents the number of channels, H and W represent the input feature height and width, and Respectively represent the mean and standard deviation of the input features; style vector Represents the summary description of the style features of the cth channel of the nth input feature;

[0026] The style ensemble first uses learnable parameters Perform channel encoding on style features:

[0027]

[0028] In the formula, is a learnable parameter The element corresponding to the cth channel in express Encoded style features;

[0029] Then the BN layer is applied to facilitate training, and the sigmoid function is used as the gating mechanism. The specific formula is as follows:

[0030]

[0031]

[0032]

[0033]

[0034] In the formula, , is the trainable parameter of the cth channel, represents the channel style weight, is the channel style weight corresponding to the nth image and the cth channel; Respectively represent the mean and standard deviation of style features, express Normalized output after scaling and shifting; After the sigmoid function, we finally get ;

[0035] Finally, the original input features are adjusted by the channel style weight K After recalibration, the output characteristics are obtained The results are as follows:

[0036] .

[0037] Furthermore, the refine module is used to process high-frequency information of the input image, including three convolutional layers, and a LeakyReLU activation function and an SRM layer are set between two adjacent convolutional layers.

[0038] Furthermore, each discrete wavelet transform generates four subbands: LL subband, LH subband, HL subband and HH subband, among which the LL subband is used as the low-frequency component obtained by the corresponding discrete wavelet transform, and the LH subband, HL subband and HH subband are all used as the high-frequency components obtained by the corresponding discrete wavelet transform.

[0039] The present invention provides a multi-scale fusion image enhancement method based on discrete wavelet transform and deep network, which mainly includes a main network module with improved ResNet50 feature extraction encoding and multi-scale fusion decoding, and an original image mapping refine module. The introduction of DWT can decompose the image into sub-bands of different scales, capture information of different frequencies, and enhance the robustness of the network to noise and distortion. Low-frequency components are insensitive to noise, while high-frequency components focus on details, which can make the network perform better when extracting noisy image features. By fusing DWT with CNN, the network can process low-frequency and high-frequency information at the same time, improving the richness and effectiveness of feature extraction. At the same time, high-frequency and low-frequency information are separated, and information of different frequencies can be processed separately. In this way, the computational amount of convolution operations can be reduced and the overall computational efficiency can be improved. Compared with the existing underwater scene image enhancement technology, the scheme of the present invention has the following advantages:

[0040] (1) In order to solve the problem of high computational load in traditional CNN when processing full-resolution images, the present invention can significantly reduce the computational burden by reducing the image size after DWT decomposition. The low-frequency components after DWT decomposition are insensitive to noise and distortion, while the high-frequency components can capture changes in image details. The present invention combines this information to enable the network to perform better when processing noisy or distorted images.

[0041] (2) The present invention improves ResNet50 as the main network module for backbone feature extraction encoding and multi-scale fusion deconvolution decoding, which improves the model's ability to express different image features and its ability to understand and process information at different scales, and improves the accuracy and robustness of image processing. Through multi-level deconvolution operations, the decoder can restore image details layer by layer, further improving the visual quality of the reconstructed image.

[0042] (3) It has strong practicality and generalization ability. The present invention has achieved relatively high structural similarity and peak signal-to-noise ratio accuracy on the public datasets UIEB and underwater_imagement. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a multi-scale fusion method based on discrete wavelet transform and deep network in an embodiment of the present invention.

[0044] Figure 2 Schematic diagram of each module of the model of the embodiment of the present invention.

[0045] Figure 3 , Figure 4 They are respectively the effect diagrams comparing different deep learning methods on two different images in the UIEB test set according to the embodiments of the present invention; wherein (a) is the original image, (b) to (i) correspond to the effect diagrams obtained by DICAM, SCNet, LitetEhanceNet, CLCC, DEANet, PUIENet_MC, DeepWaveNet, and the present invention (MFDWT), and (j) is the true label.

[0046] Figure 5 The figures are a comparison of the renderings of different deep learning methods on the underwater_imagement test set according to the embodiments of the present invention; wherein (a) is the original image, (b) to (i) correspond to the renderings obtained by DICAM, SCNet, LitetEhanceNet, CLCC, DEANet, PUIENet_MC, DeepWaveNet, and the present invention (MFDWT), respectively, and (j) is the true label.

[0047] Figure 6The following are the effect diagrams of different traditional enhancement methods compared on the UIEB test set according to the embodiments of the present invention; (a) is the original image, (b) to (k) correspond to the effect diagrams obtained by ACCE-D, HFM, l2uwe, MMLE, PCDE, TIP2016, UIEWCD, UNTV, WWPF, and the present invention (MFDWT), respectively. DETAILED DESCRIPTION

[0048] The following is a detailed description of an embodiment of the present invention. This embodiment is based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process to further explain the technical solution of the present invention.

[0049] This embodiment provides a multi-scale fusion image enhancement method based on discrete wavelet transform and deep network. Figure 1 , including the following steps:

[0050] Step 0, divide the image pairs with reference images (labels) in the two public underwater image datasets UIEB and underwater_imagement into training sets and test sets, use the training set to train the model, and use the test set to verify and test the trained model. The final model can be used for the image enhancement processing of the present invention, and the internal structure and training of the specific model are synchronously provided in the subsequent enhancement method of the model for the image to be enhanced.

[0051] The data comes from the public datasets UIEB and underwater_imagement. The UIEB dataset is the first real-world underwater image dataset with reference images. Reference image generation uses candidate underwater images to generate potential reference images through 12 image enhancement methods, with a total of 890 pairs, of which 700 pairs are classified as training sets, 190 pairs are classified as validation sets and test sets, and the image resolution is inconsistent; the underwater_imagement dataset is a subset of Imagenet containing underwater images selected for CycleGAN training, and it is manually divided into two categories based on visual inspection. Groundtruth is generated by CycleGAN, with a total of 6128 pairs, of which 5520 pairs are classified as training sets, 608 pairs are classified as validation sets and test sets, and their size is unified as 256*256.

[0052] Step 1: Use discrete wavelet transform (DWT for short) to decompose the image to be enhanced to obtain a high-frequency component feature map and a low-frequency component feature map.

[0053] The present invention uses a DWT module to divide the input image, i.e., the image to be enhanced, into a high-frequency component and a low-frequency component. The implementation process is as follows: After an image passes through DWT, four subbands will be generated, and the four subbands are: (1) LL subband: This is a low-frequency component that contains most of the information of the input matrix. It is usually used to approximate the original image because the low-frequency information retains the main structure and content of the image. (2) LH subband: This is a high-frequency component in the horizontal direction, which represents the edge and detail information of the image in the horizontal direction. (3) HL subband: This is a high-frequency component in the vertical direction, which represents the edge and detail information of the image in the vertical direction. (4) HH subband: This is a high-frequency component in the diagonal direction, which represents the edge and detail information of the image in the diagonal direction. These four subbands together constitute the decomposition result of the DWT on the image, and they respectively contain the information of the image in different frequencies and directions. Among them, the LL subband usually contains the most information, while the LH, HL and HH subbands respectively contain high-frequency information in different directions.

[0054] The basic function of two-dimensional discrete wavelet transform is:

[0055]

[0056] Where φ(x,y) is a separable two-dimensional scaling function and φ() is a one-dimensional scaling function; , , All are "direction-sensitive" separable two-dimensional wavelet functions, and they represent the grayscale changes along the horizontal direction of the column, the vertical direction of the row, and the diagonal direction of the edge; ψ() is a one-dimensional wavelet function. The two-dimensional discrete wavelet transform can be obtained by generalizing the one-dimensional discrete wavelet transform. Using the two-dimensional Mallat algorithm and a separable filter for wavelet transform, it is essentially using a one-dimensional filter to perform a one-dimensional wavelet transform on the rows and columns of the image data respectively. The wavelet basis used here is the Haar wavelet, and its formula is defined as follows:

[0057]

[0058] The corresponding scaling function is:

[0059]

[0060] By generalizing the one-dimensional Haar wavelet transform, we can further obtain the implementation process of the two-dimensional Haar wavelet transform, and the corresponding four filters are:

[0061]

[0062]

[0063] By using the above four filters in sequence to calculate the local area formed by the input image, an element in the corresponding subband after wavelet decomposition can be obtained, that is:

[0064]

[0065] In order to unify the input of the image, the image resolution of the input model is uniformly adjusted to 256*256. In order to improve the generalization ability of the model obtained by the final training, reduce overfitting, and enhance robustness, it is necessary to randomly crop, center crop, and randomly enhance the input image of the training set to train the model, and the data enhancement operation includes horizontal flipping, vertical flipping, and rotation. Some hyperparameters in the model training process of the present invention are as follows: the optimizer uses Adam (Adaptive MomentEstimation), the learning rate is 0.0002, the first-order momentum decay coefficient and the second-order momentum decay coefficient use the default values ​​of 0.9 and 0.999, respectively, and the learning rate is updated with a decay coefficient of 0.1 at 150 epochs. The batch size is 4, and a total of 310 epochs are trained. The CPU of the experiment is Intel(R) Xeon(R) CPU, the CPU memory is not less than 64G, and the solid-state hard disk is not less than 512G. Use GPU accelerated training, the GPU is GeForce RTX 3090, Ubuntu 20.04.6 LTS system or above is required, and the pytorch deep learning framework is used to train the network. The system environment requires Pytorch 1.12.1 and Python3.9.18.

[0066] Step 2, use the feature extraction module to extract features from the low-frequency component feature map obtained in step 1 to obtain a first component feature map; at the same time, the feature information obtained by several intermediate layers of the feature extraction module is subjected to discrete wavelet transformation, and the obtained high-frequency information is retained, which is recorded as the second component feature map.

[0067] The feature extraction module of this embodiment adopts a main encoding network with ResNet50 as the backbone, and improves the original ResNet50 so that the modified network can process high-resolution images in a larger receptive field while retaining more detail information. The original ResNet50 is improved as follows:

[0068] (1) Remove the fully connected layer;

[0069] (2) Convolutional layer layer0: Use a 3x3 convolutional layer and replace the original maximum pooling layer with LeakyReLU;

[0070] (3) The conv2 layer in the third residual block layer3 and the fourth residual block layer4 introduces dilated convolution to increase the receptive field, and the downsample layer is modified to make its stride 1 to avoid downsampling;

[0071] (4) A style-based recalibration module, denoted as SRM, is added after layer0, layer1, layer2, layer3, and layer4.

[0072] The feature map obtained by the output of the above-mentioned improved ResNet50 is recorded as the first component feature map.

[0073] The several intermediate layers of the feature extraction module described in this step 2 refer to the SRM layer behind layer1 and the SRM layer behind layer2, which are respectively recorded as the first SRM layer and the second SRM layer; the feature maps output by the first SRM layer and the second SRM layer are respectively subjected to discrete wavelet transform, and then the high-frequency information therein is processed using convolutional layers to obtain the first part and the second part of the second component feature map, respectively, to constitute the second component feature map.

[0074] Step 3: Use the deconvolution network decoding module to upsample, multi-scale fuse and decode the first and second component feature maps obtained in step 2 to obtain an enhanced low-frequency component feature map.

[0075] The deconvolution network decoding module of this embodiment uses two deconvolution layers (ConvTranspose2d) for upsampling operations, and a LeakyReLU activation function is followed by each deconvolution layer, and an SRM layer is followed by each LeakyReLU activation function; a convolution layer (Conv2d) is set at the end of the deconvolution network decoding module to convert the feature map into the final enhanced low-frequency component feature map.

[0076] In the deconvolution network decoding module, the second part of the second component feature map is first integrated with the first component feature map, and then sent to the first deconvolution layer for processing; after the LeakyReLU activation function and SRM layer processing after the first deconvolution layer, it is integrated with the first part of the second component feature map, and then sent to the second deconvolution layer for processing; after the LeakyReLU activation function and SRM layer processing after the second deconvolution layer, it is sent to the final convolution layer for processing, and the final enhanced low-frequency component feature map is converted. The nonlinear activation function LeakyReLU is used, and the negative slope is set to the default value 0.01.

[0077] In the encoder-decoder network formed by the feature extraction module and the deconvolution network decoding, the SRM layer is added after each layer. SRM first extracts style information from each channel of the feature map through the style pooling, and then estimates the recalibrated weight of each channel through the channel-independent style integration. By incorporating the relative importance of individual styles into the feature map, SRM effectively enhances the representation ability of CNN.

[0078] The SRM layer is divided into style pooling and style integration. Style pooling uses the channel statistics (mean and standard deviation) of each feature map as the style feature. Given the input feature , Represents the input sample, N represents the number of input samples, C represents the number of channels, H and W represent the height and width of the image, then the calculation formula of the style feature is as follows:

[0079]

[0080]

[0081] .

[0082] Style Vector Represents the style information features of the nth image and the cth channel. The style vector is represented As input, the style ensemble operator uses learnable parameters Perform channel encoding:

[0083]

[0084] in Represents the encoded style features, which can be regarded as a channel-independent fully connected layer with two input nodes and one output node, where the bias term is absorbed into the subsequent BN layer. Then, BN is applied to facilitate training, and the sigmoid function is used as the gating mechanism. The specific formula is as follows:

[0085]

[0086]

[0087]

[0088]

[0089] in, is the affine transformation parameter, which is the trainable parameter of the cth channel; K represents the channel style weight, is the channel style weight corresponding to the nth image and the cth channel; Represent the mean and standard deviation of style features respectively. express Normalized output after scaling and shifting; After the sigmoid function, we finally get .

[0090] Finally, the original input X is recalibrated by the weight G, and the output is obtained The results are as follows:

[0091] .

[0092] Step 4: Perform an inverse discrete wavelet transform (IDWT) on the high-frequency component feature map obtained in step 1 and the enhanced low-frequency component feature map obtained in step 3 to obtain a reconstructed image.

[0093] For the two-dimensional inverse discrete wavelet transform (2D-IDWT), this process is performed in two-dimensional space. The wavelet transform is reversible, so the sub-images obtained after decomposition can be combined to obtain the reconstructed image.

[0094] In step 5, the image to be enhanced and the reconstructed image obtained in step 4 are refined using their respective refine modules, and then fused to obtain the final enhanced image.

[0095] Each refine module is used to process the high-frequency information of the image and includes three convolutional layers. A LeakyReLU activation function and an SRM layer are set between two adjacent convolutional layers.

[0096] To facilitate understanding of the technical effects of the present invention, the following comparison is provided between the present invention and the traditional method:

[0097] Table 1 and Figure 3 , Figure 4 The comparison of different underwater enhancement methods on UIEB in the past two or three years is shown. The deep learning methods include DICAM, SCNet, LitetEnhanceNet, CLCC, DEANet, PUIENet_MC, and DeepWaveNet. The main comparisons are the reference indicators, non-recommended indicators, test speed (FPS) and parameter quantity (Params) of different enhancement methods between different algorithms. As can be seen from Table 1, on the UIEB test set, the present invention outperforms other state-of-the-art methods in terms of the full reference indicators PSNR and SSIM. Figure 3 , Figure 4It can be seen that by adding DWT to separate high-frequency and low-frequency images, the enhancement effect is better than other methods. In addition, the UCIQE index also shows good performance in non-critical indicators; although the model is large, the image processing capability has reached a speed of 50 frames per second, which shows the effectiveness of the method in this paper.

[0098]

[0099] Note: SSIM-Structural Similarity Index; PSNR-Peak Signal-to-Noise Ratio; LPIPS-Learning Perceptual Patch Similarity; UCIQE-Underwater Color Image Quality Evaluation; UIQM-Underwater Image Quality Evaluation; FPS-Frame Rate; FLOPs-Floating Point Operations; Parameters-Parameter Quantity. Black-highest performance; underlined-second performance.

[0100] Table 2 and Figure 5 The experimental results of the present invention and other state-of-the-art methods on the underwater_imagement test set are shown. On the underwater_imagement test set, the various indicators of the present invention also achieve good results. Considering the low resolution of the data set, the model effect is slightly reduced. The present invention is more suitable for high-resolution data images and has a better enhancement effect on high-resolution image tasks.

[0101]

[0102] In order to further verify the effectiveness of the discrete wavelet transform and multi-scale fusion of the deep network in the method of the present invention, the present invention is also compared with the traditional underwater enhancement methods. The traditional methods are mainly ACCE-D, HFM, l2uwe, MMLE, PCDE, TIP2016, UIEWCD, UNTV, WWPF, and the effect is shown in Figure 6 It can be seen that the method of the present invention is closer to the real land scene and has no obvious color cast or polarization.

[0103] The above embodiments are preferred embodiments of the present application. Ordinary technicians in this field can also make various changes or improvements on this basis. Without departing from the overall concept of the present application, these changes or improvements should fall within the scope of protection required by the present application.

Claims

1. A multi-scale fusion image enhancement method based on discrete wavelet transform and deep network, characterized in that: include: Step 1, using discrete wavelet transform to decompose the image to be enhanced to obtain a high-frequency component feature map and a low-frequency component feature map; Step 2, using a feature extraction module to extract features from the low-frequency component feature map obtained in step 1 to obtain a first component feature map; at the same time, the feature information obtained by several intermediate layers of the feature extraction module is subjected to discrete wavelet transformation, and the obtained high-frequency information is retained, which is recorded as a second component feature map; The feature extraction module adopts a main encoding network with ResNet50 as the backbone; the ResNet50 consists of a convolutional layer layer0, four residual blocks layer1~4 and a fully connected layer, and is improved as follows: (1) Remove the fully connected layer; (2) Convolutional layer layer0: Use a 3x3 convolutional layer and replace the original maximum pooling layer with LeakyReLU; (3) Introduce dilated convolution into the conv2 layer in the third residual block layer3 and the fourth residual block layer4, and modify the downsample layer so that its stride is 1; (4) A style-based recalibration module, denoted as SRM, is added after layer0, layer1, layer2, layer3, and layer4; Step 3, using a deconvolution network decoding module, upsampling, multi-scale fusion, and decoding the first and second component feature maps obtained in step 2 to obtain an enhanced low-frequency component feature map; The deconvolution network decoding module uses two deconvolution layers to perform upsampling operations, and a LeakyReLU activation function is followed by each deconvolution layer, and an SRM layer is followed by each LeakyReLU activation function; a convolution layer is set at the end of the deconvolution network decoding module to convert the feature map into a final enhanced low-frequency component feature map; Step 4, performing inverse discrete wavelet transform on the high-frequency component feature map obtained in step 1 and the enhanced low-frequency component feature map obtained in step 3 to obtain a reconstructed image; In step 5, the image to be enhanced and the reconstructed image obtained in step 4 are refined using their respective refine modules, and then fused to obtain the final enhanced image.

2. The multi-scale fusion image enhancement method based on discrete wavelet transform and deep network according to claim 1 is characterized in that: The several intermediate layers of the feature extraction module described in step 2 refer to the SRM layer behind layer1 and the SRM layer behind layer2, which are respectively recorded as the first SRM layer and the second SRM layer; the feature maps output by the first SRM layer and the second SRM layer are respectively subjected to discrete wavelet transform, and then the high-frequency information therein is processed by using convolution layers respectively, so as to obtain the first part and the second part of the second component feature map respectively; In the deconvolution network decoding module, the second part of the second component feature map is first integrated with the first component feature map, and then sent to the first deconvolution layer for processing; After being processed by the LeakyReLU activation function and the SRM layer after the first deconvolution layer, it is integrated with the first part of the second component feature map and then sent to the second deconvolution layer for processing; After being processed by the LeakyReLU activation function and the SRM layer after the second deconvolution layer, it is sent to the final convolution layer for processing and converted to obtain the final enhanced low-frequency component feature map.

3. The multi-scale fusion image enhancement method based on discrete wavelet transform and deep network according to claim 1 is characterized in that: The style-based recalibration module, i.e., the SRM layer, includes style pooling and style integration; The style pooling is applied to the input features Extract style features, the calculation formula is as follows: ； ； ； In the formula, represents the input features, N represents the number of input features, C represents the number of channels, H and W represent the input feature height and width, and Respectively represent the mean and standard deviation of the input features; style vector Represents the summary description of the style features of the cth channel of the nth input feature; The style ensemble first uses learnable parameters Perform channel encoding on style features: ； In the formula, is a learnable parameter The element corresponding to the cth channel in express Encoded style features; Then the BN layer is applied to facilitate training, and the sigmoid function is used as the gating mechanism. The specific formula is as follows: ； ； ； ； In the formula, , is the trainable parameter of the cth channel, represents the channel style weight, is the channel style weight corresponding to the cth channel of the nth image; Represent the mean and standard deviation of style features respectively. express Normalized output after scaling and shifting; After the sigmoid function, we finally get ; Finally, the original input features are adjusted by the channel style weight K After recalibration, the output characteristics are obtained The results are as follows: 。 4. The multi-scale fusion image enhancement method based on discrete wavelet transform and deep network according to claim 1, characterized in that: The refine module is used to process the high-frequency information of the input image, and includes three convolutional layers. A LeakyReLU activation function and an SRM layer are set between two adjacent convolutional layers.

5. The multi-scale fusion image enhancement method based on discrete wavelet transform and deep network according to claim 1 is characterized in that: Each discrete wavelet transform generates four subbands: LL subband, LH subband, HL subband and HH subband. The LL subband is used as the low-frequency component obtained by the corresponding discrete wavelet transform, and the LH subband, HL subband and HH subband are used as the high-frequency components obtained by the corresponding discrete wavelet transform.

Images