Underwater image enhancement method based on convolutional neural network

Abstract

The invention discloses an underwater image enhancement method based on a convolutional neural network. Although underwater image enhancement based on a convolutional neural network obtains a good progress, a special and high-quality underwater image data set limited by an underwater environment is still in a relatively deficient state, and in the face of the obstacle difficult to cross, some scholars turn sight to a generative adversarial network (GAN). Although the GAN-based image processing technology has been developed for a long time, the problem of lack of understanding of the overall structure still exists, and the edge of the image cannot be well recovered in a high-resolution weak-texture image. The invention relates to an underwater image enhancement method, in particular to an underwater image enhancement method based on a CNN (Convolutional Neural Network) by improving a U-shaped Transform network used for underwater enhancement, and discloses the underwater image enhancement method based on the CNN.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers and relates to an image processing method, in particular to an underwater image enhancement method based on a convolutional neural network. BACKGROUND

[0002] As an important carrier of underwater information, underwater images are a powerful tool for exploring the ocean. However, due to the complex imaging environment underwater, underwater images often have problems such as color distortion and blurring. This brings a lot of trouble to subsequent related research work. With the development of image processing technology, underwater image enhancement technology has also made a lot of progress. However, traditional image enhancement algorithms often have difficulty in accurately processing the complex optical phenomena unique to underwater. Traditional image enhancement techniques can be divided into three categories: spatial domain method, transform domain method and color constancy theory method.

[0003] The spatial domain method mostly guides the pixel space to perform a specified distribution to obtain better visual effects. This method is further divided into gray scale transformation method and spatial filter method. In the gray scale transformation method, stretching operation is a particularly effective means, which aims to increase the dynamic range of the image and improve the overall contrast; but when removing a wide range of noise or processing high dynamic range images, the spatial domain method performs poorly.

[0004] The transform domain method performs mapping processing on the original image, and obtains a new corresponding domain after mapping. Commonly used are Fourier transform, wavelet transform and Laplace transform, which make some properties of the image more easily analyzed and modified. Based on the Laplace transform, Iqbal et al. proposed decomposing the underwater image into low-frequency and high-frequency components using the Laplace transform. Different adjustment methods are used for different parts to obtain the final image. Although the transform domain method can accurately control the different frequency components of the image and is very effective in complex image enhancement tasks, it means that researchers need more algorithm knowledge and processing skills, and the implementation difficulty is relatively high.

[0005] Color constancy refers to the fact that a person's perception of the color of an object's surface does not change with the change of the illuminating light. Based on this characteristic, Edwin proposed the Retinex theory, which decomposes the image into two components: illumination and reflection, and the image observed by the observer is obtained by multiplying the two components. Although the method based on color constancy can simulate the perception of the human eye to color and thus process the image color more naturally, such a method involves complex color theory, and its implementation and adjustment are relatively difficult.

[0006] Although researchers have applied classic digital image processing techniques to underwater images and achieved good results, the difficulty is usually high, and the complex and variable conditions of the water body also challenge its applicability. SUMMARY

[0007] Although the underwater image enhancement based on the convolutional neural network has made good progress, but limited by the special underwater environment, the high-quality underwater image dataset is still in a relatively scarce state, in the face of this difficult to overcome obstacles, some scholars turn their attention to the generative adversarial network (GAN). Although the image processing technology based on GAN has made great progress, but there is still a lack of understanding of the overall structure of the problem, in the high-resolution weak texture picture can not be well restored its edge. The application improves the U-shaped transformer network for underwater enhancement, and discloses a convolutional neural network (CNN) based underwater image enhancement method.

[0008] The application selects the advantages of combining convolutional neural network and transformer, optimizes and improves the U-shaped transformer architecture for underwater image enhancement, adds a multi-scale feature extraction module based on convolutional neural network, and compares and evaluates the qualitative and quantitative comparison of multiple underwater image enhancement methods, and verifies the effectiveness through the ablation experiment. Finally, based on the hybrid network, a simple system for underwater image enhancement is designed. The advantages of combining CNN and transformer are added to the underwater image enhancement based on transformer. The multi-scale feature enhancement module based on convolutional neural network is compared with multiple underwater image enhancement models; based on the optimized model, a system for underwater image enhancement is designed to realize the enhancement function of underwater image.

[0009] A kind of underwater image enhancement method based on convolutional neural network, comprising the following steps, based on U-shaped transformer architecture, join the multi-scale feature extraction module based on convolutional neural network, realize underwater image enhancement.

[0010] In the application, a multi-scale feature enhancement module (MSFE) based on CNN is added in the process of obtaining shallow features, so that the features input to the later layer can have a more bottom expression.

[0011] In the application, image feature data is obtained, then based on U-shaped transformer architecture, a multi-scale feature extraction module based on convolutional neural network is added, to realize underwater image enhancement.

[0012] In the application, the original input is input to the multi-scale feature enhancement module after convolution.

[0013] In the application, in the multi-scale feature enhancement module, a kernel selection is set to automatically select the size of convolution kernel K1 and K2, and through convolution kernels of different sizes, feature maps corresponding to different size receptive fields are obtained, represented as:

[0014]

[0015] where, * represents convolution operation, W, b are corresponding convolution weights and bias.

[0016] Two feature maps are first halved in channel number by 1*1 convolution to obtain r1 and r2, which are merged in channel dimension, and after the processing of the pooling layer, the obtained result is taken as input to obtain the attention map s through convolution and Sigmoid function activation, which can be expressed as:

[0017]

[0018] wherein, Sigmoid activation function is represented, m avg and m max are average attention and maximum attention respectively.

[0019] The generated attention map s contains two channels for weighting the corresponding feature map, which is multiplied with the corresponding feature map and then added, and then the feature map is expanded to the original channel number, which can be expressed as:

[0020]

[0021]

[0022] wherein, w1 and w2 are weighted feature maps, a expanded is used to expand the feature map to the original channel number, element-wise multiplication is represented, W2, b2 are corresponding convolution weights and bias.

[0023] Finally, the obtained matrix is element-wise multiplied with the original input feature, and multi-scale feature enhancement is realized on the output result y. It can be expressed as:

[0024] wherein, element-wise multiplication is represented, W3 and b3 are corresponding convolution weights and bias.

[0025] The core of the application is to capture information of different spatial ranges by using different scale receptive fields, to adaptively emphasize or weaken certain features through learned weights, and to improve the information processing capability of the model. The application discloses the application of the above-mentioned underwater image enhancement method based on convolutional neural network in image enhancement.

[0026] The application discloses a storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to realize the underwater image enhancement method based on a convolutional neural network.

[0027] The application discloses a computer device, which comprises a storage medium, a processor and a computer program stored in the storage medium and capable of running on the processor, and the processor realizes the underwater image enhancement method based on a convolutional neural network when executing the computer program.

[0028] Underwater images are important carriers of underwater information, and generally have a series of problems such as color distortion and low contrast. Today, when the exploration of the ocean is becoming more and more important, these problems seriously affect the quality and practical value of the images, and hinder the development of many fields such as marine scientific research, seabed biological investigation and underwater engineering. Therefore, the research on underwater image enhancement is of great significance. Traditional enhancement techniques often have difficulty in accurately processing the complex optical phenomena specific to underwater images. The application combines the advantages of convolutional neural networks and Transformer models, improves the U-shaped Transformer architecture for underwater image enhancement, improves the image enhancement performance, and completes the design and implementation of the underwater image enhancement system. The main work and innovation points are as follows: a hybrid network model for underwater image enhancement is proposed by combining the advantages of convolutional neural networks and Transformer networks. In order to improve the quality of image enhancement, a multi-scale feature enhancement module based on convolutional neural networks is introduced into the Transformer architecture. The effectiveness of the method is verified by experimental results on real underwater image datasets. At the same time, based on the hybrid network model proposed in the application, a system for underwater image enhancement is designed and implemented, which provides core functions such as image uploading, image enhancement, result display and saving of enhanced images. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 It is the basic structure of the convolutional neural network.

[0030] Figure 2 It is the structure of the multi-scale feature enhancement module.

[0031] Figure 3 It is the U-shaped Transformer architecture.

[0032] Figure 4 It is the basic structure of SGFMT.

[0033] Figure 5 It is the details of the CMSFFT module.

[0034] Figure 6To enhance the effect contrast, (a): original image; (b): Retinex-based method; (c): Water-Net; (d): UGAN; (e): original method; (f): method of this paper; (g): GT.

[0035] Figure 7 To enhance the function display.

[0036] Figure 8 To show the index calculation function.

[0037] Figure 9 To show the image saving and exit function. DETAILED DESCRIPTION

[0038] In order to solve the problems existing in the current mainstream underwater image enhancement method, the application takes the network architecture based on Transformer as the basis, introduces the module based on convolutional neural network for optimization, proposes a new hybrid network for underwater image enhancement, and designs an underwater image enhancement system based on the hybrid network. Specifically, combining the advantages of CNN and Transformer, a feature enhancement module based on convolutional neural network is added to the underwater image enhancement based on Transformer, a new hybrid network for underwater image enhancement is proposed, and the model performance is evaluated. In the comparative experiment of the public data set, the proposed method shows better enhancement effect compared with other mainstream underwater image models. Finally, based on the network model, an underwater image enhancement system is designed and implemented, which can enhance the image and calculate the evaluation index.

[0039] Convolutional Neural Network (CNN) has the characteristics of local connection and weight sharing, and is very good at processing image related tasks. The basic structure of convolutional neural network is as shown in Figure 1 The input layer is mainly used for pre-processing of input data. The convolutional layer is composed of multiple convolutional kernels. They are used for element summation and multiplication, and the output result is a feature map. In this process, each convolutional kernel captures certain specific features, and using multiple convolutional kernels can analyze data in multiple dimensions and obtain a rich set of features. In the activation layer, in order to improve the learning and expression ability, the output result is processed by using the activation function for non-linear mapping. The pooling layer is used to reduce the size of the feature map, and its purpose is to reduce the amount of calculation. The commonly used pooling operations are maximum pooling and average pooling. The fully connected layer is located at the rear of the network structure, and is mainly used to integrate various local features extracted in front to realize the mapping from the extracted features to the final output.

[0040] An activation function is a non-linear mapping layer that enhances the non-linear representation ability of the network. It is similar to the way biological neurons process signals. An ideal activation function converts the input signal into a binary output after a certain threshold, but in practical applications, the continuity and differentiability of the forward propagation process and other complex processes require it. Sigmoid, Tanh, ReLU, and Softmax are commonly used activation functions.

[0041] Sigmiod function: The Sigmiod function is used a lot in early models. Its expression is: .

[0042] The output range of the Sigmoid function is 0 to 1, which is an ideal choice for processing binary problems. However, it has the problem of gradient disappearance. When the input value is too large, the derivative will approach 0, and when the error is back-propagated to the initial layer in a deep network, the gradient will be reduced to near 0, hindering the learning progress.

[0043] Tanh function: The Tanh function is a variant of the Sigmoid function, with outputs between -1 and 1, centered at 0. This helps data converge quickly during training. However, it may also face the problem of gradient disappearance when dealing with extreme values. The expression of the Tanh function is: .

[0044] Relu function: The ReLU function keeps its value unchanged when dealing with positive numbers, but outputs 0 when dealing with negative numbers. Using it in deep networks can effectively reduce the problem of gradient disappearance. However, some neurons may be permanently inactive, causing some information to be lost. The expression of the ReLU function is as follows: .

[0045] Softmax function: The Softmax function is commonly used for multi-class classification problems. Each element has a probability distribution ranging from 0 to 1, and the sum of all element values is 1. Softmax is very suitable for handling probability estimation problems. The expression is: .

[0046] After Alexnet was proposed, many scholars or use small convolution kernel, or increase multi-scale to study how to optimize its structure, but VGGNet network chooses a different way, through the increase of network depth, more simple and efficient to improve the network performance. The feature map size of VGGNet is gradually reduced, and the number of feature maps will increase. From the model diagram, the width and height of the solid block are constantly decreasing, but the thickness is slowly increasing. VGG16 is the typical structure of VGGNet, unlike the large convolution kernel in AlexNet, it uses multiple small convolution kernels in series, which makes it learn more complex patterns at a lower cost.

[0047] The most innovative place of Transformer is its unique self-attention mechanism, the correlation weight of input elements with all other elements in the sequence is calculated one by one and used in subsequent data processing, which makes it better at capturing long-distance dependencies. The special network mainly includes two parts of encoder and decoder, when the data is input into the Transformer network, the following processing steps will be mainly experienced. First, input encoding, the input data will be converted into vectors, each word is mapped to a high-dimensional vector space; the second step is position encoding, in order to enable the model to distinguish different positions of words in the sequence, position encoding needs to be added to the input vector; then, the core of Transformer, self-attention mechanism, the completed encoding sequence will be sent to multiple attention heads, each head pays attention to different information, in order to improve the diversity of attention, each head will calculate the importance of each position in the input sequence, and distribute these importance to other positions, these operations enable the model to pay more attention to the dependency relationship of different positions; then two fully connected layers: feedforward neural network and residual connection, the former consists of two linear layers and an activation function, used for transformation of attention and extraction of information, the latter adds the input and output, avoids gradient disappearance and helps information flow, after that, layer normalization will be applied to the output of each sublayer, its main purpose is to speed up the training process of the whole model; in the output layer, the probability distribution can be obtained after the Softmax function processing, the word with the maximum probability will be the output result.

[0048] The present application selects the existing constructed UIEB dataset [MFFN: an underwater sensing scene image enhancement method based on multiscale feature fusion network] for subsequent training. The dataset mainly contains 950 real underwater images, covering a variety of underwater scenes, and the image quality is high. The prior art uses 12 mature methods to process the original image, and then selects the best image with the help of volunteers, and finally obtains 890 high-quality reference images, and the remaining 60 are used as challenging image sets. As an important resource in the field of underwater image processing, the UIEB dataset is now widely used to test and compare various underwater image processing algorithms.

[0049] PSNR peak signal-to-noise ratio and SSIM structural similarity are two commonly used image quality evaluation indicators. PSNR is used to calculate the difference between the reference image and the generated image, usually in decibels (dB). The higher the value, the less the loss of image quality. For a given size of m * n reference image I and the image to be evaluated K , the mean square error MSE can be expressed as:

[0050] where i and j represent the pixel points of the corresponding coordinates in the image, and then the PSNR can be defined as:

[0051] where MSE is the mean square error, and the maximum possible pixel value of the image is represented by 255. If each pixel is represented by eight binary, it is 255. For color images, there are three calculation methods: first, calculate the PSNR value on the RGB three channels respectively, and then divide by three; second, calculate the MSE of the RGB three channels, and take the average; the third needs to convert the image to YCbCr format first, and then calculate the PSNR value of the Y channel component. The disadvantage of the PSNR index is that it completely analyzes the image as isolated pixel points, ignoring the internal relationship of the local content, and at the same time, the quality of the image is subjective to people to a certain extent, some images with high PSNR scores may not be better than those with low scores in people's eyes.

[0052] In view of the shortcomings of PSNR, SSIM is proposed. SSIM measures the image from three aspects of brightness, contrast and structure, and considers the structural information of the image, which is closer to the perception of the human visual system.

[0053]

[0054]

[0055]

[0056] where x, y represent two samples to be measured respectively denotes the mean of x, denotes the mean of y, is the variance of x, is the variance of y, denotes the covariance of x and y. , is a constant.L is the dynamic range of the pixel value of the picture, and k1 and k2 take default values 0.01 and 0.03.So, the SSIM can be represented as:

[0057] set to 1, then: , , .

[0058] The application adds a multi-scale feature extraction module based on a convolutional neural network in the existing U-shaped Transformer architecture, combines the advantages of convolutional neural networks and Transformers, proposes a new technical idea, and achieves good results in experiments.

[0059] The existing module based on the Transformer network is embedded in the encoder and the encoder interaction part of the Unet architecture, which realizes excellent capture ability of global information, but the local information capture ability is poor. Therefore, in order to ensure that the input features have both global information and local information, the application adds a multi-scale feature enhancement module (MSFE) based on CNN in the link of obtaining shallow features, so that the features input to the later layer can have a more bottom layer expression. After the original input x is convolved by 3*3, it will be input to the multi-scale feature enhancement module. In this module, a kernel selection is set to automatically select the size of the convolution kernel K1 and K2, and through convolution kernels of different sizes, feature maps corresponding to different size receptive fields are obtained. It can be represented as:

[0060]

[0061] where * represents the convolution operation, and W and b are the corresponding convolution weights and bias.

[0062] ​Two feature maps first pass through 1*1 convolution to reduce the number of channels by half to get r1 and r2, which are merged in the channel dimension, and after the processing of the pooling layer, the obtained result is input to the convolution and Sigmoid function activation to obtain the attention map s, which is expressed as:

[0063]

[0064] wherein, Sigmoid activation function, m avg and m max respectively, the average attention and the maximum attention.

[0065] The generated attention map s contains two channels for weighting the corresponding feature map, which is multiplied with the corresponding feature map after being added, and then the feature map is expanded to the original channel number. It can be recorded as:

[0066]

[0067]

[0068] wherein, w1 and w2 are weighted feature maps, a expanded for expanding the feature map to the original channel number, element-wise multiplication, W2, b2 is the corresponding convolution weight and bias.

[0069] Finally, the obtained matrix is element-wise multiplied with the original input feature, and the output result y is implemented multi-scale feature enhancement. It can be expressed as: , wherein, element-wise multiplication, W3 and b3 are the corresponding convolution weight and bias.

[0070] The module structure is shown in Figure 2 The core of this module is to capture information in different spatial ranges using different scales of receptive fields, and to adaptively emphasize or weaken some features through learned weights, thereby improving the information processing ability of the model.

[0071] The existing U-shaped Transformer module is based on the existing research [Image-to-image translation with conditional adversarial networks[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition], and adds a multi-scale feature fusion module (CMSFFT) and a spatial global feature modeling module (SGFMT) in the generator and a discriminator. The overall structure is as follows Figure 3 The generator includes two steps of encoding and decoding. The encoding process will be down-sampled three times, and the obtained feature maps will be input into the corresponding convolution block. The result after convolution will be input into the two specially designed modules of CMSFFT and SGFMT; in the decoding process, the output of SGFMT is mapped by feature remapping and accepted by the first convolution block, and the four outputs of CMSFFT are sent to four different scale convolution blocks.

[0072] The input of the discriminator convolution block is: the output of the last layer, the feature map output of the decoding part, and in addition, the feature map generated by 1*1 convolution after down-sampling of the reference image is also input. The specific structure of SGFMT and CMSFFT will be described below.

[0073] SGFMT: SGFMT is used to replace the original bottleneck layer of the generator, which can help the network to model the global information, and at the same time, it can strengthen the network's attention to the seriously degraded part, Figure 4 is its basic structure.

[0074] Assuming that the size of the input feature map is , it needs to be stretched into a feature sequence using linear projection first. In order to preserve the position information of each region, a position embedding operation needs to be performed, which can be represented as:

[0075] where, is the linear projection operation, and PE is the position information embedding.

[0076] Then, the feature sequence is input into the transformer module, which has 4 standard transformer layers, each containing a multi-head attention module and a feedforward network. The output of the first layer in the transformer module can be calculated in the following way:

[0077]

[0078] where LN denotes layer normalization, and denotes the output sequence of the layer. The feature sequence output by the last Transformer module is mapped back to the feature map of size

[0079] CMSFFT: The CMSFFT module is used to strengthen the network's attention to color channels with more severe degradation, which consists of three parts: multi-scale feature encoding, channel multi-head attention, and feedforward network. Figure 5 The basic structure is shown in Fig. 4.

[0080] Multi-scale feature encoding: The input of CMSFFT is a feature map with different scales , which is linearly projected by a convolution kernel with size and stride . In this paper, P is set to 32, and four feature sequences will be obtained, where . The convolution kernel divides the feature map into the same number of blocks, but the number of channels remains unchanged. According to the following formula, the corresponding query vectors , and can be calculated.

[0081]

[0082]

[0083]

[0084] where , , are the learnable weight matrices; S is generated by concatenating in the channel dimension, and in this paper, C1, C2, C3, and C4 are set to 64, 128, 256, and 512, respectively.

[0085] Channel multi-head attention: This module has four inputs, which are , and . The channel-wise attention can be obtained by the following formula:

[0086] where IN denotes instance normalization to guide the network to pay more attention to channels with more severe image quality degradation. In addition, IN is also used for similar images to help the gradient flow diffuse smoothly.

[0087] The output of the first CMHA layer can be represented as:

[0088] where N represents the number of attention heads, which is set to 4 in this paper.

[0089] The output of the feed-forward network can be represented as:

[0090] where, MLP represents a multi-layer perceptron, and within the MLP, the operation in equation 3.7 needs to be repeated multiple times to build a layer of the Transformer network, and the value of is 4 in this paper.

[0091] Finally, in order to recombine the four different output features into four feature maps as input to the convolutional modules of the decoder part of the generator, they need to be feature remapped.

[0092] The present application uses the same loss function as the existing method, in order to make use of the wider color gamut representation range of LAB and LCH color spaces and more accurate description of color saturation and brightness, the loss function combines the three color spaces of RGB, LAB and LCH. The image in RGB space is converted to LAB and LCH space and then read:

[0093]

[0094] where x, y and G(x) represent the original image, the reference image and the image output by the generator respectively.

[0095] The loss function of the LAB color space is:

[0096] The loss function of the LCH color space is:

[0097] where Q represents the quantization operator.

[0098] The four loss functions of the generator: the L2 loss LOSS RGB in the RGB color space, the perceptual loss LOSS per , LOSS LAB and LOSS LCH ​In addition, a standard GAN loss function is introduced to minimize the loss between the generated image and the reference image, written as:

[0099] The final loss function is expressed as:

[0100] where, , , , μ is a hyperparameter, respectively set to 0.001, 1, 0.1, 100.

[0101] Example one The present application uses the existing UIEB dataset [MFFN: an underwater sensing scene image enhancement method based on multiscale feature fusion network] for training, and tests on the existing Test-L400 dataset [U-shape transformer for underwater image enhancement]. The images are all cropped to a fixed size of 256*256, and the pixel values are normalized to zero to one. Training is performed using python and pytorch framework on Ubuntu 20.04 and RTX4090. A total of 200 epochs are trained, and every 50 epochs, the learning rate is reduced by 20%.

[0102] Comparative experiment and ablation experiment. In the comparative experiment, in addition to comparing with the original method, three existing methods are selected: the method based on retinex [7], U-Net

[12] and UGAN

[15] are compared, and the PSNR and SSIM are calculated on the RGB channel to evaluate the final results.

[0103] Figure 6To enhance the effect contrast, (a): original image; (b): Retinex-based method; (c): Water-Net; (d): UGAN; (e): original method; (f): the method of the present application; (g): GT. From the contrast image, the performance of various methods can be intuitively felt to some extent. In processing the first group of greenish images, methods (a), (b) and (c) perform similarly, the image color is yellowish, and the original method (e) and the method (f) of the present application both correct the color of the image better. The method of the present application has the most clear details of the diver's backpack. In the second group of low-illumination images, the three reproduced contrast methods still perform similarly, the image is dark, and the original method and the method of the present application obviously improve the brightness and contrast, and the color of the stone after processing by the method of the present application is closest to the reference image. In the third group of images, methods (a), (b) and (c) have clearer levels after processing, but the color restoration degree needs to be improved, the original method restores the color better, but the clarity of the middle fish group is the lowest. The color correction effect of the method of the present application is the best, and the details are also more abundant.

[0104] Subjective judgment is relatively one-sided, and only by calculating the corresponding image evaluation index can a more objective and accurate result be obtained. In Table 1, the higher the PSNR value, the closer the enhanced result is to the reference image, and the higher the SSIM score means the better the enhancement result. It can be seen that the method proposed in the present application not only performs well in subjective vision, but also has good results in the comparison of objective indicators.

[0105] Table 1 Comparison results (* marked are all reproduced results)

[0106] It can be seen that the complete network performs best, and the two modules SGFMT and CMSFFT in the original method are used to strengthen the attention to the severely attenuated channel, and the two modules complement each other, and the indicators decrease more after removal. The addition of the MSFE module plays a certain role in improving the performance of the network.

[0107] The present application first introduces the mixed network architecture used in detail, mainly explains the principles of the three modules MSFE, SGFMT and CMSFFT and introduces the loss function, and then compares the method proposed in the present application with a plurality of existing methods, and proves the effectiveness through ablation experiments.

[0108] Example two The underwater image enhancement method based on deep learning technology is designed in the foregoing text, and on this basis, an underwater image enhancement system is designed to realize the color correction and detail enhancement of underwater images. Users can upload images of different formats to be processed and save the processed results. The following will introduce the underwater image enhancement system from three aspects of environment building, interface writing and function display.

[0109] 1 Environment building The system mainly uses Pyqt5 to design the interface, and calls the OpenCV library when processing images.

[0110] PyQt5 is widely used in the development of graphical user interface applications. PyQt5 can help developers quickly create modern, beautiful and powerful desktop applications through Python language, with cross-platform compatibility, rich controls, event-driven, multithreading and many other advantages. It can be used in various Python integrated development environments such as PyCharm, VSCode, etc. PyQt5 is easy to install, and can be installed through the package management tool pip of Python.

[0111] OpenCV is a computer vision library. It is mainly used for object detection, structure analysis, motion and other fields, and supports multiple programming languages. The Python version of OpenCV is an interface for calling OpenCV, usually called CV2. From simple image processing to complex machine vision applications, the OpenCV library can handle it. If you need to quickly integrate modern vision technology into your application, Python's OpenCV is an indispensable tool.

[0112] 2 Interface writing The system mainly provides the functions of uploading, enhancing, displaying and saving images, and calculating evaluation indexes. The specific writing process is as follows: First, import the necessary libraries and modules, including various components of PyQt5 and common libraries. Then define the main window class Mainwindow, which inherits from PyQt5 component QtabWidget. The initUI() method is used to initialize the interface, including creating labels to display the original and enhanced pictures, creating buttons to upload pictures, execute image enhancement, store pictures and calculate indexes. Each button is bound to the corresponding event handling function. When executing image enhancement, the myenhancement() function defined externally is called to process the image, and the interface is updated to display the enhanced image.

[0113] 3 Function display This section will specifically show the functions.

[0114] The overall interface is as follows: Figure 7As shown, after uploading the picture, click on image enhancement, the pre-selected model will be called to process and output the image, and the result saving function is also provided, users can choose the save path by themselves. Before image enhancement, the original image must be uploaded, otherwise a dialog box will pop up to prompt the user to upload the original image. If you need to upload a new image, the user can upload again to replace the original image.

[0115] In addition to enhancement, an index calculation function is also provided, see Figure 8 . Before performing index calculation, the reference image must be uploaded first, otherwise a corresponding dialog box will pop up to prompt the user to perform the necessary pre-operation first. If you need to upload a new reference image, the user can upload again to replace the original image.

[0116] In addition, users can also save the enhanced image to a specified path, see Figure 9 .

[0117] Although underwater images are one of the most intuitive ways to present underwater information, due to the special underwater environment and the limitations of imaging equipment, the actual obtained underwater images often have color distortion and blurring problems, and the efficiency of researchers in obtaining effective information from them is greatly reduced. This invention focuses on how to enhance underwater images, introduces the commonly used methods in this field and the development background of deep learning, and also explains the underwater image dataset and evaluation indicators used in this invention. Then a hybrid network for underwater image enhancement is proposed, which gets better results after optimization testing, and a system for underwater image enhancement is designed based on the trained model. This invention combines the advantages of convolutional neural networks and Transformers, and designs a hybrid network for underwater image enhancement. By introducing CNN-based modules into the Transformer-based architecture, the quality of image enhancement is improved, and the model performance is evaluated; based on the proposed method, a simple system for underwater image enhancement is designed, users can easily process and save images.

Claims

1. A method for underwater image enhancement based on convolutional neural network, characterized in that, The method comprises the following steps: based on a U-shaped Transformer architecture, a multi-scale feature extraction module based on a convolutional neural network is added to realize underwater image enhancement.

2. The method of claim 1, wherein the method further comprises: A multi-scale feature enhancement module based on CNN is added in the process of obtaining shallow features, so that the features input to the later layers can have a more bottom-level expression. 3.The method of claim 1, wherein, Image feature data is obtained, and then based on a U-shaped Transformer architecture, a multi-scale feature extraction module based on a convolutional neural network is added to realize underwater image enhancement.

4. The method of claim 3, wherein the method further comprises: The original input is input to the multi-scale feature enhancement module after convolution; In the multi-scale feature enhancement module, a kernel selection is set to automatically select the size of the convolution kernel K1 and K2, and through convolution kernels of different sizes, feature maps corresponding to different size receptive fields are obtained, represented as: ， ， Where * represents convolution operation, W and b are the corresponding convolution weights and bias.

5. The method of claim 4, wherein the method further comprises: Two feature maps are first processed by 1*1 convolution to reduce the number of channels by half to obtain r1 and r2, which are combined in the channel dimension, and after the processing of the pooling layer, the obtained result is input to the convolution and Sigmoid function activation to obtain the attention map s, represented as: ， ， where, denotes the Sigmoid activation function, m avg and m max are the average attention and maximum attention, respectively.

6. The method of claim 5, wherein the method further comprises: The generated attention map s contains two channels for weighting the corresponding feature maps, which are multiplied with the corresponding feature maps and then added, and then the feature map is expanded to the original channel number; denoted as: ， ， ， where w1 and w2 are the weighted feature maps, a expanded for expanding the feature map to the original channel number, denotes element-wise multiplication, W2, b2 are the corresponding convolution weights and bias.

7. The method of claim 6, wherein the method further comprises: The obtained matrix is multiplied element by element with the original input feature to realize multi-scale feature enhancement on the output result y; represented as: ， wherein, denotes element-wise multiplication, and W3and b3are the corresponding convolution weights and bias.

8. The application of the underwater image enhancement method based on a convolutional neural network according to any one of claims 1 to 7 in image enhancement.

9. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the underwater image enhancement method based on a convolutional neural network according to any one of claims 1 to 7.

10. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor executes the computer program to realize the underwater image enhancement method based on a convolutional neural network according to any one of claims 1 to 7.

Images