An unsupervised underwater image enhancement method and related devices

Abstract

The present invention discloses an unsupervised underwater image enhancement method and related devices. The method includes: obtaining underwater image data from a public dataset, where the underwater image data includes a training dataset and a test dataset, and performing normalization processing and scaling processing on the underwater image data; constructing a neural network model, and using the processed training dataset to train the neural network model, where the neural network model includes a parameter estimation network and an image enhancement network; inputting the processed test dataset into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset. The present invention restores a distorted underwater image to an image with balanced color, rich details, and good contrast through image enhancement, and realizes the enhancement of image data in different water body environments through an unsupervised method, with better versatility.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an unsupervised underwater image enhancement method, system, terminal and computer-readable storage medium. Background Art

[0002] With the development of marine information technology, the application of underwater target detection technology has become increasingly widespread, covering fields such as the laying of submarine optical cables, the establishment and maintenance of underwater oil platforms, the salvage of sunken ships at the bottom of the sea, and the research of marine ecosystems. Underwater target detection places high demands on imaging and post-processing capabilities. How to correctly identify the objects and their positions in the image is a common problem in the fields of underwater robots and machine learning. In the process of traditional deep learning for underwater image enhancement, supervised training is usually required with paired or unpaired clear-degraded images, or image restoration needs to be carried out with the help of industrial cameras that can provide depth information.

[0003] Limited by the special underwater imaging environment, underwater images often face serious problems such as severe noise interference, texture blur, and color distortion, posing a serious challenge to underwater target detection tasks.

[0004] Most of the existing underwater image enhancement methods require paired (two clear images taken at the same shooting position and scene) or unpaired (image pairs at different positions) underwater distorted (blurred images) and clear images for supervised training. However, the marine environment is complex and intricate, and such supervised training data is often difficult to obtain.

[0005] Although the existing technology can perform image enhancement through classical algorithms or deep neural networks, which takes into account clarity and color accuracy, from the perspective of the data used for training, most of the existing technologies use clear ground-truth images as training targets, which can be paired data pairs at the same shooting location or an unpaired set of clear-degraded images. Such data is often difficult to obtain when the water environment changes; in terms of the enhancement algorithm, when not using clear images as the target, classical image processing algorithms are used to process the images. One drawback of such methods is poor generality, and different water environments should correspond to different algorithm parameters, with low credibility.

[0006] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0007] The main purpose of the present invention is to provide an unsupervised underwater image enhancement method, system, terminal and computer-readable storage medium, aiming to solve the problems of difficult acquisition of underwater supervised training data and poor generality in the existing technology.

[0008] To achieve the above object, the present invention provides an unsupervised underwater image enhancement method, and the unsupervised underwater image enhancement method includes the following steps:

[0009] Obtain underwater image data from a public dataset, where the underwater image data includes a training dataset and a test dataset, and perform normalization processing and scaling processing on the underwater image data;

[0010] Construct a neural network model, and use the processed training dataset to train the neural network model. The neural network model includes a parameter estimation network and an image enhancement network;

[0011] Input the processed test dataset into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset.

[0012] Optionally, for the unsupervised underwater image enhancement method, where obtaining underwater image data from a public dataset, where the underwater image data includes a training dataset and a test dataset, and performing normalization processing and scaling processing on the underwater image data specifically includes:

[0013] Obtain the underwater image data from two preset public datasets, use the first preset number of images in the underwater image data as the training dataset, and use the second preset number of images in the underwater image data as the test dataset;

[0014] Perform normalization processing on the training dataset and the test dataset, and use the min-max normalization method to scale the pixel values of the training dataset and the test dataset from 0 - 255 to the 0 - 1 interval.

[0015] Optionally, for the unsupervised underwater image enhancement method, where the parameter estimation network includes 5 convolutional layers, 2 fully connected layers and an activation function, and the activation function is the LeakyReLU function;

[0016] The image enhancement network includes a CLAHE module, 4 convolutional layers and 3 deconvolutional layers.

[0017] Optionally, for the unsupervised underwater image enhancement method, where constructing a neural network model and using the processed training dataset to train the neural network model specifically includes:

[0018] Input the scaled training dataset into the parameter estimation network. The scaled training dataset passes through 5 convolutional layers and 2 fully connected layers in sequence and then outputs 9 real values, and use the 9 real values as the parameters of the image enhancement network;

[0019] Input the training data set into the image enhancement network. Use the CLAHE module to generate an image after contrast-limited adaptive histogram equalization. Then, through 4 convolutional layers and 3 deconvolutional layers, slice the generated 12-channel image to produce 4 three-channel images. Iteratively perform a quadratic curve transformation on the original training data set, and then perform USM sharpening to obtain the enhanced image, so as to complete the training of the neural network model.

[0020] Optionally, in the unsupervised underwater image enhancement method, the formula for the quadratic curve transformation is as follows:

[0021] G(x) = G4(x);

[0022]

[0023] where x represents the original training data set; G(x) is the image after the quadratic curve transformation, G i (x) is the image after the i-th quadratic curve processing of the original training data set iteration; G i-1 (x) is the image after the (i - 1)-th quadratic curve processing of the original training data set iteration; G4(x) is the image after the 4th quadratic curve processing of the original training data set iteration; r i represents the i-th sliced image; a i and b i are the weights of the quadratic curve, given by the parameter estimation network.

[0024] Optionally, in the unsupervised underwater image enhancement method, the formula for the USM sharpening is as follows:

[0025] U(x) = α * (x - GaussBlur(x)) + x, α ∈ [0, 5];

[0026] where U(x) represents performing USM sharpening on the original training data set x; α is the sharpening weight, given by the parameter estimation network; GaussBlur(x) represents performing Gaussian blur on the original training data set x;

[0027] The input x of the USM sharpening function is equal to the output G(x) of the quadratic curve transformation.

[0028] Optionally, in the unsupervised underwater image enhancement method, the total loss function of the neural network model consists of a CLAHE image contrast loss, a color constancy loss, an exposure control loss, and an illumination smoothness loss;

[0029] The total loss function is obtained by weighted summation of the CLAHE image contrast loss, the color constancy loss, the exposure control loss, and the illumination smoothness loss:

[0030] L = 20 * L CLAHE + 5 * L color + 10 * L exp + 200 * L tv ;

[0031] Wherein, L represents the total loss function; L CLAHE represents the CLAHE image contrast loss; L exp represents the exposure control loss; L tv represents the illumination smoothness loss;

[0032] Wherein, the CLAHE image contrast loss is:

[0033] L CLAHE = L1 smooth (Norm(CLAHE(x)) - Noem(G(x)));

[0034] Wherein, L1 smooth is the Smooth L1 Loss function; CLAHE(x) is the limited contrast adaptive histogram equalization processing on the original training dataset x; Norm represents the normalization processing;

[0035] Wherein, the color constancy loss is:

[0036]

[0037] Wherein, (p, q) represents traversing all pairwise combinations in the three color channels; J p represents the average brightness of the p color channel in the enhanced image; J q represents the average brightness of the q color channel in the enhanced image;

[0038] Wherein, the exposure control loss is:

[0039]

[0040] Wherein, Y k represents the brightness of the kth large pixel; E is a constant; M is the total number of large pixels;

[0041] Wherein, the illumination smoothness loss is:

[0042]

[0043] Wherein, N represents the number of iterations; and respectively represent the horizontal and vertical gradient operators; ξ represents the RGB color space, including three channels; Denotes the slice corresponding to the c color channel in the n-th conic processing iteration.

[0044] In addition, to achieve the above object, the present invention also provides an unsupervised underwater image enhancement system, wherein the unsupervised underwater image enhancement system includes:

[0045] A data acquisition and preprocessing module, configured to acquire underwater image data from a public dataset, the underwater image data including a training dataset and a test dataset, and perform normalization processing and scaling processing on the underwater image data;

[0046] A network construction and training module, configured to construct a neural network model, and use the processed training dataset to train the neural network model, the neural network model including a parameter estimation network and an image enhancement network;

[0047] An image enhancement processing module, configured to input the processed test dataset into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset.

[0048] In addition, to achieve the above object, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an unsupervised underwater image enhancement program stored on the memory and executable on the processor, and when the unsupervised underwater image enhancement program is executed by the processor, the steps of the above-mentioned unsupervised underwater image enhancement method are implemented.

[0049] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an unsupervised underwater image enhancement program, and when the unsupervised underwater image enhancement program is executed by a processor, the steps of the above-mentioned unsupervised underwater image enhancement method are implemented.

[0050] In the present invention, underwater image data is acquired from a public dataset, the underwater image data including a training dataset and a test dataset, and the underwater image data is subjected to normalization processing and scaling processing; a neural network model is constructed, and the processed training dataset is used to train the neural network model, the neural network model including a parameter estimation network and an image enhancement network; the processed test dataset is input into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset. The present invention restores a distorted underwater image to an image with balanced color, rich details, and good contrast through image enhancement, and realizes the enhancement of image data in different water body environments through an unsupervised method, with better versatility. Description of the Drawings

[0051] Figure 1 is a flowchart of a preferred embodiment of the unsupervised underwater image enhancement method of the present invention;

[0052] Figure 2 is a schematic diagram of the composition of the parameter estimation network and the image enhancement network and the principle of the image processing process in a preferred embodiment of the unsupervised underwater image enhancement method of the present invention;

[0053] Figure 3 is a schematic diagram of the principle of a preferred embodiment of the unsupervised underwater image enhancement system of the present invention;

[0054] Figure 4 is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. Specific Embodiments

[0055] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the present invention will be further described in detail below with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] The image enhancement method proposed by the present invention combines classical image processing algorithms and deep neural networks, and can enhance image data in different water body environments through an unsupervised method, with better versatility.

[0057] The unsupervised underwater image enhancement method described in the preferred embodiment of the present invention, as Figure 1 shown, the unsupervised underwater image enhancement method includes the following steps:

[0058] Step S10: Obtain underwater image data from a public dataset. The underwater image data includes a training dataset and a test dataset, and perform normalization processing and scaling processing on the underwater image data.

[0059] Specifically, obtain the underwater image data from two preset public datasets. For example, the two public datasets are the underwater dataset of the 2020 National Underwater Robot (Zhanjiang) Competition and the European Open Underwater Dataset (The Brackish Dataset). Among them, the underwater dataset of the 2020 National Underwater Robot (Zhanjiang) Competition includes 4 types of marine organisms, a total of 5543 color three-channel pictures, with resolutions of 1920*1080, 586*480, etc.; the European Open Underwater Dataset (The Brackish Dataset) includes 6 types of marine organisms, a total of 14674 color three-channel pictures, with a resolution of 960*540.

[0060] Then, use the images with the first preset quantity in the underwater image data (for example, the first preset quantity is 4434 and 11739 images in the underwater datasets of the 2020 National Underwater Robot (Zhanjiang) Competition and the European Open Underwater Dataset) as the training dataset, and use the images with the second preset quantity in the underwater image data (for example, the second preset quantity is 555 and 1468 images in the underwater datasets of the 2020 National Underwater Robot (Zhanjiang) Competition and the European Open Underwater Dataset) as the test dataset.

[0061] Then, perform normalization processing on the training dataset and the test dataset. The function of normalization processing is to transform the dimensional expression into a dimensionless expression through transformation, becoming a scalar, and it can also solve the gradient problem and accelerate the convergence of the model; and use the min-max normalization method (scale the data proportionally so that it falls into a small specific interval) to scale the pixel values of the training dataset and the test dataset from 0 - 255 to the 0 - 1 interval. Converting the data to decimals in the (0, 1) interval is mainly for the convenience of data processing. Mapping the data into the range of 0 to 1 for processing is more convenient and fast.

[0062] Step S20: Construct a neural network model, and use the processed training dataset to train the neural network model. The neural network model includes a parameter estimation network and an image enhancement network.

[0063] Specifically, as Figure 2 shown, the parameter estimation network and the image enhancement network are respectively responsible for estimating certain parameters in the image enhancement network and generating enhanced images; the parameter estimation network includes 5 convolutional layers, 2 fully connected layers, and an activation function, and the activation function is the LeakyReLU function; the image enhancement network includes a CLAHE module, 4 convolutional layers, and 3 deconvolutional layers.

[0064] Input the pre - processed and scaled training dataset into the parameter estimation network (for example, the input is the original image with a size of 256 * 256 after scaling, and the size varies according to the dataset and is equal to the original resolution of the dataset images). After passing through 5 convolutional layers and 2 fully connected layers in sequence, the scaled training dataset outputs 9 real values, and use these 9 real values as the parameters of the image enhancement network (the 9 real values are 9 parameters, corresponding to a and b in the following quadratic curve transformation formula, 4 pairs, a total of 8, and α in the USM sharpening formula, a total of 1).

[0065] Input the training data set into the image enhancement network. Through the CLAHE module, an image after contrast-limited adaptive histogram equalization is generated. Let the image enhancement network generate an image after contrast-limited adaptive histogram equalization and input it into the image enhancement network. At the same time, in the subsequent histogram loss formula, the loss will be calculated based on this image, so that the image histogram generated by the image enhancement network is close to or even exceeds that of the traditional CLAHE method. Then, through 4 convolutional layers and 3 deconvolutional layers, slice processing is performed on the generated 12-channel image (the purpose of slice processing is to perform a quadratic curve transformation iteratively between the original image and the generated 4 image slices), generating 4 three-channel images, which are iteratively subjected to a quadratic curve transformation with the original training data set, and then sharpened by USM to obtain the enhanced image, so as to complete the training of the neural network model.

[0066] Among them, the formula for the quadratic curve transformation is as follows:

[0067] G(x) = G4(x);

[0068]

[0069] Among them, x represents the original training data set; G(x) is the image after the quadratic curve transformation, G i (x) is the image after the i-th quadratic curve processing of the original training data set iteration; G i-1 (x) is the image after the (i - 1)-th quadratic curve processing of the original training data set iteration; G4(x) is the image after the 4th quadratic curve processing of the original training data set iteration; r i represents the i-th slice image; a i and b i are the weights of the quadratic curve, given by the parameter estimation network.

[0070] Among them, the formula for the USM sharpening is as follows:

[0071] U(x) = α * (x - GaussBlur(x)) + x, α ∈ [0, 5];

[0072] Among them, U(x) represents the USM sharpening process for the original training data set x; α is the sharpening weight, given by the parameter estimation network; GaussBlur(x) represents the Gaussian blur process for the original training data set x; in the entire image enhancement network, the input x of the USM sharpening function is equal to the output G(x) of the quadratic curve transformation.

[0073] Furthermore, the total loss function (objective function) of the neural network model in the present invention is composed of CLAHE image contrast loss, color constancy loss, exposure control loss, and illumination smoothness loss. After the forward propagation of the deep neural network is completed, the total loss function needs to be calculated, and according to this function, the gradients of each layer are calculated in reverse to achieve the purpose of parameter update, so that the network can be trained.

[0074] The total loss function is obtained by weighted summation of CLAHE image contrast loss, color constancy loss, exposure control loss, and illumination smoothness loss:

[0075] L = 20 * L CLAHE + 5 * L color + 10 * L exp + 200 * L tv ;

[0076] where L represents the total loss function; L CLAHE represents the CLAHE image contrast loss; L exp represents the exposure control loss; L tv represents the illumination smoothness loss.

[0077] Among them, the CLAHE image contrast loss is:

[0078] L CLAHE = L1 smooth (Norm(CLAHE(x)) - Norm(G(x)));

[0079] where L1 smooth is the Smooth L1 Loss function; CLAHE(x) is the limited contrast adaptive histogram equalization processing of the original training dataset x; Norm represents the normalization processing.

[0080] Among them, the color constancy loss is:

[0081]

[0082] where (p, q) traverses all pairwise combinations in the three color channels; J p represents the average brightness of the p color channel in the enhanced image; J q represents the average brightness of the q color channel in the enhanced image.

[0083] Among them, the exposure control loss is:

[0084]

[0085] where Y kIt represents the brightness of the k-th largest pixel in the enhanced image; E is a constant, taken as 0.6 here; M is the total number of large pixels.

[0086] Among them, the illumination smoothness loss is:

[0087]

[0088] Among them, N represents the number of iterations; and respectively represent the horizontal and vertical gradient operators, which here are the differences in values between the pixel and the adjacent pixels on the left and above; ξ represents the RGB color space, which contains three channels; represents the slice corresponding to the c color channel in the n-th iteration of quadratic curve processing.

[0089] In the training stage of the neural network model, the training dataset is converted into the MS COCO format supported by the model through a script. The number of training epochs is set to 50, the number of training batches is 3, the device CPU (Central Processing Unit) is Intel Core i7 12700, the GPU (Graphics Processing Unit) is NVIDIA RTX 3090, and the operating system is Windows 11 22H2. After the neural network model is trained, the program automatically shuts down, generates an enhanced image example and saves it locally.

[0090] Step S30: Input the processed test dataset into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset.

[0091] Specifically, after the neural network model is trained, load the trained neural network model, and input the obtained preprocessed test dataset into the trained neural network model. Then, the trained neural network model (the parameter estimation network and the image enhancement network) can directly output an underwater enhanced image according to the test dataset. The underwater enhanced image has better color accuracy, rich details, and good contrast, thus providing an image with rich features for downstream tasks such as object detection and instance segmentation, and further expanding the application.

[0092] Beneficial effects:

[0093] (1) The present invention proposes an unsupervised underwater image enhancement method. First, multiple underwater images are collected as datasets (training dataset and test dataset), and the network is trained using a neural network and classical image processing algorithms to obtain an enhanced underwater image with better color accuracy, rich details, and good contrast.

[0094] (2) The present invention is not applicable to paired or unpaired supervised data and can complete the image enhancement work only relying on the objective function, greatly reducing the requirements for the data set.

[0095] (3) The present invention introduces a parameter estimation network to estimate the weights of quadratic curves and the parameters of classical sharpening algorithms, which has differentiable properties and can participate in the training of the overall network to adapt to image enhancement tasks in different environments.

[0096] (4) The present invention optimizes the imaging quality. By setting the objective function with different parameters, the enhancement effect can be further improved without using supervised data, which is more in line with the human eye's perception.

[0097] Compared with the existing underwater image enhancement methods, the present invention does not require paired or unpaired clear-degraded image data and can complete the enhancement task only relying on the original image, getting rid of the limitation of the data set to a certain extent and having a wider application range. The model also uses a parameter estimation network and combines classical image processing algorithms, which can adapt to different water environments and the image restoration effect is more accurate in some scenarios. The model is lightweight compared to complex underwater image restoration networks, can save computing resources and improve the computing speed. The underwater image enhancement method adopted by the present invention has been well verified on public data. The generated enhanced images conform to the human eye's perception and do not lose details, proving that the method is feasible.

[0098] In addition, the data acquisition method adopted in the present invention can use other vision cameras or download other public data sets instead; the devices used to train the model can also be replaced by other operating systems, processors, graphics processors, etc.

[0099] Furthermore, as Figure 3 shown, based on the above unsupervised underwater image enhancement method, the present invention also correspondingly provides an unsupervised underwater image enhancement system, wherein the unsupervised underwater image enhancement system includes:

[0100] A data acquisition and preprocessing module 51, configured to acquire underwater image data in a public data set, where the underwater image data includes a training data set and a test data set, and perform normalization processing and scaling processing on the underwater image data;

[0101] A network construction and training module 52, configured to construct a neural network model and use the processed training data set to train the neural network model, where the neural network model includes a parameter estimation network and an image enhancement network;

[0102] An image enhancement processing module 53, configured to input the processed test data set into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test data set.

[0103] Further, as Figure 4 shown, based on the above-mentioned unsupervised underwater image enhancement method and system, the present invention also correspondingly provides a terminal, which includes a processor 10, a memory 20, and a display 30. Figure 4 Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0104] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as the hard disk or memory of the terminal. In some other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk equipped on the terminal, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 20 may also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data, such as program codes for installing the terminal. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, an unsupervised underwater image enhancement program 40 is stored on the memory 20, and the unsupervised underwater image enhancement program 40 can be executed by the processor 10, so as to implement the unsupervised underwater image enhancement method in this application.

[0105] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chips, and is used to run the program codes stored in the memory 20 or process data, such as executing the unsupervised underwater image enhancement method, etc.

[0106] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other through a system bus.

[0107] In one embodiment, when the processor 10 executes the unsupervised underwater image enhancement program 40 in the memory 20, the following steps are implemented:

[0108] Obtain underwater image data from the public dataset. The underwater image data includes a training dataset and a test dataset, and perform normalization processing and scaling processing on the underwater image data;

[0109] Construct a neural network model, and use the processed training dataset to train the neural network model. The neural network model includes a parameter estimation network and an image enhancement network;

[0110] Input the processed test dataset into the trained neural network model. The parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset.

[0111] Among them, the obtaining of underwater image data from the public dataset, where the underwater image data includes a training dataset and a test dataset, and performing normalization processing and scaling processing on the underwater image data specifically includes:

[0112] Obtain the underwater image data from two preset public datasets. Use the first preset number of images in the underwater image data as the training dataset, and use the second preset number of images in the underwater image data as the test dataset;

[0113] Perform normalization processing on the training dataset and the test dataset, and use the min-max normalization method to scale the pixel values of the training dataset and the test dataset from 0 - 255 to the 0 - 1 interval.

[0114] Among them, the parameter estimation network includes 5 convolutional layers, 2 fully connected layers and an activation function. The activation function is the LeakyReLU function;

[0115] The image enhancement network includes a CLAHE module, 4 convolutional layers and 3 deconvolutional layers.

[0116] Among them, the constructing of the neural network model and using the processed training dataset to train the neural network model specifically includes:

[0117] Input the scaled training dataset into the parameter estimation network. The scaled training dataset passes through 5 convolutional layers and 2 fully connected layers in sequence and then outputs 9 real values, and use the 9 real values as the parameters of the image enhancement network;

[0118] Input the training data set into the image enhancement network. Use the CLAHE module to generate an image after limited contrast adaptive histogram equalization, and then pass it through 4 convolutional layers and 3 deconvolutional layers. Slice the generated 12-channel image to produce 4 three-channel images, perform a quadratic curve transformation iteratively with the original training data set, and then perform USM sharpening to obtain the enhanced image, so as to complete the training of the neural network model.

[0119] Among them, the formula for the quadratic curve transformation is as follows:

[0120] G(x) = G4(x);

[0121]

[0122] Among them, x represents the original training data set; G(x) is the image after the quadratic curve transformation, and G i (x) is the image after the i-th quadratic curve processing of the original training data set iteration; G i-1 (x) is the image after the (i - 1)-th quadratic curve processing of the original training data set iteration; G4(x) is the image after the 4th quadratic curve processing of the original training data set iteration; r i represents the i-th sliced image; a i and b i are the weights of the quadratic curve, given by the parameter estimation network.

[0123] Among them, the formula for the USM sharpening is as follows:

[0124] U(x) = α * (x - GaussBlur(x)) + x, α ∈ [0, 5];

[0125] Among them, U(x) represents the USM sharpening process on the original training data set x; α is the sharpening weight, given by the parameter estimation network; GaussBlur(x) represents the Gaussian blur process on the original training data set x;

[0126] The input x of the USM sharpening function is equal to the output G(x) of the quadratic curve transformation.

[0127] Among them, the total loss function of the neural network model consists of CLAHE image contrast loss, color constancy loss, exposure control loss, and illumination smoothness loss;

[0128] The total loss function is obtained by weighted summation of the CLAHE image contrast loss, color constancy loss, exposure control loss, and illumination smoothness loss:

[0129] L = 20 * L CLAHE + 5 * L color + 10 * Lexp +200*L tv ;

[0130] Among them, L represents the total loss function; L CLAHE represents the CLAHE image contrast loss; L exp represents the exposure control loss; L tv represents the illumination smoothness loss;

[0131] Among them, the CLAHE image contrast loss is:

[0132] L CLAHE = L1 smooth (Norm(CLAHE(x)) - Norm(G(x)));

[0133] Among them, L1 smooth is the Smooth L1 Loss function; CLAHE(x) is the limited contrast adaptive histogram equalization processing on the original training dataset x; Norm represents the normalization processing;

[0134] Among them, the color constancy loss is:

[0135]

[0136] Among them, (p, q) represents traversing all pairwise combinations in the three color channels; J p represents the average brightness of the p color channel in the enhanced image; J q represents the average brightness of the q color channel in the enhanced image;

[0137] Among them, the exposure control loss is:

[0138]

[0139] Among them, Y k represents the brightness of the k-th large pixel; E is a constant; M is the total number of large pixels;

[0140] Among them, the illumination smoothness loss is:

[0141]

[0142] Among them, N represents the number of iterations; and respectively represent the horizontal and vertical gradient operators; ξ represents the RGB color space, which contains three channels; represents the slice corresponding to the c color channel in the n-th quadratic curve processing iteration.

[0143] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an unsupervised underwater image enhancement program, and when the unsupervised underwater image enhancement program is executed by a processor, the steps of the above-mentioned unsupervised underwater image enhancement method are implemented.

[0144] In summary, the present invention provides an unsupervised underwater image enhancement method and related devices. The method includes: obtaining underwater image data in a public dataset, where the underwater image data includes a training dataset and a test dataset, and performing normalization processing and scaling processing on the underwater image data; constructing a neural network model, and using the processed training dataset to train the neural network model, where the neural network model includes a parameter estimation network and an image enhancement network; inputting the processed test dataset into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset. Through image enhancement, the present invention restores a distorted underwater image into an image with balanced color, rich details, and good contrast, and realizes the enhancement of image data in different water body environments through an unsupervised method, with better versatility.

[0145] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or terminal. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or terminal including that element.

[0146] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by a computer program instructing related hardware (such as a processor, a controller, etc.), and the program can be stored in a computer-readable storage medium readable by a computer. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0147] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. An unsupervised underwater image enhancement method, characterized in that, The unsupervised underwater image enhancement method includes: Obtain underwater image data from a public dataset. The underwater image data includes a training dataset and a test dataset, and perform normalization processing and scaling processing on the underwater image data. Construct a neural network model, and use the processed training dataset to train the neural network model. The neural network model includes a parameter estimation network and an image enhancement network. Input the processed test dataset into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset. The total loss function of the neural network model consists of a CLAHE image contrast loss, a color constancy loss, an exposure control loss, and an illumination smoothness loss. The total loss function is obtained by weighted summation of the CLAHE image contrast loss, the color constancy loss, the exposure control loss, and the illumination smoothness loss. ； Among them, represents the total loss function; represents the CLAHE image contrast loss; represents the color constancy loss; represents the exposure control loss; represents the illumination smoothness loss; Among them, the CLAHE image contrast loss is: ； Among them, is the Smooth L1 Loss function; is to perform limited contrast adaptive histogram equalization on the original training dataset ; represents normalization processing; Among them, the color constancy loss is: ； Among them, indicates that all pairwise combinations in the three color channels have been traversed; indicates the average brightness of the color channel in the enhanced image; indicates the average brightness of the color channel in the enhanced image; Among them, the exposure control loss is: ； Among them, represents the luminance of the th large pixel; is a constant; is the total number of large pixels; Among them, the illumination smoothness loss is: ； Among them, represents the number of iterations; and represent the horizontal and vertical gradient operators respectively; represents the RGB color space, which contains three channels; represents the slice corresponding to the c color channel in the nth iteration of conic processing.

2. The unsupervised underwater image enhancement method according to claim 1, wherein The obtaining of the underwater image data from the public dataset, where the underwater image data includes a training dataset and a test dataset, and performing normalization processing and scaling processing on the underwater image data specifically includes: Obtain the underwater image data from two preset public datasets, use the first preset number of images in the underwater image data as the training dataset, and use the second preset number of images in the underwater image data as the test dataset. Perform normalization processing on the training dataset and the test dataset, and use the min-max normalization method to scale the pixel values of the training dataset and the test dataset from 0 - 255 to the 0 - 1 interval.

3. The unsupervised underwater image enhancement method according to claim 2, characterized in that, The parameter estimation network includes 5 convolutional layers, 2 fully connected layers, and an activation function, and the activation function is the LeakyReLU function. The image enhancement network includes a CLAHE module, 4 convolutional layers, and 3 deconvolutional layers.

4. The unsupervised underwater image enhancement method according to claim 3, wherein The constructing of the neural network model and using the processed training dataset to train the neural network model specifically includes: Input the scaled training dataset into the parameter estimation network. The scaled training dataset passes through 5 convolutional layers and 2 fully connected layers in sequence and then outputs 9 real values, and use the 9 real values as the parameters of the image enhancement network. Input the training dataset into the image enhancement network, generate an image after limited contrast adaptive histogram equalization through the CLAHE module, then pass through 4 convolutional layers and 3 deconvolutional layers, perform slicing processing on the generated 12-channel image to generate 4 three-channel images, perform a quadratic curve transformation iteratively with the original training dataset, and then perform USM sharpening to obtain an enhanced image to complete the training of the neural network model.

5. The unsupervised underwater image enhancement method according to claim 4, wherein The formula for the quadratic curve transformation is as follows: ； ； Among them, represents the original training data set; is the image after completing the conic transformation; is the image after the -th conic processing of the original training data set; is the image after the -1-th conic processing of the original training data set; is the image after the 4-th conic processing of the original training data set; represents the -th slice image; and are the weights of the conic, given by the parameter estimation network.

6. The unsupervised underwater image enhancement method according to claim 5, characterized in that The formula for the USM sharpening is as follows: ； Among them, indicates performing USM sharpening on the original training dataset ; is the sharpening weight given by the parameter estimation network; indicates performing Gaussian blurring on the original training dataset ; Input of the USM sharpening function Equal to the output of the conic transformation .

7. An unsupervised underwater image enhancement system, characterized in that, The unsupervised underwater image enhancement system is applied to the unsupervised underwater image enhancement method according to any one of claims 1-6, and the unsupervised underwater image enhancement system includes: A data acquisition and preprocessing module, configured to acquire underwater image data from a public dataset, where the underwater image data includes a training dataset and a test dataset, and perform normalization processing and scaling processing on the underwater image data; A network construction and training module, configured to construct a neural network model, and use the processed training dataset to train the neural network model, where the neural network model includes a parameter estimation network and an image enhancement network; An image enhancement processing module, configured to input the processed test dataset into the trained neural network model, and the parameter estimation network and the image enhancement network output an underwater enhanced image according to the processed test dataset.

8. A terminal, characterized in that, The terminal includes: a memory, a processor, and an unsupervised underwater image enhancement program stored on the memory and executable on the processor. When the unsupervised underwater image enhancement program is executed by the processor, the steps of the unsupervised underwater image enhancement method according to any one of claims 1-6 are implemented.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an unsupervised underwater image enhancement program. When the unsupervised underwater image enhancement program is executed by a processor, the steps of the unsupervised underwater image enhancement method according to any one of claims 1-6 are implemented.

Images