Underwater image enhancement method based on channel attention and generative adversarial network

Abstract

The application discloses an underwater image enhancement method based on a channel attention and a deformation generative adversarial network, and comprises the following steps: acquiring an underwater image to construct a data set, and dividing the data set into a training set and a test set; constructing an adaptive channel attention module with a multi-scale receptive field for recalibrating channel weights; constructing a deformation convolution module for feature extraction and facing a convolution kernel offset; fusing the adaptive channel attention module and the deformation convolution module to generate a generative adversarial network; training the generative adversarial network based on the training set data to obtain a trained generative adversarial network; inputting the test set data into the trained generative adversarial network to obtain an enhanced underwater image; and constructing an adaptive channel attention module with different receptive fields by using a single hidden layer neural network and a global average pooling technology, which is helpful to reduce the influence of mixed noise on a feature layer and improve the enhancement consistency of an object of interest under different scene depths.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of underwater intelligent fishing robots, and relates to an underwater image enhancement method based on channel attention and deformation generative adversarial network. BACKGROUND

[0002] The ocean contains valuable wealth for human sustainable development and is also a strategic location for high-quality development. Underwater optical images have played a crucial role in underwater archaeology, shipwreck salvage, marine ranching and monitoring due to their strong information carrying capacity, low cost and rich information. It is noted that due to the inherent water absorption and scattering effect, underwater optical images usually exhibit uneven illumination, color cast, low contrast and blurred edge details during imaging, which seriously affects subsequent detection and recognition, image segmentation and other tasks.

[0003] With the rapid development of artificial intelligence, data-driven deep learning technology has been widely applied to underwater image denoising, detail restoration and super-resolution. It is noted that underwater image enhancement can be used as a domain conversion operation from degraded images to clear images. At present, underwater image enhancement methods mainly include model-free enhancement methods, model-based restoration methods and data-driven domain mapping methods.

[0004] (1) Model-free enhancement method

[0005] By solving the color distortion and low contrast problem in a single color space, related scholars proposed the Contrast Limited Adaptive Histogram Equalization (CLAHE) and White Balance (WB) technology. With the help of bilateral and trilateral filtering technology, related scholars designed a multi-scale Retinex framework to realize underwater image enhancement. In addition, by using white balance and histogram equalization strategies to fuse four representative weight maps of Laplacian contrast, local contrast, saliency and exposure, related scholars proposed a Fusion underwater image enhancement method.

[0006] (2) Model-based restoration method

[0007] By estimating the transmission map and background light, related scholars proposed a method based on Dark Channel Prior (DCP). By considering the conversion differences between aerial and underwater scenes, related scholars constructed an Underwater Dark Channel Prior (UDCP) framework. By solving the actual problem that the pixel value in the blue channel is occasionally lower than that in the red channel, related scholars proposed a two-channel-based scheme to realize underwater image enhancement.

[0008] (3) Data-driven domain mapping method

[0009] By combining aerial images and depth images, relevant scholars proposed a two-stage WaterGAN scheme to realize underwater image enhancement. By using the corrected underwater image imaging model and scene parameters, relevant scholars proposed a UWCNN method based on synthetic images. In order to eliminate the restriction of using paired underwater images, relevant scholars proposed a weakly supervised learning method with cycle consistency loss. In order to significantly improve the training stability in the process of underwater image enhancement, by calculating the distance between data distribution and model distribution, relevant scholars created a Wasserstein GAN method.

[0010] The existing underwater image enhancement method under the complex seabed environment mainly has the following defects: (1) It is difficult to solve multiple degradation types based on model-free enhancement method; (2) The method based on model restoration needs to estimate a large number of parameters, and the modeling process is complex; (3) The domain mapping method based on data-driven introduces a large amount of underwater mixed noise, the foreground object is not complete, and the enhancement consistency of the object of interest under different scene depths is poor. SUMMARY

[0011] In order to solve the above problems, the technical scheme adopted by the present application is: an underwater image enhancement method based on channel attention and deformation generative adversarial network, comprising the following steps:

[0012] Obtain underwater image construction dataset, divide the dataset into training set and test set;

[0013] Construct an adaptive channel attention module with multi-scale receptive field for recalibrating channel weight;

[0014] Construct a deformation convolution module for feature extraction and convolution kernel offset;

[0015] Fuse the adaptive channel attention module and the deformation convolution module to generate an adversarial network;

[0016] Train the generative adversarial network based on the training set data to obtain the trained generative adversarial network;

[0017] Input the test set data into the trained generative adversarial network to obtain the enhanced underwater image.

[0018] Further, the process of constructing an adaptive channel attention module with multi-scale receptive field for recalibrating channel weight is as follows:

[0019] The convolution operation is expressed as:

[0020]

[0021] where, represents the i-th layer of input features X, represents the i-th layer weight of the k-th filter, represents the convolution operation, U k represents the convolution output, c represents the number of feature layers;

[0022] The fusion operation is represented as:

[0023]

[0024] where, and represent the features obtained using 3x3 and 5x5 filters, respectively, is the fused feature obtained using pixel-wise addition operation;

[0025] The global average pooling is used to calculate the average response of each feature map, which is represented as:

[0026]

[0027] where, represents the channel response value, represents the fused feature, w and h are the width and height of the feature map, respectively, and m and n are the horizontal and vertical coordinate indices of the feature, respectively;

[0028] The single-hidden-layer neural network is used to improve the non-linear mapping capability, which is represented as:

[0029] z fc = R(W fc s) (4)

[0030] where, represents the output of the hidden layer, represents the weight between the input layer and the hidden layer, d = max(c / r, L) represents the number of hidden layer nodes, r represents the reduction rate of the channel dimension, L represents the minimum value of the channel number, and R is the Leaky ReLU activation function;

[0031] Further, the appropriate spatial receptive field is adaptively selected, which is represented as:

[0032]

[0033]

[0034] where, and are the weights between the hidden layer and the output layer for the 3x3 and 5x5 branches, respectively, and Channel attention weights of 3x3 and 5x5 branches, respectively;

[0035] The re-labeled feature V is represented as:

[0036]

[0037] wherein, is the output feature of the entire adaptive channel attention module, and ⊙ represents element-wise multiplication.

[0038] Further, the process of constructing the deformation convolution module for feature extraction and oriented to the convolution kernel offset is as follows:

[0039] The sampling position of the standard convolution kernel is represented as:

[0040] P={(u,v)|u,v∈{-k,-(k-1),…,k}} (8)

[0041] wherein, u and v represent the horizontal coordinate and the vertical coordinate of the sampling position of the convolution kernel, represent the edge position of the convolution kernel, w f represents the current convolution kernel width;

[0042] The sampling position of the deformation convolution is represented as:

[0043]

[0044] wherein, p0 represents the center point of the convolution kernel, p m =(x m ,y m ) is the standard sampling position of the mth element in the convolution kernel, x m and y m represent the horizontal coordinate and the vertical coordinate of the standard sampling position of the mth element in the convolution kernel, m=1,2,…,n represents the element index in the convolution kernel, n=|P| is the total number of elements in the convolution kernel, Δp m represents the offset size of the mth element in the convolution kernel, is the deformation sampling position of the mth element in the convolution kernel, and represent the horizontal coordinate and the vertical coordinate of the deformation position of the mth element in the convolution kernel;

[0045] The sampling position of the deformed standard convolution kernel should satisfy:

[0046]

[0047]

[0048] The non-integer position is calculated from the X direction using the bilinear interpolation techniqueEigenvalue of Eigenvalue of is expressed as:

[0049]

[0050] wherein p tl =(x i ,y j ), p tr =(x i+1 ,y j ), p bl =(x i ,y j+1 ) and p br =(x i+1 ,y j+1 ) represent the nearest left-up, right-up, left-down and right-down integer coordinate positions of the deformation position respectively;

[0051] Eigenvalue of Eigenvalue of is expressed as:

[0052]

[0053] Eigenvalue of Eigenvalue of is expressed as:

[0054]

[0055] wherein, is the eigenvalue of the mth element in the convolution kernel at the deformation position .

[0056] The output of the entire deformation convolution module is expressed as:

[0057]

[0058] wherein O(x0,y0) is the output of performing the deformation convolution operation, represents the weight of the mth element in the convolution kernel, and σ(w m ) is a modulation operation for emphasizing the importance of the mth bias position.

[0059] Further, the generative adversarial network comprises a generator and a discriminator connected in series.

[0060] The generator comprises an encoder and a decoder connected in series.

[0061] The encoder comprises an adaptive channel attention module and a deformation convolution module connected in series.

[0062] Further, the loss used for training the generative adversarial network based on the training set data comprises a WGAN-GP loss, a loss, and an image gradient difference loss.

[0063] An underwater image enhancement device based on channel attention and deformation generative adversarial network comprises:

[0064] An acquisition module is configured to acquire underwater image construction data sets, and divide the data sets into a training set and a test set.

[0065] A construction module I is configured to construct an adaptive channel attention module with a multi-scale receptive field for recalibrating channel weights.

[0066] A construction module II is configured to construct a deformation convolution module for feature extraction and oriented to convolution kernel offset.

[0067] A fusion module is configured to fuse the adaptive channel attention module and the deformation convolution module to generate a generative adversarial network.

[0068] A training module is configured to train the generative adversarial network based on the training set data to obtain a trained generative adversarial network.

[0069] A obtaining module is configured to input test set data into the trained generative adversarial network to obtain an enhanced underwater image.

[0070] The underwater image enhancement method based on the channel attention and deformation generative adversarial network has the following advantages:

[0071] (1) The single hidden layer neural network and the global average pooling technology are used to construct the adaptive channel attention module with different receptive fields, which helps to reduce the influence of mixed noise (speckle noise, Gaussian noise and impulse noise) on the feature layer and improve the enhancement consistency of the object of interest under different scene depths.

[0072] (2) The convolution kernel offset method and the bias position modulation mechanism are constructed, and the feature extraction strategy based on the deformation convolution network is proposed, so as to enhance the foreground object coding ability from the spatial level.

[0073] (3) The adaptive channel attention module and the deformation convolution network are combined, an L1 loss, an image gradient error loss and a generative adversarial loss are utilized, a generative adversarial network framework is constructed, and thus the underwater image enhancement performance is improved from two aspects of channels and space. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0075] Figure 1 Adaptive channel attention module diagram;

[0076] Figure 2 Deformation convolution module diagram;

[0077] Figure 3 Overall underwater image enhancement framework diagram;

[0078] Figure 4 UIEB dataset subjective evaluation result diagram;

[0079] Figure 5 URPC dataset subjective evaluation result diagram;

[0080] Figure 6 Key point matching comparison result diagram. DETAILED DESCRIPTION

[0081] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0082] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0083] It is to be understood that the terms so far as the grammar used herein is concerned are to be interpreted in their dictionary meanings and should not be construed to open down the scope of the present application based on the usage as per se. It is also to be understood that the terminology and description used herein is only by way of explanation of the illustrative embodiments and should not be construed to limit the scope of the present application.

[0084] The relative arrangement of parts and steps, numerical expressions, and numerical values set forth in the examples are not intended to limit the scope of the present application unless otherwise specifically stated. It is also to be understood that the drawings are not necessarily drawn to scale of the actual proportions of the various parts and components shown therein. Techniques, methods, and apparatus known to those of ordinary skill are not discussed in detail because such techniques, methods, and apparatus are considered to be part of the art. All examples shown and discussed herein are intended to be exemplary and non-limiting. Therefore, other examples of the illustrative embodiments can have different values. It is noted that like numbers and letters on the figures identify like parts throughout the disclosure, thus, once an item is defined in one figure, it is not necessary to discuss it further in connection with other figures where it is not explicitly defined.

[0085] In the description of the present application, it is to be understood that the orientation or positional relationships indicated by terms such as "front", "back", "upper", "lower", "left", "right", "horizontal", "vertical", "top", "bottom", and the like are based on the orientation or positional relationships shown in the drawings, and are merely intended to facilitate the description of the present application and simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation or be constructed and operated in a particular orientation, and therefore should not be construed as limiting the scope of protection of the present application. The orientation terms "inner", "outer" refer to the inner and outer relative to the outline of the components themselves.

[0086] For ease of description, spatial relative terms such as "above," "over," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation besides the orientation of the device as described in the figures. For example, if the device in the figures is inverted, a device described as "above" or "above" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0087] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.

[0088] An underwater image enhancement method based on channel attention and deformation generative adversarial networks includes the following steps:

[0089] S1: Obtain underwater images to construct a dataset, and divide the dataset into training and testing sets;

[0090] S2: Construct an adaptive channel attention module with multi-scale receptive fields to recalibrate channel weights;

[0091] S3: Construct deformable convolutional modules for feature extraction and kernel offset orientation;

[0092] S4: The adaptive channel attention module and the deformable convolution module are fused to generate an adversarial network;

[0093] S5: Train the generative adversarial network based on the training set data to obtain a trained generative adversarial network;

[0094] S6: Input the test set data into the trained generative adversarial network to obtain the enhanced underwater image.

[0095] Steps S1, S2, S3, S4, S5, and S6 are executed sequentially.

[0096] Furthermore, the process of constructing the adaptive channel attention module with multi-scale receptive fields used to recalibrate channel weights is as follows:

[0097] The adaptive channel attention module is mainly used to recalibrate the channel weights. It is noted that using two different receptive field convolution kernels at the channel level to enhance features can significantly improve visual quality. Therefore, as shown in Figure 1 the convolution operation can be represented as:

[0098]

[0099] wherein, represents the i-th layer of input features X, represents the i-th layer weight of the k-th filter, represents the convolution operation, U k represents the convolution output, and c represents the number of feature layers.

[0100] Subsequently, the fusion operation can be represented as:

[0101]

[0102] wherein, and represent the features obtained using 3x3 and 5x5 filters, respectively. is the fused feature obtained using the pixel-wise addition operation.

[0103] Subsequently, the global average pooling is used to calculate the average response of each feature map, which can be represented as:

[0104]

[0105] wherein, represents the channel response value, represents the fused feature, w and h are the width and height of the feature map, respectively, and m and n are the horizontal and vertical coordinate indices of the feature, respectively.

[0106] In addition, a single hidden layer neural network is used to improve the non-linear mapping capability, which can be represented as:

[0107] z fc = R(W fc s) (4)

[0108] wherein, represents the output of the hidden layer, represents the weight between the input layer and the hidden layer, d = max(c / r, L) represents the number of hidden layer nodes, r represents the reduction rate of the channel dimension, L represents the minimum value of the channel number, and R is the Leaky ReLU activation function.

[0109] Further, the appropriate spatial receptive field is adaptively selected, which can be represented as:

[0110]

[0111]

[0112] where, and are the weights between the hidden layer and the output layer for 3x3 and 5x5 branches, respectively. and are the channel attention weights for 3x3 and 5x5 branches, respectively.

[0113] Finally, the re-calibrated feature V can be represented as:

[0114]

[0115] where, is the output feature of the entire adaptive channel attention module, and represents the element-wise product.

[0116] Further, the process of constructing the deformation convolution module for feature extraction and kernel offset is as follows:

[0117] In order to enhance the integrity of the foreground object construction, by constructing the convolution kernel offset method, a feature extraction strategy based on deformation convolution network is proposed, which helps to enhance the coding ability of foreground objects from the spatial level. As shown in Figure 2 the sampling position of the standard convolution kernel can be represented as:

[0118] P = {(u, v) | u, v e {-k, -(k-1), …, k}} (8)

[0119] where u and v represent the horizontal and vertical coordinates of the sampling position of the convolution kernel, represent the edge position of the convolution kernel, w f represents the current convolution kernel width;

[0120] Correspondingly, the sampling position of the deformation convolution can be represented as:

[0121]

[0122] where p0represents the center point of the convolution kernel, p m = (x m , y m ) is the standard sampling position of the mth element in the convolution kernel, x m and y m represent the horizontal and vertical coordinates of the standard sampling position of the mth element in the convolution kernel, m = 1, 2, …, n represents the element index in the convolution kernel, n = |P| is the total number of elements in the convolution kernel, and Δp m represents the bias size of the mth element in the convolution kernel, is the warped sampling position of the m-th element in the convolution kernel, and respectively represent the horizontal and vertical coordinates of the warped position of the m-th element in the convolution kernel;

[0123] Note that the warped sampling position should satisfy:

[0124]

[0125]

[0126] To solve the problem of directly obtaining the feature value at the non-integer position, the present application uses the bilinear interpolation technology to calculate the feature value at the non-integer position from the X direction which can be expressed as:

[0127]

[0128] where p tl = (x i , y j ), p tr = (x i+1 , y j ), p bl = (x i , y j+1 ) and p br = (x i+1 , y j+1 ) respectively represent the left-up, right-up, left-down and right-down integer coordinate positions closest to the warped position.

[0129] Similarly, the feature value at the coordinate can be expressed as:

[0130]

[0131] Further, the feature value at the position can be expressed as:

[0132]

[0133] where is the feature value of the m-th element in the convolution kernel at the warped position .

[0134] Finally, the output of the entire warped convolution module can be expressed as:

[0135] ​​​

[0136] wherein O(x0, y0) is an output of performing the deformation convolution operation, representing an mth element weight in the convolution kernel, σ(w m is a modulation operation used to emphasize the importance of the mth bias position.

[0137] Further, as shown in Figure 3 the generative adversarial network includes a generator and a discriminator connected in series;

[0138] The generator includes an encoder and a decoder connected in series;

[0139] Wherein the adaptive channel attention module and the deformation convolution module are integrated into the encoder framework in an organic series.

[0140] The loss used for training the generative adversarial network based on the training set data includes a WGAN-GP loss, a loss, and an image gradient difference loss.

[0141] An underwater image enhancement device based on a channel attention and deformation generative adversarial network, comprising:

[0142] An acquisition module: acquiring underwater image construction data sets, and dividing the data sets into a training set and a test set;

[0143] A construction module I: constructing an adaptive channel attention module for recalibrating channel weights and having a multi-scale receptive field;

[0144] A construction module II: for constructing a deformation convolution module for feature extraction and oriented to convolution kernel offset;

[0145] A fusion module: for fusing the adaptive channel attention module and the deformation convolution module to generate a generative adversarial network;

[0146] A training module: for training the generative adversarial network based on the training set data to obtain a trained generative adversarial network;

[0147] A obtaining module: for inputting test set data into the trained generative adversarial network to obtain an enhanced underwater image.

[0148] In order to fully prove the effectiveness and superiority of the proposed method, the underwater image enhancement method based on the channel attention and deformation generative adversarial network (referred to as: ACADGAN) proposed in the present application is compared with the restoration method based on the physical model, the model-free enhancement method, and the image-to-image conversion technology.

[0149] Figure 4 The scene of the comparison of the UIEB data set subjective evaluation result graph is shown in Figure 4As shown, it mainly includes artificial light (the first column), blue light (the second column), green light (the third column), dark light (the fourth column) and violet light (the fifth column). From Figure 4 We can clearly observe from Table 1 that the image restoration method based on the physical model is difficult to obtain satisfactory visual performance. Specifically, the UDCP method makes the color deviation effect in the above-mentioned scene more obvious. Obviously, the underwater image enhancement method based on the model, i.e. the UCM method, may cause a red effect. It is noted that the FUnIE-GAN and UWCNN methods can only achieve very limited enhancement effect. At the same time, the FUnIE-GAN method will cause obvious color deviation under the condition of violet light. Intuitively, the ACADGAN method proposed in the present application can finally achieve the best visual enhancement quality. In addition, consistent conclusions can also be summarized from Table 2, Figure 5 Figure 5 URPC dataset subjective evaluation result graph.

[0150] In order to further fully prove the superiority of the ACADGAN method proposed in the present application, the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), underwater image color (UICM), sharpness (UISM) and clarity (UIConM) and the like are used as indexes for comparison on the UIEB and URPC datasets respectively, and the comparison results are summarized in Table 1 and Table 2.

[0151] Table 1 is the objective image quality quantization comparison of the UIEB dataset

[0152]

[0153] Table 2 is the objective image quality quantization comparison of the URPC dataset

[0154]

[0155] From Table 1 and Table 2, we can see that the ACADGAN method proposed in the present application can achieve the best (indicated in bold) or suboptimal (indicated in underline) performance in most indicators. Specifically, on the one hand, according to the optimal SSIM and UIConM indicators obtained on the UIEB dataset, we can infer that the ACADGAN method proposed in the present application can effectively preserve the image structure, texture and contrast. At the same time, the ACADGAN method proposed in the present application can perfectly preserve the image content. In addition, in the evaluation of the URPC dataset, the ACADGAN method proposed in the present application achieves the best UIQM score, which means that the enhanced image is more consistent with human visual perception. More importantly, the ACADGAN method proposed in the present application can obtain the best UCIQE score, which means that the enhanced image achieves a better balance in terms of chroma, saturation and contrast. ​

[0156] To verify the effectiveness and superiority of the ACADGAN method proposed in this patent from the perspective of basic feature expression, this patent uses the SIFT, Harris and Canny methods to extract key points, corner points and pixel-level edges, respectively. Correspondingly, the key point matching results are shown in FIGS. 6, 7 and 8, respectively, from which we can clearly see that, due to the poor quality of the original underwater images, few key points can be correctly extracted and matched. At the same time, the key point matching performance can be enhanced by using restoration or enhancement methods. In addition, the ACADGAN method proposed in this patent can achieve the best key point matching performance, which means that the ACADGAN method proposed in this patent can restore the degraded underwater images to more basic features. It should be emphasized that the average evaluation performance of the UIEB and URPC datasets is provided in Tables 3 and 4, respectively. Figure 6

[0157] Table 3: Feature expression comparison of the UIEB dataset

[0158]

[0159]

[0160] Table 4: Feature expression comparison of the URPC dataset

[0161]

[0162] From this we can clearly see that the ACADGAN method proposed in this patent can achieve the best or suboptimal performance in extracting SIFT key points, Harris corner points and Canny edges, which indicates that the ACADGAN method proposed in this patent is helpful for the extraction of basic features.

[0163] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent substitutions for part or all of the technical features; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.​

Claims

1. An underwater image enhancement method based on channel attention and deformation generative adversarial networks, characterized in that: Includes the following steps: Acquire underwater images to construct a dataset, and divide the dataset into training and testing sets; An adaptive channel attention module with multi-scale receptive field is constructed using a single hidden layer neural network and global average pooling technique to recalibrate channel weights. The process of constructing an adaptive channel attention module with multi-scale receptive fields to recalibrate channel weights is as follows: The convolution operation is represented as: in, The i-th layer represents the input feature X. This represents the weights of the i-th layer of the k-th filter. Represents the convolution operation, U k represents the convolution output, and c represents the number of feature layers; The fusion operation is represented as: IN f =U+U (2) Where U and U represent the features obtained using 3×3 and 5×5 filters, respectively. It is a fusion feature obtained by performing pixel-by-pixel addition; Global average pooling is used to compute the average response for each feature map, which is expressed as follows: in, Represents the channel response value. The fused features are represented by w and h, which are the width and height of the feature map, respectively, and m and n are the x and y coordinate indices of the features, respectively. The ability to improve nonlinear mapping is achieved using a single hidden layer neural network, as expressed in the following: z fc =R(W fc s) (4) in, This represents the output of the hidden layer. The weights between the input layer and the hidden layer are represented by d = max(c / r, L), which represents the number of hidden layer nodes, r represents the rate of reduction of the channel dimension, L represents the minimum number of channels, and R is the Leakly ReLU activation function. Furthermore, the appropriate spatial receptive field is adaptively selected, which is expressed as: in, and These are the weights between the hidden layer and the output layer for 3×3 and 5×5 branches, respectively. and These are the channel attention weights for the 3×3 and 5×5 branches, respectively; The recalibrated feature V is represented as: in, It represents the output feature of the entire adaptive channel attention module, where ⊙ represents element-wise multiplication; Based on the kernel offset method and the bias position modulation mechanism, a deformable convolution module oriented towards kernel offset is constructed for feature extraction. The process of constructing the deformable convolutional module for feature extraction and kernel offset is as follows: The sampling positions of a standard convolutional kernel are represented as follows: P={(u,v)|u,v∈{-k,-(k-1),…,k}} (8) Where u and v represent the x-coordinate and y-coordinate of the sampling position of the convolution kernel, respectively. w represents the edge position of the convolution kernel f Represents the current convolutional kernel width; The sampling positions of deformable convolution are represented as follows: p m =p0+p m +Δp m (9) Where p0 represents the center point of the convolution kernel, p m =(x m ,y m ) represents the standard sampling position of the m-th element in the convolution kernel, x m and y m Let x and y represent the x and y coordinates of the standard sampling position of the m-th element in the convolution kernel, respectively, where m = 1, 2, ..., n represents the element index in the convolution kernel, n = |P| is the total number of elements in the convolution kernel, and Δp is the index of the m-th element in the kernel. m This represents the bias magnitude of the m-th element in the convolution kernel. It is the deformation sampling position of the m-th element in the convolution kernel. and These represent the x and y coordinates of the deformation position of the m-th element in the convolution kernel, respectively. The sampling positions of the deformed standard convolution kernel should satisfy: Non-integer positions are calculated from the X direction using bilinear interpolation. eigenvalues It is represented as: Where: p tl =(x i ,y j ), p tr =(x i+1 ,y j ), p bl =(x i ,y j+1 ) and p br =(x i+1 ,y j+1 These represent the top-left, top-right, bottom-left, and bottom-right integer coordinates closest to the deformation location, respectively. Coordinates eigenvalues Represented as: Location eigenvalues Represented as: in, It is the m-th element in the convolution kernel at the deformation position. eigenvalues; The output of the entire deformable convolution module is represented as: Where O(x0,y0) is the output of the deformable convolution operation. σ(w) represents the weight of the m-th element in the convolution kernel. m This is a modulation operation used to emphasize the importance of the m-th offset position; The adaptive channel attention module and the deformable convolution module are fused to generate an adversarial network; The generative adversarial network is trained based on the training set data to obtain a trained generative adversarial network; The test set data is input into the trained generative adversarial network to obtain the enhanced underwater image.

2. The underwater image enhancement method based on channel attention and deformation generative adversarial networks according to claim 1, characterized in that: The generative adversarial network includes a generator and a discriminator connected in series; The generator includes an encoder and a decoder connected in series; The encoder includes an adaptive channel attention module and a deformation convolution module connected in series.

3. The underwater image enhancement method based on channel attention and deformation generative adversarial networks according to claim 1, characterized in that: The loss methods used to train the generative adversarial network based on the training set data include WGAN-GP loss, loss, and image gradient difference loss.

4. An underwater image enhancement device based on channel attention and deformation generative adversarial networks, characterized in that: include: Acquisition module: Acquires underwater images to construct a dataset, and divides the dataset into training and testing sets; Module I: Using a single hidden layer neural network and global average pooling, an adaptive channel attention module with multi-scale receptive fields is constructed to recalibrate channel weights. The process of constructing an adaptive channel attention module with multi-scale receptive fields to recalibrate channel weights is as follows: The convolution operation is represented as: in, The i-th layer represents the input feature X. This represents the weights of the i-th layer of the k-th filter. Represents the convolution operation, U k represents the convolution output, and c represents the number of feature layers; The fusion operation is represented as: IN f =U+U (2) Where U and U represent the features obtained using 3×3 and 5×5 filters, respectively. It is a fusion feature obtained by performing pixel-by-pixel addition; Global average pooling is used to compute the average response for each feature map, which is expressed as follows: in, Represents the channel response value. The fused features are represented by w and h, which are the width and height of the feature map, respectively, and m and n are the x and y coordinate indices of the features, respectively. The ability to improve nonlinear mapping is achieved using a single hidden layer neural network, as expressed in the following: z fc =R(W fc s) (4) in, This represents the output of the hidden layer. The weights between the input layer and the hidden layer are represented by d = max(c / r, L), which represents the number of hidden layer nodes, r represents the rate of reduction of the channel dimension, L represents the minimum number of channels, and R is the Leakly ReLU activation function. Furthermore, the appropriate spatial receptive field is adaptively selected, which is expressed as: in, and These are the weights between the hidden layer and the output layer for 3×3 and 5×5 branches, respectively. and These are the channel attention weights for the 3×3 and 5×5 branches, respectively; The recalibrated feature V is represented as: in, It represents the output feature of the entire adaptive channel attention module, where ⊙ represents element-wise multiplication; Module II: This module is used to construct a deformable convolutional module for feature extraction based on the kernel offset method and the bias position modulation mechanism. The process of constructing the deformable convolutional module for feature extraction and kernel offset is as follows: The sampling positions of a standard convolutional kernel are represented as follows: P={(u,v)|u,v∈{-k,-(k-1),…,k}} (8) Where u and v represent the x-coordinate and y-coordinate of the sampling position of the convolution kernel, respectively. w represents the edge position of the convolution kernel f Represents the current convolutional kernel width; The sampling positions of deformable convolution are represented as follows: p m =p0+p m +Δp m (9) Where p0 represents the center point of the convolution kernel, p m =(x m ,y m ) represents the standard sampling position of the m-th element in the convolution kernel, x m and y m Let x and y represent the x and y coordinates of the standard sampling position of the m-th element in the convolution kernel, respectively, where m = 1, 2, ..., n represents the element index in the convolution kernel, n = |P| is the total number of elements in the convolution kernel, and Δp is the index of the m-th element in the kernel. m This represents the bias magnitude of the m-th element in the convolution kernel. It is the deformation sampling position of the m-th element in the convolution kernel. and These represent the x and y coordinates of the deformation position of the m-th element in the convolution kernel, respectively. The sampling positions of the deformed standard convolution kernel should satisfy: Non-integer positions are calculated from the X direction using bilinear interpolation. eigenvalues It is represented as: Where: p tl =(x i ,y j ), p tr =(x i+1 ,y j ), p bl =(x i ,y j+1 ) and p br =(x i+1 ,y j+1 These represent the top-left, top-right, bottom-left, and bottom-right integer coordinates closest to the deformation location, respectively. Coordinates eigenvalues Represented as: Location eigenvalues Represented as: in, It is the m-th element in the convolution kernel at the deformation position. eigenvalues; The output of the entire deformable convolution module is represented as: Where O(x0,y0) is the output of the deformable convolution operation. σ(w) represents the weight of the m-th element in the convolution kernel. m This is a modulation operation used to emphasize the importance of the m-th offset position; Fusion module: Used to fuse the adaptive channel attention module and the deformable convolution module to generate an adversarial network; Training module: Used to train the generative adversarial network based on the training set data to obtain a trained generative adversarial network; The "Get" module is used to input test set data into the trained generative adversarial network to obtain enhanced underwater images.

Images