Lightweight underwater image enhancement method

Abstract

The invention relates to the field of underwater image enhancement, in particular to a lightweight underwater image enhancement method, which comprises the following steps: constructing an underwater image data set; constructing a convolution down-sampling layer and a convolution up-sampling layer based on the network basic module and the channel attention module; constructing an underwater image enhancement network, and constructing a fusion loss function L used for network training to construct a fusion loss function of the underwater image enhancement network; according to the fusion loss function, using the underwater image number of the underwater image data set to train an underwater image enhancement network; and deploying the trained underwater image enhancement network to an end application device for operation, and enhancing the underwater image through the underwater image enhancement network to obtain an enhanced underwater image. According to the method, the underwater image enhancement network parameter quantity and the calculation complexity can be reduced, the image feature extraction capability is improved, the quality of the underwater image is improved, the key degradation problem of the underwater image is solved, and the perception capability and the task execution efficiency of end equipment are improved.

Description

Technical Field

[0001] The present invention relates to the field of underwater image enhancement, and in particular to a lightweight underwater image enhancement method. Background Art

[0002] Underwater operations, due to their high risk and complexity, have always been a major engineering challenge. To improve operational safety and efficiency, underwater vision robots are widely used in fields such as marine exploration and resource development. However, due to the harsh underwater environment, visual images collected by underwater robots often suffer from quality degradation issues such as low contrast, blur, and color distortion, which severely reduce the robots' operational efficiency. Therefore, it is necessary to research applicable underwater image enhancement methods to improve the quality of underwater images. At the same time, due to the limited computing power and storage resources of underwater robots, the image enhancement methods deployed must have low storage requirements and low computational complexity.

[0003] Traditional underwater image enhancement algorithms have clear principles and strong interpretability, achieving good enhancement results when water depth varies little and the underwater conditions are stable. However, these algorithms rely on imaging theories, mostly based on surface imaging, that cannot fully explain the underwater imaging mechanisms. Consequently, they are not well suited for complex imaging scenarios with varying depths. Deep learning-based methods, leveraging the powerful feature extraction capabilities of deep networks, have advanced underwater image enhancement technology. However, deep networks still face challenges in image enhancement and network lightweighting. Deep learning models, especially deep networks, typically contain a large number of parameters, resulting in high computational complexity. For example, models such as convolutional neural networks (CNNs) and generative adversarial networks (GANs) require significant computational resources and time to process high-resolution underwater images, limiting their deployment in real-time applications. Furthermore, the complex network structure complicates training and inference, especially in resource-constrained underwater equipment. Lightweight networks exhibit poor image restoration performance. To reduce computational complexity, researchers have proposed various lightweight network designs. However, while lightweight networks reduce parameters and computational complexity, they often sacrifice feature extraction capabilities, resulting in poor image restoration effects. Especially when dealing with complex underwater scenes, problems such as detail loss and color distortion are prone to occur. Summary of the Invention

[0004] In order to solve the technical problems existing in the prior art, the present invention provides a lightweight underwater image enhancement method. Underwater images are enhanced through a lightweight underwater image enhancement network to obtain enhanced underwater images. The quality of underwater images can be improved, the key degradation problems of underwater images can be solved, and the availability of underwater images can be improved, thereby significantly improving the perception capability and task execution efficiency of terminal devices.

[0005] The first object of the present invention can be achieved by adopting the following technical solutions:

[0006] A lightweight underwater image enhancement method comprises the following steps:

[0007] S1. Construct an underwater image dataset and divide the underwater image data of the underwater image dataset into a training set, a validation set, and a test set;

[0008] S2. Based on the network basic module and channel attention module, build the convolution downsampling layer and convolution upsampling layer;

[0009] S3. Build an underwater image enhancement network based on the convolution downsampling layer and the convolution upsampling layer.

[0010] S4. Construct a fusion loss function of the underwater image enhancement network based on L1 loss, SSIM loss and contrastive learning loss;

[0011] S5. According to the fusion loss function, the underwater image enhancement network is trained using the number of underwater images in the underwater image dataset to obtain a trained underwater image enhancement network.

[0012] S6. Deploy the trained underwater image enhancement network to the terminal application device for operation, enhance the underwater image through the lightweight underwater image enhancement network, and obtain an enhanced underwater image.

[0013] Specifically, the convolutional downsampling layer and the convolutional upsampling layer are constructed based on the network basic module and the channel attention module, including:

[0014] The construction is based on the basic network module, which consists of convolutional layers, batch normalization layers and activation functions;

[0015] Construct a channel attention module, which includes an average pooling layer, a point convolution layer, a ReLU activation function, a point convolution layer, and a Hardsigmoid activation function. The average pooling layer, the point convolution layer, the ReLU activation function, the point convolution layer, and the Hardsigmoid activation function are connected in sequence.

[0016] The parameters of the basic network modules are adjusted to obtain the point convolution layer and the depth convolution layer respectively. The convolution downsampling layer and the convolution downsampling layer are constructed based on the point convolution layer, the depth convolution layer and the channel attention module respectively.

[0017] Specifically, the downsampling layer includes a first point convolution layer, a depth convolution, a channel attention layer and a second point convolution layer. The first point convolution layer, the depth convolution layer, the channel attention layer and the second point convolution layer are connected in sequence. The activation function of the first point convolution is ReLU or Hard-swish activation function, and the activation function of the second point convolution layer is Identity activation function.

[0018] Specifically, the upsampling layer includes an upsampling module, depth convolution, channel attention layer and point convolution. The upsampling module, depth convolution, channel attention layer and point convolution are connected in sequence. The upsampling module is used to increase the height and width of the input feature map through bilinear interpolation.

[0019] Specifically, the underwater image enhancement network is constructed based on the convolutional downsampling layer and the convolutional upsampling layer, including:

[0020] Build a standard convolutional layer based on the network basic module. The standard convolutional layer consists of a convolutional layer, batch normalization, and ReLU activation function.

[0021] Four downsampling layers and five upsampling layers are set, namely the first downsampling layer, the second downsampling layer, the third downsampling layer, the fourth downsampling layer, the first upsampling layer, the second upsampling layer, the third upsampling layer, the fourth upsampling layer, and the fifth upsampling layer. The underwater degraded image is connected through the input of the standard convolution layer, and the standard convolution layer, the first downsampling layer, the second downsampling layer, the third downsampling layer, the fourth downsampling layer, and the first upsampling layer are connected in sequence; the output of the third downsampling layer and the output channel of the first upsampling layer are spliced and input into the second upsampling layer, the output of the second downsampling layer and the output channel of the second upsampling layer are spliced and input into the third upsampling layer, the output of the first downsampling layer and the output channel of the third upsampling layer are spliced and input into the fourth upsampling layer, the output of the standard convolution layer and the output channel of the fourth upsampling layer are spliced and input into the fifth upsampling layer, and the fifth upsampling layer outputs the enhanced image.

[0022] Specifically, the activation functions of the point convolution layer and the depth convolution layer in the first upsampling layer, the second upsampling layer, the third upsampling layer, and the fourth upsampling layer are ReLU or Hard-swish, the activation function of the depth convolution layer of the fifth upsampling layer is ReLU or Hard-swish, and the activation function of the point convolution layer of the fifth upsampling layer is Tanh function.

[0023] Specifically, the terminal equipment includes underwater robots, underwater vehicles, and remotely operated submersibles.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0025] The present invention provides a lightweight underwater image enhancement method. By constructing convolutional downsampling layers and convolutional upsampling layers based on a network foundation module and a channel attention module, the method reduces the number of parameters and computational complexity of the underwater image enhancement network while improving the image feature extraction capability. Underwater images are enhanced using the lightweight underwater image enhancement network to obtain enhanced underwater images. This method can improve underwater image quality, resolve key degradation issues of underwater images, and enhance the usability of underwater images, thereby significantly improving the perception capability and task execution efficiency of terminal devices. The present invention can achieve good underwater image enhancement effects while achieving network lightweighting. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0027] Figure 1 is a flow chart of a lightweight underwater image enhancement method according to an embodiment of the present invention;

[0028] Figure 2 is a schematic diagram of the network basic module structure in an embodiment of the present invention;

[0029] Figure 3 Schematic diagram of the channel attention module structure in an embodiment of the present invention;

[0030] Figure 4 Schematic diagram of the structure of the convolutional downsampling layer and the convolutional upsampling layer in an embodiment of the present invention;

[0031] Figure 5 1 is a diagram of the overall architecture of the underwater image enhancement network in an embodiment of the present invention;

[0032] Figure 6 2 is a diagram of the network training process of the method in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is obvious that the embodiments described are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited to these. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] Example 1:

[0035] like Figure 1 FIG. 1 is a flow chart of a lightweight underwater image enhancement method. The lightweight underwater image enhancement method of the present invention includes the following steps:

[0036] S1. Construct an underwater image dataset and divide the underwater image data of the underwater image dataset into a training set, a validation set, and a test set.

[0037] S11. Underwater image data was obtained from public underwater image enhancement datasets to construct an underwater image dataset. The underwater image datasets contain underwater image data of different degradation scenarios and depths, as well as reference image data corresponding to the degraded underwater images. Public underwater image enhancement datasets include UIEBD, EUVP, RUIE, SUID, and LSUI. UIEBD (Underwater Image Enhancement Benchmark Dataset) contains 950 real underwater images and 890 reference images, covering different water depths, turbidity, and lighting conditions. EUVP (Enhancing Underwater Visual Perception Dataset) consists of paired blurry and clear underwater images, categorized into three scene types: "low light," "high light," and "turbid," and is suitable for supervised learning. Underwater image degradation is primarily caused by factors such as light absorption, scattering, suspended particles, and water disturbances. Different degradation scenarios include: shallow water, with ample light, moderately reduced visual contrast, and slight color distortion; mid-water, with significantly reduced light intensity, a significant blue cast, and a hazy blur. In deep water, strong blue dominates, highlights are significantly reduced, and contrast attenuation is evident. In turbid waters, forward scattering is caused by suspended particles, blurring close-range targets. In dynamic environments, water flow disturbances are simulated in shallow waters to observe dynamic imaging degradation characteristics.

[0038] S12. Split the underwater image data set into a training set, a validation set, and a test set in a ratio of 8:1:1 to provide data support for training the underwater image enhancement network. To ensure the generalization ability of the model, maintain a balanced number of samples at different depths and a reasonable distribution of samples for each degradation type.

[0039] S2. Based on the network basic module and channel attention module, construct the convolutional downsampling layer and the convolutional upsampling layer.

[0040] S21. Construct a network-based basic module, which consists of a convolutional layer, a batch normalization layer, and an activation function. The activation function is one of the ReLU activation function, the Hard-swish activation function, the Identity activation function, and the Tanh activation function. The activation function is used to introduce nonlinear factors into the network model, thereby enhancing the network's expressive power and enabling it to learn and approximate complex nonlinear functions. Figure 2 As shown in the figure, it is a schematic diagram of the network basic module structure.

[0041] In this embodiment, Hard-swish is used when the number of channels is greater than 256; Identity is used to form the final point convolution of each downsampling layer; and Tanh is used to generate the enhanced image at upsampling layer 5. The ReLU activation function (Rectified Linear Unit) effectively alleviates the vanishing gradient problem of deep networks by maintaining a constant gradient in the positive interval. At the same time, the calculation only requires threshold judgment, making hardware implementation extremely efficient. The ReLU formula is as follows:

[0042] ReLU(x)=max(0,x) (1)

[0043] Hard-swish replaces exponential operations with linear segmentation to reduce the computational complexity of mobile devices.

[0044] The negative interval retains nonlinear responses, and the non-saturation characteristics make gradient propagation more stable, which is often used in lightweight models. The formula of Hard-swish is as follows:

[0045]

[0046] Tanh output range (-1,1) and is centered at zero. This symmetry makes the gradient descent process more stable. The formula of Tanh is as follows:

[0047]

[0048] The identity activation function implements identity mapping through skip connections, allowing gradients to be directly propagated back to shallow layers, solving the gradient decay problem in deep network training. The formula for the identity activation function is as follows:

[0049] Identity(x)=x (4)

[0050] S22. Construct a channel attention module. The channel attention module includes an average pooling layer, a point convolution layer, a ReLU activation function, a point convolution layer, and a Hardsigmoid activation function. The average pooling layer, the point convolution layer, the ReLU activation function, the point convolution layer, and the Hardsigmoid activation function are connected in sequence.

[0051] like Figure 3 Figure 2 shows the structure of the channel attention module. The input feature map is of size H × W × C, where H, W, and C are the height, width, and number of channels of the feature map, respectively. After global average pooling, it is converted into a 1 × 1 × C vector. This 1 × 1 × C vector undergoes a point convolution to reduce the number of channels by a factor of 2. It then undergoes a ReLU activation function and a point convolution to restore the number of channels. This vector then undergoes a Hardsigmoid activation function, becoming the channel attention coefficient vector. This vector is then multiplied by the input feature map channels to produce the final output feature map. The channel attention module is primarily used to dynamically learn the importance weights of different channels, highlighting key features and suppressing redundant information, thereby improving the model's representational capabilities and task performance.

[0052] S23. Adjust the parameters of the network basic module to obtain the point convolution layer and the depth convolution layer respectively. Based on the point convolution layer, the depth convolution layer and the channel attention module, construct the convolution downsampling layer and the convolution downsampling layer respectively.

[0053] Specifically, the downsampling layer includes a first point convolution layer, a depthwise convolution layer, a channel attention layer, and a second point convolution layer. The first point convolution layer, the depthwise convolution layer, the channel attention layer, and the second point convolution layer are connected in sequence. The activation function of the first point convolution layer is ReLU or Hard-swish activation function, and the activation function of the second point convolution layer is Identity activation function. Among them, Hard-swish is more effective than ReLU in deep features, so as the network operates, Hard-swish is used in deep features. Using ReLU or Hard-swish in the second point convolution layer will cause feature information to be lost, affecting the network effect.

[0054] The basic network module consists of a convolution layer, a batch normalization layer, and an activation function. Point convolution and depth convolution also include convolution layers, batch normalization layers, and activation functions. When the convolution kernel size is 3, the padding is 1, and the number of convolution kernels is α times the number of input channels, the network basic module is αC. in , the group convolution is αC in When , it is depth convolution, that is, the number of groups is equal to the number of input channels. When the convolution kernel size is 1, the padding is 0, and the number of convolution kernels is αC in or C out When , it is point convolution, that is, the number of output channels is α times the number of input channels, or directly Cout , when C out When , it is directly adjusted to the target output channel number. Point convolution is usually used to adjust the number of channels, such as dimensionality increase or decrease.

[0055] like Figure 4 As shown, the structure diagram of convolution downsampling layer and convolution upsampling layer, H in ×W in ×C in The feature map enters the downsampling layer, first through the point convolution layer to increase the number of channels to α times the original, to obtain H in ×W in ×αC in Feature map, α is 4 in downsampling layer 1, and 2 in downsampling layers 2 to 4. Then the height and width of the feature map are changed through the depth convolution layer to obtain H 0ut ×W out ×αC in The feature map is obtained by reducing the number of channels to the number of output channels through the attention layer and a point convolution layer. 0ut ×W out ×αC 0ut Feature map.

[0056] Specifically, the upsampling layer includes an upsampling module, depth convolution, channel attention layer and point convolution. The upsampling module, depth convolution, channel attention layer and point convolution are connected in sequence. The upsampling module is used to increase the height and width of the input feature map through bilinear interpolation.

[0057] like Figure 4 As shown, the structure diagram of convolution downsampling layer and convolution upsampling layer, H in ×W in ×C in The feature map first enters the upsampling module in the upsampling layer to increase the height and width of the feature map to twice the original, and obtain H 0ut ×W out ×C in The feature map then directly enters the depthwise convolution module, which does not change the dimension of the feature map. After passing through the channel attention module and a point convolution to reduce the number of channels, the number of channels becomes the output channel number.

[0058] S3. Build an underwater image enhancement network based on the convolutional downsampling layer and the convolutional upsampling layer, and construct the fusion loss function of the underwater image enhancement network.

[0059] S31. Build a standard convolutional layer based on the basic network module. The standard convolutional layer consists of a convolutional layer, batch normalization, and ReLU activation function.

[0060] S32. Set 4 downsampling layers and 5 upsampling layers, namely the first downsampling layer, the second downsampling layer, the third downsampling layer, and the fourth downsampling layer; the first upsampling layer, the second upsampling layer, the third upsampling layer, the fourth upsampling layer, and the fifth upsampling layer. The underwater degraded image is connected through the input of the standard convolution layer, and the standard convolution layer, the first downsampling layer, the second downsampling layer, the third downsampling layer, the fourth downsampling layer, and the first upsampling layer are connected in sequence; the output of the third downsampling layer and the output channel of the first upsampling layer are spliced and input into the second upsampling layer, the output of the second downsampling layer and the output channel of the second upsampling layer are spliced and input into the third upsampling layer, the output of the first downsampling layer and the output channel of the third upsampling layer are spliced and input into the fourth upsampling layer, the output of the standard convolution layer and the output channel of the fourth upsampling layer are spliced and input into the fifth upsampling layer, and the fifth upsampling layer outputs the enhanced image.

[0061] Specifically, the activation functions for the point-wise convolutional layers and depth-wise convolutional layers in the first, second, third, and fourth upsampling layers are ReLU or Hard-swish. The activation function for the depth-wise convolutional layer in the fifth upsampling layer is also ReLU or Hard-swish, and the activation function for the point-wise convolutional layer in the fifth upsampling layer is the Tanh function. Hard-swish performs better than ReLU for deep features, so as the network continues to operate, Hard-swish is used for deep features. The point-wise convolutional layer in the fifth upsampling layer finally outputs an enhanced image, so the Tanh function is used.

[0062] like Figure 5 As shown in the overall architecture diagram of the underwater image enhancement network, in this embodiment, the downsampling layer and upsampling layer are formed according to the data in Table 1, and then according to Figure 5 Each layer of modules is gradually spliced together and a jump connection is formed. The standard convolution layer is the module described in step (4-1), and the downsampling layers 1 to 4 are constructed according to the parameter structure in Table 1. The upsampling layers 1 to 5 are constructed according to the parameter structure in Table 1. The underwater degraded image obtains deep features after passing through the standard convolution layer and the downsampling layers 1 to 4, and then the final enhanced image is obtained by combining the output of the upsampling layers 1 to 5 with the standard convolution layer and the downsampling layers 1 to 4. Figure 5 middle, This is a channel concatenation operation, with H and W being 256 and C being 32. Black arrows indicate the output paths of each layer. For example, the output of a standard convolutional layer enters downsampling layer 1, where it is also channel-concatenated with the output of upsampling layer 4 and input to upsampling layer 5. Specifically, in Table 1: Activation function: H represents Hard-swish, R represents ReLU, and T represents Tanh. The first point-wise convolution layer and depth-wise convolution in upsampling layer 5 use the ReLU function, while the final point-wise convolution uses the Tanh activation function.

[0063] Table 1

[0064]

[0065] S4. Construct a fusion loss function of the underwater image enhancement network based on L1 loss, SSIM loss and contrastive learning loss.

[0066] S41. Construct L1 loss, SSIM loss and contrastive learning loss respectively, where:

[0067] L1 loss (mean absolute error, MAE) calculates the average of the absolute difference between each pixel of the predicted image and the target image. In image restoration tasks, L1 loss usually produces clearer images. SSIM loss (structural similarity index) L1 loss L a As shown in Formula 5:

[0068]

[0069] Where, φ(I(p),w i ) represents the parameter w i and underwater image enhancement network with input image I(p), p is the pixel coordinate in pixel region P, N is the number of pixels in pixel region P. I is the degraded image and J is the reference image.

[0070] SSIM loss measures the similarity between two images in terms of brightness, contrast, and structure. Its formula combines these three comparisons and outputs a value between 0 and 1, with 1 indicating identical images. SSIM loss is usually defined as 1-SSIM to minimize structural differences. SSIM is shown in Formula 6:

[0071]

[0072] Where x and y are two independent images, μ is the mean of the image, σ represents the standard deviation of the image, and σ xy Represents the covariance of images x and y, C1 and C2 are two small constants to prevent The SSIM value is between 0 and 1. The closer it is to 1, the higher the image quality. b As shown in Formula 7:

[0073] L b =1-SSIM(φ(I(p),w i ),J(P)) (7)

[0074] Contrastive learning is a type of self-supervised learning that learns meaningful feature representations by shortening the distance between positive sample pairs (similar samples) and pushing the distance between negative sample pairs (dissimilar samples). In image enhancement tasks, contrastive learning may be used to ensure that the enhanced image is close to the clear image in the feature space and away from the degraded input image or other irrelevant samples. c , as shown in Formula 8:

[0075]

[0076] Where G is the VGG-19 feature extractor, G i is the vector of the i-th layer output from VGG-19. φ(x,w) represents the underwater image enhancement network φ with parameters w and input x. I, J, φ(I,w) represent the degraded image, reference image, and enhanced image respectively. ω i Represents the weight coefficient of the i-th layer output vector participating in the loss calculation. When i=1,…,5, Deep features are extracted from shallow features. Deep features tend to extract common characteristics and are more suitable for contrastive learning. Therefore, when setting weight parameters, we tend to give deeper features (features with higher i values) higher weights.

[0077] S42. Construct a fusion loss function L for network training based on L1 loss, SSIM loss and contrastive learning loss.

[0078] L1 and SSIM can act on both pixel and structure levels, while contrastive learning loss acts on feature space. It is necessary to determine the weight of each loss. The fusion loss function L is shown in Formula 9:

[0079] L=λ1L a +λ2L b +λ3L c (9)

[0080] Where λ1, λ2, and λ3 represent the loss weight coefficients used by the user to measure the importance of the loss. In this embodiment, let λ1 = 1, λ2 = 1, and λ3 = 0.1.

[0081] like Figure 6 As shown in Figure 1, the schematic diagram of the generation process of the fusion loss is shown. After the underwater degraded image is input into the network, the enhanced image is output. Among them, L1 loss and SSIM loss are obtained by the enhanced image and the reference image through formula (5) and formula (7). After the degraded image, enhanced image, and reference image pass through VGG-19, three feature vector groups are obtained, each with 5 feature vectors. The feature vector group generated by the degraded image is recorded as G i (I) The feature vector group generated by the enhanced image is recorded as G i(φ(I,w)), the feature vector group generated by the reference image is recorded as G i (J), where i = 1, 2, 3, 4, 5, representing five different feature vectors in each group. The three different feature vectors are used to obtain the contrastive learning loss using formula (8). The L1 loss, SSIM loss, and contrastive learning loss are combined using formula (9) to obtain the fusion loss, which is used for network training.

[0082] S5. According to the fusion loss function, the underwater image dataset is used to train the underwater image enhancement network to obtain a trained underwater image enhancement network;

[0083] After obtaining the underwater image dataset, underwater image enhancement network, and fusion loss, network training is performed using a computer or cloud server. In this example, a server is used for training, using the PyTorch deep learning framework, the Ubuntu 22.04 operating system, a 16GB NVIDIA RTX4080 GPU, an Intel Xeon Gold 5218R CPU (20 cores, 40 threads, 2.1GHz), and 256GB of RAM. Basic training parameters include: input image size of 256×256, a batch size of 24, a learning rate of 0.001, and training with the Adam optimizer for 300 epochs.

[0084] Load the trained network weights into the underwater image enhancement network, load the image in the test set and input it into the underwater image enhancement network to return the enhanced image. The enhanced image should have clear, moderate contrast, rich colors and other visually friendly characteristics to human vision.

[0085] S6. Deploy the underwater image enhancement network to the terminal device for operation, enhance the underwater image through the lightweight underwater image enhancement network, and obtain an enhanced underwater image.

[0086] Specifically, terminal devices include one or more of underwater robots, underwater vehicles, and remotely operated vehicles. In the fields of ocean exploration, deep-sea operations, and submarine archaeology, terminal devices such as underwater robots (ARVs), underwater vehicles (AUVs), and remotely operated vehicles (ROVs) are taking on increasingly complex tasks. However, light scattering, absorption, and interference from suspended particles in the underwater environment cause a serious degradation in image quality, which directly affects the device's target recognition, path planning, and task execution capabilities. The underwater image enhancement network is deployed in an underwater vehicle or remotely operated vehicle for operation. The underwater image is enhanced by the lightweight underwater image enhancement network to obtain an enhanced underwater image. The enhanced underwater image should have the characteristics of clarity, moderate contrast, and rich colors in terms of visual effects.

[0087] In response to the problem of applying underwater image enhancement methods in underwater robots, this embodiment proposes a lightweight underwater image enhancement network suitable for resource-constrained devices. First, by constructing convolutional downsampling layers and convolutional upsampling layers based on network basic modules and channel attention modules, the network parameters and computational complexity are reduced, while the image feature extraction capability is improved. Secondly, contrastive learning loss is used in training to further improve the performance of the model by learning the similarities and differences between images. The trained underwater image enhancement network is deployed to the terminal application device for operation. The underwater image is enhanced by the lightweight underwater image enhancement network to obtain an enhanced underwater image. This can improve the quality of the underwater image, solve the key degradation problem of the underwater image, and improve the availability of the underwater image in real time, thereby significantly improving the perception capability and task execution efficiency of the terminal device.

[0088] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A lightweight underwater image enhancement method, characterized in that: The following steps are involved: S1. Construct an underwater image dataset and divide the underwater image data of the underwater image dataset into a training set, a validation set, and a test set; S2. Based on the network basic module and channel attention module, build the convolution downsampling layer and convolution upsampling layer; S3. Build an underwater image enhancement network based on convolutional downsampling layer and convolutional upsampling layer; S4. Construct a fusion loss function of the underwater image enhancement network based on L1 loss, SSIM loss and contrastive learning loss; S5. According to the fusion loss function, the underwater image enhancement network is trained using the number of underwater images in the underwater image dataset to obtain a trained underwater image enhancement network. S6. Deploy the trained underwater image enhancement network to the terminal application device for operation, enhance the underwater image through the lightweight underwater image enhancement network, and obtain an enhanced underwater image.

2. The method for lightweight underwater image enhancement according to claim 1, characterized in that: The convolution downsampling layer and the convolution upsampling layer are constructed based on the network basic module and the channel attention module, including: Build the basic network module, which consists of convolutional layer, batch normalization layer and activation function; Construct a channel attention module, which includes an average pooling layer, a point convolution layer, a ReLU activation function, a point convolution layer, and a Hardsigmoid activation function. The average pooling layer, the point convolution layer, the ReLU activation function, the point convolution layer, and the Hardsigmoid activation function are connected in sequence. The parameters of the basic network modules are adjusted to obtain the point convolution layer and the depth convolution layer respectively. The convolution downsampling layer and the convolution downsampling layer are constructed based on the point convolution layer, the depth convolution layer and the channel attention module respectively.

3. The lightweight underwater image enhancement method according to claim 1, characterized in that: The downsampling layer includes a first point convolution layer, a depth convolution, a channel attention layer and a second point convolution layer. The first point convolution layer, the depth convolution layer, the channel attention layer and the second point convolution layer are connected in sequence. The activation function of the first point convolution is ReLU or Hard-swish activation function, and the activation function of the second point convolution layer is Identity activation function.

4. The lightweight underwater image enhancement method according to claim 1, characterized in that: The upsampling layer includes an upsampling module, depth convolution, channel attention layer and point convolution. The upsampling module, depth convolution, channel attention layer and point convolution are connected in sequence. The upsampling module is used to increase the height and width of the input feature map through bilinear interpolation.

5. The lightweight underwater image enhancement method according to claim 1, characterized in that: The underwater image enhancement network is constructed based on the convolution downsampling layer and the convolution upsampling layer, including: Build a standard convolutional layer based on the network basic module. The standard convolutional layer consists of a convolutional layer, batch normalization, and ReLU activation function. Four downsampling layers and five upsampling layers are set, namely the first downsampling layer, the second downsampling layer, the third downsampling layer, the fourth downsampling layer, the first upsampling layer, the second upsampling layer, the third upsampling layer, the fourth upsampling layer, and the fifth upsampling layer. The underwater degraded image is connected through the input of the standard convolution layer, and the standard convolution layer, the first downsampling layer, the second downsampling layer, the third downsampling layer, the fourth downsampling layer, and the first upsampling layer are connected in sequence; the output of the third downsampling layer and the output channel of the first upsampling layer are spliced and input into the second upsampling layer, the output of the second downsampling layer and the output channel of the second upsampling layer are spliced and input into the third upsampling layer, the output of the first downsampling layer and the output channel of the third upsampling layer are spliced and input into the fourth upsampling layer, the output of the standard convolution layer and the output channel of the fourth upsampling layer are spliced and input into the fifth upsampling layer, and the fifth upsampling layer outputs the enhanced image.

6. The lightweight underwater image enhancement method according to claim 5, characterized in that: The activation functions of the point convolution layer and the depth convolution layer in the first upsampling layer, the second upsampling layer, the third upsampling layer, and the fourth upsampling layer are ReLU or Hard-swish, the activation function of the depth convolution layer of the fifth upsampling layer is ReLU or Hard-swish, and the activation function of the point convolution layer of the fifth upsampling layer is Tanh function.

7. The lightweight underwater image enhancement method according to claim 1, characterized in that: The fusion loss function is expressed as: L=λ1L a +λ2L b +λ3L c ; Where L represents the fusion loss, L a represents L1 loss, L b represents the SSIM loss, L c Represents the ratio learning loss, λ1, λ2, λ3 represent the loss weight coefficients respectively; L1 loss is expressed as: Where, φ(I(p),w i ) represents the parameter w i and the underwater image enhancement network with the input image I(p), p is the pixel coordinate in the pixel region P, N is the number of pixels in the pixel region, I is the degraded image, and J is the reference image; The SSIM loss is expressed as: L b =1-SSIM(φ(I(p),w i ),J(p)); In the formula, x and y are two independent images, μ is the mean of the image, σ represents the standard deviation of the image, and σ xy Represents the covariance of images x and y, C1 and C2 are two small constants; The specific learning loss is expressed as: Where G is the VGG-19 feature extractor, G i is the vector of the i-th layer output from VGG-19, φ(x,w) represents the underwater image enhancement network φ with parameter w and input x, I, J, φ(I,w) represent the degraded image, reference image and enhanced image respectively, ω i Represents the weight coefficient of the i-th layer output vector participating in the loss calculation.

8. The lightweight underwater image enhancement method according to claim 1, characterized in that: The terminal equipment includes underwater robots, underwater vehicles, and remote-controlled submersibles.

Images