Image enhancement method, device and medium for underwater visual navigation and mapping

Abstract

Embodiments of the present application relate to the technical field of image processing, and disclose an image enhancement method, device and medium for underwater visual navigation and mapping. The method comprises: acquiring a plurality of sample underwater images and corresponding reference images, and generating a training set and a verification set; training a pre-constructed generator and discriminator based on the training set and a preset loss function, to obtain a trained generator; inputting each sample underwater image in the verification set into the trained generator, and adjusting each hyperparameter in the trained generator according to the output of the trained generator and the corresponding reference image of each sample underwater image, to obtain an enhancement model; inputting a to-be-enhanced underwater image into the enhancement model, to obtain an enhanced image, and performing visual SLAM, so as to quickly improve the quality of the original underwater image, increase the number of feature points in the image, and ensure that underwater visual navigation and mapping can be smoothly performed.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image enhancement method, device and medium for underwater visual navigation and mapping. Background Technology

[0002] Since the beginning of the 21st century, with the gradual depletion of land resources, research has focused on the development of marine resources. AUVs (Autonomous Underwater Vehicles) can better replace manual labor in underwater operations such as seabed resource exploration, seabed archaeology, and seabed mine clearance. A prerequisite for AUVs to perform underwater tasks is a high-precision navigation and positioning system. To achieve this, researchers both domestically and internationally have conducted extensive research. Inertial navigation and positioning systems, underwater acoustic positioning systems, and SLAM (Simultaneous Localization and Mapping) technology have all provided solutions to the underwater navigation problem of AVUs.

[0003] Visual SLAM is a navigation and positioning system based on image information, and image quality directly affects the proper functioning of the entire visual SLAM algorithm. When light propagates in water, due to the selective absorption of light by water, different wavelengths of light exhibit varying degrees of attenuation. Combined with water turbidity and suspended organic particles, underwater images often suffer from severe color distortion, displaying a blue-green hue. Furthermore, the scattering of light by fine particles in the water reduces the contrast of the captured image, blurring details and increasing the difficulty of feature extraction. This significantly impacts the feature extraction part of the visual SLAM algorithm. Directly extracting ORB (Oriented Fast and Rotated BRIEF) features from the raw underwater image yields a very small number, leading to decreased accuracy or inability to calculate the transformation matrix between subsequent image frames. Ultimately, this increases the navigation error of the entire visual SLAM algorithm or renders it inoperable. Summary of the Invention

[0004] The purpose of this application is to provide an image enhancement method, device, and medium for underwater visual navigation and mapping, which can quickly improve the quality of the original underwater image, increase the number of feature points in the image, meet the requirements of the visual SLAM algorithm for front-end features, and ensure that underwater visual navigation and mapping can be carried out smoothly.

[0005] To address the aforementioned technical problems, embodiments of this application provide an image enhancement method for underwater visual navigation and mapping, comprising the following steps: acquiring several sample underwater images and their corresponding reference images to generate a training set and a validation set; training a pre-constructed generator and discriminator based on the training set and a preset loss function to obtain a trained generator; wherein the pre-constructed generator includes a hybrid attention module, which focuses on and extracts important features including color and texture features, and the loss function includes an adversarial loss term, an L1 loss term, a color bias loss term, and a gradient loss term; inputting each sample underwater image from the validation set into the trained generator, adjusting each hyperparameter in the trained generator according to the output of the trained generator and the reference image corresponding to each sample underwater image to obtain an enhancement model; inputting the underwater image to be enhanced into the enhancement model to obtain the enhanced image for visual SLAM.

[0006] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described image enhancement method for underwater visual navigation and mapping.

[0007] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image enhancement method for underwater visual navigation and mapping.

[0008] The image enhancement method, device, and medium for underwater visual navigation and mapping provided in this application are based on generative adversarial network (GAN) technology. A generator and discriminator are trained, and an enhancement model is obtained based on the trained generator. Considering that underwater images are mainly plagued by color distortion and blurred details, this application incorporates a hybrid attention module in the generator. This module enables the generator to focus on important features, including color and texture features. Thus, the trained enhancement model acquires the ability to achieve color balance and detail extraction. Using this enhancement model to process underwater images can quickly improve the quality of the original underwater image, increase the number of feature points in the image, meet the requirements of visual SLAM algorithms for front-end features, and ensure smooth underwater visual navigation and mapping. Simultaneously, the model training process includes two main parts: training and validation. Training enables the model to acquire basic image enhancement capabilities, while validation allows for fine-tuning of the model's hyperparameters to optimize the image enhancement effect. During training, loss functions including adversarial loss, L1 loss, color bias loss, and gradient loss are used. This improves the training efficiency of the entire model while paying more attention to color and texture information, making the enhanced model well adapted to underwater scenes.

[0009] In some optional embodiments, both the sample underwater image and the underwater image to be enhanced are RGB images. The pre-built generator consists of several two-dimensional convolutional blocks, several transposed convolutional blocks, and several hybrid attention modules. Each hybrid attention module includes a channel attention module and a spatial attention module. The two-dimensional convolutional blocks are used to downsample, extract features, and reduce the size of their own input feature maps. The transposed convolutional blocks are used to upsample and enlarge the size of their own input feature maps. The channel attention module is used to determine the weight values ​​of each channel of its own input feature map, so that the generator focuses on the features of the channels with large weight values. The spatial attention module is used to divide the features of each channel into important features and secondary features according to the average value of each channel of its own input feature map, so that the generator focuses on the important features. The hybrid attention module mainly consists of a channel attention module and a spatial attention module. The channel attention module is responsible for enhancing the features of important channels and suppressing the features of secondary channels. Since the input images are all RGB three-channel underwater images, channel attention is also color attention, which can enhance the red and orange tones and suppress the blue and green tones, effectively solving the color distortion problem. The spatial attention module is responsible for making the generator (enhanced model) pay more attention to the high-texture parts of the image, which can enrich the details of the underwater image and effectively solve the problem of image blurring.

[0010] In some optional embodiments, the channel attention module determines the weight values ​​of each channel of its input feature map through the following steps, so that the generator focuses on the features of the channels with large weight values: It passes its input feature map through an average pooling layer and a max pooling layer to obtain an average feature tensor and a maximum feature tensor, respectively, and then multiplies them by a first learnable parameter and a second learnable parameter to obtain a first intermediate tensor and a second intermediate tensor; it multiplies the average feature tensor and the maximum feature tensor by a first preset coefficient and then adds them element-wise to obtain a third intermediate tensor; it then adds the first intermediate tensor, the second intermediate tensor, and the third intermediate tensor element-wise to obtain a channel tensor; it performs dimensionality reduction on the channel tensor through a first nonlinear operation and then performs dimensionality increase through a convolutional layer to obtain a processed channel tensor; it normalizes the processed channel tensor to obtain a channel weight tensor, and then multiplies the channel weight tensor element-wise with the input feature map of the channel attention module to obtain an output feature map corresponding to the input feature map of the channel attention module. The channel attention module mainly consists of three parts: adaptive selection operation, compression operation, and fusion operation. It uses max pooling and average pooling to aggregate the information of the feature map, so that the two pooling results have different weights to obtain different weights for the channels, thereby enabling the generator (enhancement model) to focus on the features of the channels with large weight values.

[0011] In some optional embodiments, the channel tensor is reduced in dimension by a first nonlinear operation and then increased in dimension by a convolutional layer to obtain the processed channel tensor, which is achieved by the following formula:

[0012]

[0013] Where Φ1(·) represents the first nonlinear operation, by It consists of convolutional layers, batch normalization (BN) layers, and ReLU activation functions in that order. This indicates that the number of output channels is d and the number of input channels is C. in 1×1 two-dimensional convolution, This indicates that the number of output channels is C. in And the input channel is a 1×1 two-dimensional convolution with d channels, where δ(·) represents the ReLU activation function, f add Let f represent the channel tensor. S Let f represent the reduced-dimensional channel tensor. Z This represents the processed channel tensor;

[0014] The processed channel tensor is normalized to obtain a channel weight tensor. This channel weight tensor is then multiplied element-wise with the input feature map of the channel attention module to obtain an output feature map corresponding to the input feature map of the channel attention module. This is achieved using the following formula:

[0015]

[0016] Wherein, X1 represents the input feature map of the channel attention module itself. This represents the element-wise multiplication, σ(·) represents the softmax activation function, and f CA Y1 represents the channel weight tensor, and Y1 represents the output feature map corresponding to the input feature map of the channel attention module itself.

[0017] In some optional embodiments, the spatial attention module divides the features of each channel into important features and minor features based on the average value of each channel of its input feature map, thereby enabling the generator to focus on the important features: Adaptive average pooling is performed on each channel of its input feature map to obtain the average pixel value corresponding to each channel; for each channel, features with pixel values ​​greater than the average value are identified as important features, and features with pixel values ​​less than or equal to the average value are identified as minor features; an important tensor with the same size as the input feature map of the spatial attention module is generated based on each of the important features, and a minor tensor with the same size as the input feature map of the spatial attention module is generated based on each of the minor features; the important tensor and the minor tensor are multiplied pixel-by-pixel with the input feature map of the spatial attention module to obtain the important feature map and the minor feature map. The process involves: First, performing average pooling and max pooling on the important feature maps and summing them pixel-by-pixel to obtain a first feature map. Then, performing average pooling and max pooling on the secondary feature maps and summing them pixel-by-pixel to obtain a second feature map. Finally, performing a shared convolution on the first and second feature maps yields a fourth and a fifth intermediate tensor. Next, performing a second nonlinear operation on the fourth and fifth intermediate tensors yields a first and a second spatial attention tensor. Then, multiplying the first spatial attention tensor pixel-by-pixel with the important feature maps yields a first spatially weighted feature map. Finally, multiplying the first and second spatially weighted feature maps pixel-by-pixel yields a second spatially weighted feature map. Finally, summing the first and second spatially weighted feature maps pixel-by-pixel yields an output feature map corresponding to the input feature map of the spatial attention module itself. The spatial attention module mainly consists of three parts: feature separation, selection, and fusion operations. It divides the features in each channel into important and secondary features, and then applies average pooling and max pooling operations along the channel dimension. The two pooling results are then subjected to a series of convolutions to obtain the first spatial weighted feature map and the second spatial feature map, and finally obtain the spatial attention map. In this process, the generator (enhanced model) can pay more attention to important features.

[0018] In some optional embodiments, the important feature maps are subjected to average pooling and max pooling respectively, and then summed pixel by pixel to obtain a first feature map. The secondary feature maps are subjected to average pooling and max pooling respectively, and then summed pixel by pixel to obtain a second feature map. The first feature map and the second feature map are then subjected to a shared convolution to obtain a fourth intermediate tensor and a fifth intermediate tensor. A second nonlinear operation is then performed on the fourth intermediate tensor and the fifth intermediate tensor respectively to obtain a first spatial attention tensor and a second spatial attention tensor, which is achieved by the following formula:

[0019]

[0020]

[0021] Where F1 represents the important feature map, F2 represents the minor feature map, AvgPool(·) represents the average pooling, and MaxPool(·) represents the max pooling. This represents the first feature map. This represents the second feature map. This represents a 7×7 two-dimensional convolution with 1 output channel and 1 input channel. This represents the fourth intermediate tensor. The fifth intermediate tensor is represented by Φ2(·), which represents the second nonlinear operation, consisting of a BN layer and a ReLU activation function in sequence. SA1 Let f represent the first spatial attention tensor. SA2 This represents the second spatial attention tensor;

[0022] The process involves multiplying the first spatial attention tensor pixel-by-pixel with the important feature map to obtain a first spatially weighted feature map, and multiplying the second spatial attention tensor pixel-by-pixel with the secondary feature map to obtain a second spatially weighted feature map. Then, the first spatially weighted feature map and the second spatial feature map are summed pixel-by-pixel to obtain an output feature map corresponding to the input feature map of the spatial attention module itself. This is achieved through the following formula:

[0023]

[0024] Among them, F S1 F represents the first spatially weighted feature map. S2 This represents the second weighted spatial feature map. This indicates that the pixel-by-pixel addition is performed. Y1 represents the element-wise multiplication, and Y2 represents the output feature map corresponding to the input feature map of the spatial attention module itself.

[0025] In some optional embodiments, the discriminator includes a 3×3 max pooling branch, a 1×1 convolutional branch, a 3×3 convolutional branch, a 5×5 convolutional branch, and a 3×3 average pooling branch. Each branch is followed by a 3×3 dilated convolution. The dilation coefficients of the 3×3 dilated convolutions at the ends of different branches are different. Each 3×3 dilated convolution is connected to the same 1×1 aggregated convolution. The 1×1 aggregated convolution is then connected to a Leaky pooling algorithm. The ReLU activation function layer; the 3×3 max pooling branch, the 1×1 convolution branch, the 3×3 convolution branch, and the 5×5 convolution branch are used to increase the discriminator's adaptability to different scales; the 1×1 convolution branch is also used to limit the number of channels in the discriminator; the 3×3 average pooling branch is used to enhance the discriminator's learning of background features; the 3×3 dilated convolution with different dilation coefficients is used to expand the receptive field; the 1×1 aggregation convolution is used to aggregate the information from each branch and restore the number of channels to be the same as the input of the discriminator. The main function of the discriminator is to distinguish between real and fake enhanced images obtained by the generator. The size of the receptive field can reflect the model's utilization rate of contextual information. In order to reduce the loss of contextual information between different sub-regions, this application sets up diverse branches with multi-scale receptive fields in the discriminator, which greatly improves the discriminator's discrimination ability.

[0026] In some optional embodiments, for the generator, the preset loss function is expressed by the following formula:

[0027] L G =L adv +λ g1 L1+λ g2 L col +λ g3 L gra

[0028] L adv =E(y){log[D(y)]}+E(x){log{1-D[G(x)]}}

[0029] L1 = E x，y [||yG(x)||1]

[0030] L col =DCh(y) / M(y)-DCh(x) / M(x)

[0031]

[0032] Among them, L G Let λ be the loss function for the generator. g1 , λ g2 , λ g3L represents the preset weighting coefficients for the loss term. adv Let L1 be the adversarial loss term, and L1 be the L1 loss term. col For the color shift loss term, L gra Let G(·) represent the gradient loss term, G(·) represent the generator, D(·) represent the discriminator, E(·) represent the expected value, ||·||1 represent the L1 norm, DCh(·) represent the average color difference function, and M(·) represent the center distance function. This represents the gradient operator in the horizontal direction. This represents the gradient operator in the vertical direction. Attached Figure Description

[0033] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0034] Figure 1 This is a flowchart of an image enhancement method for underwater visual navigation and mapping provided in one embodiment of this application;

[0035] Figure 2 This is a schematic diagram of a generative adversarial network provided in one embodiment of this application;

[0036] Figure 3 This is a schematic diagram of the network architecture of a generator provided in one embodiment of this application;

[0037] Figure 4 This is a schematic diagram of the structure of a hybrid attention module provided in one embodiment of this application;

[0038] Figure 5 This is a flowchart illustrating the working principle of the channel attention module, provided in one embodiment of this application.

[0039] Figure 6 This is a schematic diagram of the structure of a channel attention module provided in one embodiment of this application;

[0040] Figure 7 This is a flowchart illustrating the working principle of the spatial attention module, provided in one embodiment of this application.

[0041] Figure 8 This is a schematic diagram of the structure of a spatial attention module provided in one embodiment of this application;

[0042] Figure 9 This is a schematic diagram of the network architecture of a discriminator provided in one embodiment of this application;

[0043] Figure 10This is an embodiment of the present application, showing the effect of different image enhancement methods on a test set;

[0044] Figure 11 This is an embodiment of the present application, showing ORB feature detection results for different image enhancement methods;

[0045] Figure 12 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0047] One embodiment of this application relates to an image enhancement method for underwater visual navigation and mapping, applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the image enhancement method for underwater visual navigation and mapping in this embodiment are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution.

[0048] The specific process of the image enhancement method for underwater visual navigation and mapping provided in this embodiment can be as follows: Figure 1 As shown, it includes:

[0049] Step 101: Obtain several sample underwater images and their corresponding reference images to generate a training set and a validation set.

[0050] In the specific implementation, three types of datasets are used in the network model learning process: training set, validation set, and test set. The server pre-acquires several sample underwater images and their corresponding reference images, generating training and validation sets according to a preset ratio. Both the training and validation sets consist of pairs of sample underwater images (raw values) and their corresponding reference images (reference values). The training set is used to train the model, enabling it to discover the relationship between the raw and reference values, thus determining the initial network model. The validation set assumes the reference values ​​are unknown. The trained model is applied to the validation set, predicting the reference values ​​based on the known raw values, and comparing these predictions with the true reference values ​​to adjust the model's hyperparameters, ultimately determining the final network model. The test set only includes underwater images and is used to evaluate the model's performance.

[0051] In one example, the server can select 10,500 underwater images from the EUVP dataset based on factors such as color, contrast, sharpness, and content diversity. At the same time, to increase the network's generalization ability, 1,500 underwater images of various types can be selected from the Internet. The above 12,000 images are combined to form the final dataset and divided in a 6:2:2 ratio, as shown in Table 1.

[0052] Table 1: Specific composition of training set, validation set and test set

[0053]

[0054] Step 102: Train the pre-built generator and discriminator based on the training set and the preset loss function to obtain the trained generator. The pre-built generator is equipped with a hybrid attention module, which is used to focus on and extract important features, including color features and texture features.

[0055] In its implementation, the server trains a pre-built generator and discriminator based on a training set and a preset loss function, resulting in a trained generator and discriminator. This is equivalent to training the generative adversarial network (GAN) using the training set and the preset loss function. The GAN consists of two parts: a generator and a discriminator. During training, sample underwater images from the training set serve as input to the generator, which outputs enhanced images corresponding to the sample underwater images. Simultaneously, reference images corresponding to the sample underwater images and the enhanced images output by the generator serve as input to the discriminator, which then distinguishes between genuine and fake enhanced images corresponding to the sample underwater images output by the generator.

[0056] In one example, the sample underwater image has dimensions of 3×480×640, where 3 represents the number of image channels (C), 480 represents the image height (H), and 640 represents the image width (W). The enhanced image corresponding to the sample underwater image output by the generator also has dimensions of 3×480×640. Since the sample underwater image is an RGB image, the three channels are, in order: R channel, G channel, and B channel.

[0057] In one example, a generative adversarial network can be like... Figure 2 As shown, a Generative Adversarial Network (GAN) consists of a generator G and a discriminator D, which optimize the network parameters through adversarial interaction. GANs are characterized by their independence from any prior assumptions and the simplicity of generating real-like samples, making them less demanding on datasets and better suited for real-time requirements. Figure 2 In the diagram, z represents the input random noise, signifying the sample underwater image; x' represents the generated data obtained by generator G from the input random noise z, signifying the enhanced image corresponding to the sample underwater image; x represents the real data, signifying the reference image corresponding to the sample underwater image; and y represents the discrimination result, with a value range of [0, 1]. The generative adversarial network (GAN) inputs random noise z into generator G, which transforms the random noise z into generated data x'. Based on the generated result x', generator G is trained to have the ability to deceive the discriminator D. Real data x and generated data x' are input into discriminator D, and discriminator D is trained based on the discrimination result y to have the ability to distinguish between real and fake data.

[0058] In the specific implementation, the pre-built generator includes a hybrid attention module, which focuses on and extracts important features, including color and texture features. Considering that underwater images are mainly plagued by color distortion and blurred details, this embodiment incorporates a hybrid attention module in the generator, thus enabling the trained augmented model to achieve color balancing and detail extraction capabilities.

[0059] In one example, the sample underwater image is an RGB image. The pre-built generator consists of several 2D convolutional blocks (Conv2d), several transposed convolutional blocks (ConvTranspose2d), and several Hybrid Attention Modules (HAM). Each HAM includes a Channel Attention Module (CAM) and a Space Attention Module (SAM). The 2D convolutional blocks downsample, extract features, and reduce the size of their input feature maps; the transposed convolutional blocks upsample and enlarge their input feature maps; the Channel Attention Module determines the weight values ​​of each channel in its input feature map, causing the generator to focus on the features of channels with large weight values; the Space Attention Module divides the features of each channel into important and minor features based on the average value of each channel in its input feature map, causing the generator to focus on important features. The hybrid attention module mainly consists of a channel attention module and a spatial attention module. The channel attention module is responsible for enhancing important channel features and suppressing secondary channel features. Since the input images are all RGB three-channel underwater images, channel attention, also known as color attention, can enhance red and orange tones and suppress blue and green tones, effectively solving the color distortion problem. The spatial attention module is responsible for enabling the generator (enhancement model) to pay more attention to the high-texture parts of the image, thus enriching the details of the underwater image and effectively solving the image blur problem.

[0060] In one example, a generator can be like... Figure 3 As shown, there are 10 layers in total, including layer m1, layer d1, layer d2, layer d3, layer d4, layer μ1, layer μ2, layer μ3, layer μ4, and layer m2. The specific details of each layer are shown in Table 2.

[0061] Table 2: Specific details of each layer of the generator

[0062]

[0063] Note: In Table 2, k is the convolution size, s is the compensation, and D is the padding.

[0064] In its implementation, the pre-defined loss function is obtained by linearly weighting the adversarial loss, L1 loss, color bias loss, and gradient loss. For the generator, the pre-defined loss function is expressed by the following formula:

[0065] L G =L adv +λ g1 L1+λ g2 L col+λ g3 L gra

[0066] L adv =E(y){log[D(y)]}+E(x){log{1-D[G(x)]}}

[0067] L1 = E x，y [||yG(x)||1]

[0068] L col =DCh(y) / M(y)-DCh(x) / M(x)

[0069]

[0070] In the formula, L G Let λ be the loss function for the generator. g1 , λ g2 , λ g3 L represents the preset weighting coefficients for the loss term. adv To counteract the loss term, L1 is the L1 loss term, L col For color bias loss, L gra Here, G(·) represents the gradient loss term, G(·) represents the generator, D(·) represents the discriminator, E(·) represents the expected value, ||·||1 represents the L1 norm, DCh(·) represents the average color difference function, and M(·) represents the center distance function. This represents the gradient operator in the horizontal direction. This represents the gradient operator in the vertical direction. It's worth noting that the pre-defined loss function for the discriminator is similar to that for the generator, and will not be elaborated upon here.

[0071] In one example, λ g1 , λ g2 , λ g3 They were set to 0.3, 0.3, and 0.4 respectively.

[0072] Step 103: Input the underwater images of each sample in the validation set into the trained generator. Based on the output of the trained generator and the reference image corresponding to each sample underwater image, adjust the hyperparameters in the trained generator to obtain the enhanced model.

[0073] Step 104: Input the underwater image to be enhanced into the enhancement model to obtain the enhanced image for visual SLAM.

[0074] In the specific implementation, after the server obtains the trained generator, it no longer needs to consider the discriminator. It directly inputs the underwater images of each sample in the validation set into the trained generator. Based on the output of the trained generator and the reference image corresponding to each sample underwater image, the hyperparameters in the trained generator are adjusted to obtain the augmentation model. After obtaining the augmentation model, the underwater image to be augmented can be input into the augmentation model to obtain the augmented image for visual SLAM.

[0075] In this embodiment, a generator and discriminator are trained based on generative adversarial network (GAN) technology. An augmentation model is obtained based on the trained generator. Considering that underwater images are mainly plagued by color distortion and blurred details, this application incorporates a hybrid attention module in the generator. This module enables the generator to focus on important features, including color and texture features. Thus, the trained augmentation model acquires the ability to achieve color balance and detail extraction. Using this model to process underwater images can quickly improve the quality of the original underwater images, increase the number of feature points in the images, meet the requirements of visual SLAM algorithms for front-end features, and ensure smooth underwater visual navigation and mapping. The model training process includes two main parts: training and validation. Training enables the model to acquire basic image augmentation capabilities, while validation allows for fine-tuning of the model's hyperparameters to optimize the image augmentation effect. During training, a loss function including adversarial loss, L1 loss, color bias loss, and gradient loss is used. This improves the overall training efficiency of the model while paying more attention to color and texture information, making the augmentation model highly adaptable to underwater scenes.

[0076] In one embodiment, the structure of the hybrid attention module can be as follows: Figure 4 As shown, the input of the hybrid attention module enters the CAM. The input of the hybrid attention module and the output of the CAM are added pixel by pixel and then input into the SAM. The output of the CAM, the output of the SAM, and the input of the SAM are concatenated along the channels and fed into a 1×1 convolutional + ReLU activation function layer, finally obtaining the output of the hybrid attention module. The channel attention module is implemented through... Figure 5 The steps shown determine the weight values ​​of each channel of the input feature map, causing the generator to focus on the features of channels with large weight values. Specifically, this includes:

[0077] Step 201: Pass the input feature map through the average pooling layer and the max pooling layer respectively to obtain the average feature tensor and the max feature tensor, and then multiply them by the first learnable parameter and the second learnable parameter respectively to obtain the first intermediate tensor and the second intermediate tensor.

[0078] Step 202: Multiply the average feature tensor and the maximum feature tensor by the first preset coefficient and then add them element by element to obtain the third intermediate tensor. Then add the first intermediate tensor, the second intermediate tensor and the third intermediate tensor element by element to obtain the channel tensor.

[0079] In its specific implementation, the channel attention module is structured as follows: Figure 6 As shown, let the input feature map of the channel attention module itself be denoted as . The channel attention module first performs two pooling operations on X1: passing X1 through an average pooling layer and a max pooling layer, respectively, to obtain the average feature tensor f. avg and the largest eigentensor f max Then f avg and f max Multiplying by the first learnable parameter α and the second learnable parameter β respectively, we obtain the first intermediate tensor f. m1 Second intermediate tensor f m2 The channel attention module then applies f avg and f max Each element is multiplied by a first preset coefficient and then summed element by element to obtain the third intermediate tensor f. m3 and for f m1 f m2 and f m3 By performing element-wise addition, we obtain the channel tensor. This part is called the adaptive selection operation, which can be expressed by the following formula (the first preset coefficient is 0.5):

[0080]

[0081] In the formula, This indicates element-wise multiplication. This indicates that each pixel is added together.

[0082] Step 203: The channel tensor is reduced in dimension by the first nonlinear operation and then increased in dimension by a convolutional layer to obtain the processed channel tensor.

[0083] In its implementation, after the channel attention module completes the adaptive selection operation, it can perform compression. This compression operation, involving two steps—dimensionality reduction and dimensionality increase—achieves cross-channel information interaction, reducing computational load and improving learning efficiency. First, for f... add The channel tensor f is obtained by performing dimensionality reduction through the first nonlinear operation Φ1. S ∈R a×1×1 Then f S The channel tensor is obtained by dimensionality increase through convolutional layers. Compression can be expressed by the following formula:

[0084]

[0085] In the formula, Φ1(·) represents the first nonlinear operation, derived from... It consists of convolutional layers, batch normalization (BN) layers, and ReLU activation functions in that order. This indicates that the number of output channels is d and the number of input channels is C. in 1×1 two-dimensional convolution, This indicates that the number of output channels is C. in And the input channel is a 1×1 two-dimensional convolution, where δ(·) represents the ReLU activation function, f add Let f represent the channel tensor. S Let f represent the reduced-dimensional channel tensor. Z The processed channel tensor d can be controlled using the hyperparameter r of the channel attention module.

[0086]

[0087] In the formula, L represents the minimum value of d, which is usually taken as 32.

[0088] Step 204: Normalize the processed channel tensor to obtain the channel weight tensor, and multiply the channel weight tensor element-wise with the input feature map of the channel attention module itself to obtain the output feature map corresponding to the input feature map of the channel attention module itself.

[0089] In its implementation, after performing compression, the channel attention module can perform a fusion operation on f. X Normalization is performed to obtain the channel weight tensor. and f CA Element-wise multiplication with X1 yields the output feature map Y1 corresponding to X1. The fusion operation of the channel attention module can be expressed by the following formula:

[0090]

[0091] In the formula, X1 represents the input feature map of the channel attention module itself. σ(·) represents element-wise multiplication, and f represents the softmax activation function. CA Y1 represents the channel weight tensor, while Y1 represents the output feature map corresponding to the input feature map of the channel attention module itself.

[0092] In this embodiment, the channel attention module mainly consists of three parts: adaptive selection operation, compression operation, and fusion operation. It uses max pooling and average pooling to aggregate the information of the feature map, so that the two pooling results have different weights to obtain different weights for the channels, thereby enabling the generator (enhancement model) to focus on the features of the channels with large weight values.

[0093] In one embodiment, the spatial attention module can be used as follows: Figure 7 Each step, as shown, divides the features of each channel into important and secondary features based on the average value of each channel of its input feature map, so that the generator focuses on the important features. Specifically, this includes:

[0094] Step 301: Perform adaptive average pooling on each channel of the input feature map to obtain the average value of the pixel value corresponding to each channel. For each channel, the features with pixel values ​​greater than the average value are identified as important features, and the features with pixel values ​​less than or equal to the average value are identified as secondary features.

[0095] Step 302: Generate important tensors with the same size as the input feature map of the spatial attention module based on each important feature, and generate secondary tensors with the same size as the input feature map of the spatial attention module based on each secondary feature.

[0096] Step 303: Multiply the important tensor and the minor tensor pixel by pixel with the input feature map of the spatial attention module to obtain the important feature map and the minor feature map.

[0097] In a concrete implementation, the structure of the spatial attention module can be as follows: Figure 8 As shown, it first performs a feature separation operation, and the input feature map of the spatial attention module itself is denoted as... The spatial attention module performs adaptive average pooling on each channel of X2 to obtain the average pixel value for each channel. For each channel, features with pixel values ​​greater than the average are identified as important features, while features with pixel values ​​less than or equal to the average are identified as minor features. Subsequently, the spatial attention module defines two dimensions... The feature separation results are stored in tensors. Specifically, important tensors of the same size as X2 are generated based on each important feature, and secondary tensors of the same size as X2 are generated based on each minor feature. In the important tensor, important features are assigned a value of 1, and minor features are assigned a value of 0. In the secondary tensor, important features are assigned a value of 0, and minor features are assigned a value of 1. Finally, the important and secondary tensors are multiplied pixel-by-pixel by X2 to obtain the important feature map F1 and the secondary feature map F2.

[0098] Step 304: Perform average pooling and max pooling on the important feature maps respectively and add them pixel by pixel to obtain the first feature map. Perform average pooling and max pooling on the secondary feature maps respectively and add them pixel by pixel to obtain the second feature map. Perform a shared convolution on the first feature map and the second feature map to obtain the fourth intermediate tensor and the fifth intermediate tensor.

[0099] Step 305: Perform a second nonlinear operation on the fourth intermediate tensor and the fifth intermediate tensor respectively to obtain the first spatial attention tensor and the second spatial attention tensor.

[0100] In its implementation, after the spatial attention module completes the feature separation operation, it can then perform the selection operation. The spatial attention module performs average pooling and max pooling on the F1 algorithm and adds them pixel-by-pixel to obtain the first feature map. The second feature map is obtained by performing average pooling and max pooling on F2 and then summing the results pixel by pixel. Again and Perform shared convolution to obtain the fourth intermediate tensor. and the fifth intermediate tensor Spatial attention modules respectively and Perform the second nonlinear operation Φ2 to obtain the first spatial attention tensor f. SA1 Second space attention tensor f SA2 The selection operation can be represented by the following formula:

[0101]

[0102]

[0103] In the formula, F1 represents the important feature map, F2 represents the minor feature map, AvgPool(·) represents average pooling, and MaxPool(·) represents max pooling. Represents the first feature map. This represents the second feature map. This represents a 7×7 two-dimensional convolution with 1 output channel and 1 input channel. This represents the fourth intermediate tensor. Φ2(·) represents the fifth intermediate tensor, and Φ2(·) represents the second nonlinear operation, which consists of a BN layer and a ReLU activation function in sequence. SA1 Let f represent the first-space attention tensor. SA2 This represents the attention tensor of the second space.

[0104] Step 306: Multiply the first spatial attention tensor with the important feature map pixel by pixel to obtain the first spatial weighted feature map, and multiply the second spatial attention tensor with the minor feature map pixel by pixel to obtain the second spatial weighted feature map.

[0105] Step 307: The first spatial weighted feature map and the second spatial weighted feature map are added pixel by pixel to obtain the output feature map corresponding to the input feature map of the spatial attention module itself.

[0106] In its implementation, after the spatial attention module completes the selection operation, it can perform the fusion operation. The spatial attention module will then perform the f... SA1 Multiplying F1 pixel by pixel yields the first spatially weighted feature map F. S1 and f SA2 Multiplying F2 pixel by pixel yields the second weighted spatial feature map F. S2 Finally, regarding F... S1 and F S2 Pixel-by-pixel summation is performed to obtain the output feature map Y2 corresponding to X2. The fusion operation of the spatial attention module can be implemented using the following formula:

[0107]

[0108] In the formula, F S1 F represents the first spatial weighted feature map. S2 This represents the second weighted spatial feature map. This indicates pixel-by-pixel addition. X2 represents element-wise multiplication, and F2 represents the output feature map corresponding to X2.

[0109] In this embodiment, the spatial attention module mainly consists of three parts: feature separation operation, selection operation, and fusion operation. First, the features in each channel are divided into important features and secondary features. Then, average pooling and max pooling operations are applied along the channel dimension. The results of the two pooling operations are then subjected to a series of convolutions to obtain the first spatial weighted feature map and the second spatial feature map, and finally, the spatial attention map is obtained. In this process, the generator (enhanced model) can pay more attention to important features and reduce attention to secondary features.

[0110] In one embodiment, the main function of the discriminator is to distinguish between real and fake enhanced images obtained by the generator. The size of the receptive field reflects the model's utilization of contextual information. To reduce the loss of contextual information between different sub-regions, this embodiment, based on the Markov discriminator, incorporates diverse branches with multi-scale receptive fields, significantly improving the discriminator's discrimination capability. The discriminator's network architecture can be as follows: Figure 9As shown, the discriminator includes a 3×3 max pooling branch, a 1×1 convolution branch, a 3×3 convolution branch, a 5×5 convolution branch, and a 3×3 average pooling branch. Each branch is followed by a 3×3 dilated convolution. The dilation coefficients of the 3×3 dilated convolutions at the back end of different branches vary. Each 3×3 dilated convolution is connected to the same 1×1 aggregated convolution, which is then connected to a Leaky ReLU activation function layer. The 3×3 max pooling, 1×1, 3×3, and 5×5 convolution branches are used to increase the discriminator's adaptability to different scales. The 1×1 convolution branch also limits the number of channels in the discriminator. The 3×3 average pooling branch enhances the discriminator's learning of background features. The 3×3 dilated convolutions with different dilation coefficients are used to expand the receptive field. The 1×1 aggregated convolution aggregates the information from each branch and restores the number of channels to be the same as the discriminator's input.

[0111] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0112] In one embodiment, to verify the image enhancement effect of the proposed image enhancement method for underwater visual navigation and mapping, underwater image enhancement experiments were conducted on various image enhancement methods. The proposed image enhancement method for underwater visual navigation and mapping is developed and implemented based on the PyTorch framework. Considering observability and computational complexity, and to maintain consistency with the SLAM used in subsequent experiments, the image size was uniformly set to 3×480×640. The experimental configuration for the underwater image enhancement experiments is shown in Table 3, and the experimental parameter settings are shown in Table 4.

[0113] Table 3: Experimental configuration for underwater image enhancement experiments

[0114] processor AMD Ryzen 7 5800H with Radeon Graphics@3.20GHz Memory 8G CPU NVIDIA GeForce RTX 3060 programming language Python 3.6 operating system Ubuntu 18.04

[0115] Table 4: Experimental parameter settings for underwater image enhancement experiments

[0116] Optimizer Adam Exponential decay factor 0.9 Learning rate 0.0001 Batch processing data volume 16 Number of iterations 200

[0117] The EUVP dataset, proposed by Islam et al. in 2020, consists of three paired datasets and one unpaired dataset, as shown in Tables 5 and 6. Most of the underwater images in the dataset were captured using seven different camera devices (GoPros, uEye, and low-light USB cameras, etc.) in various water conditions across different sea areas, with a small portion extracted from video websites. These images contain a wealth of content, including common underwater life (sea cucumbers, corals, and schools of fish, etc.), and are representative of the vast majority of underwater environments.

[0118] Table 5: EUVP Paired Datasets

[0119] Underwater Dark 5550 570 11670 Underwater ImageNet 3700 1270 8670 Underwater Scenes 2185 130 4500

[0120] Table 6: EUVP Unpaired Dataset

[0121] 3195 3140 330 6665

[0122] from Figure 10 As can be seen, RayleighDistribution enhances the image brightness well, but introduces a red color cast, exacerbating the color imbalance. RGHS enhances the image contrast to some extent, but its color recovery is poor, still exhibiting a strong underwater style. ULAP enhances the image with blurred details and introduces a yellow color cast. FunieGAN and Shallow-Uwnet enhance the image tones are similar, with varying degrees of yellow tint overall, and insufficient image clarity, resulting in detail loss. WaterGAN and TOPAL do not effectively remove color casts when processing pure green background images, but perform well when processing other background images. The image enhancement method proposed in this application for underwater visual navigation and mapping, while restoring normal image colors, also demonstrates excellent performance in improving clarity, contrast, and brightness, without any significant shortcomings.

[0123] To objectively evaluate and analyze the performance of the enhancement model proposed in this application, Underwater Image Quality Assessment (UIQM) is selected as the evaluation index of this invention, with a higher value being better. UIQM is specifically composed of a linear combination of three metrics: Color Measurement Index (UICM), Sharpness Measurement Index (UISM), and Contrast Measurement Index (UIConM), as detailed below:

[0124] UIQM=c1×UICM+c2×UISM+c3×UIConM

[0125] In the formula, c1, c2 and c3 are scaling factors, which can be set to 0.0282, 0.2953 and 3.5753 respectively.

[0126] The average scores of the proposed image enhancement method (Ours) for underwater visual navigation and mapping, along with other image enhancement methods, on the test set (UICM, UISM, UIConM, and UIQM) are shown in Table 7. Among traditional methods, RayleighDistribution scored the highest, but the enhanced image had an abnormally high proportion of red. Among deep learning methods, TOPAL scored slightly lower than the proposed image enhancement method for underwater visual navigation and mapping, but its score was lowered due to the introduction of yellow color shift in some images. The proposed image enhancement method for underwater visual navigation and mapping, by mixing attention modules and adding loss terms, achieves higher contrast, brightness, and clearer detail in the enhanced image, and the overall color of the image achieves a good balance. Therefore, it achieved the highest score, demonstrating the effectiveness and generalization of the proposed image enhancement method for underwater visual navigation and mapping.

[0127] Table 7: Scores of each image enhancement method on the test set

[0128]

[0129] In one embodiment, to verify the effectiveness of the image enhancement method proposed in this application for underwater visual navigation and mapping in extracting ORB features, visual SLAM application experiments were conducted on various image enhancement methods.

[0130] The purpose of image enhancement is to provide high-quality input images for underwater visual SLAM, making it easier for the front-end and loop closure parts of visual SLAM to obtain image feature information. To maintain consistency with subsequent SLAM, the effectiveness of the image enhancement method is further demonstrated by extracting ORB features. The hardware configuration during the extraction process is the same as in Table 3, and will not be repeated here. Feature points are usually concentrated in texture-rich areas, while the number of feature points in texture-deficient areas is very small. This results in an uneven distribution of feature points in space, leading to a decrease in the accuracy of spatial geometric relationships obtained through subsequent feature matching. To prevent this problem, the image enhancement method for underwater visual navigation and mapping proposed in this application employs a feature point homogenization method when extracting ORB features to avoid excessive concentration of feature points. Feature point homogenization means that no other feature points (e.g., k=12) are allowed within k pixels surrounding a given feature point. Under the condition of feature point homogenization, Figure 11 The image enhancement method proposed in this application for underwater visual navigation and mapping and other image enhancement methods are presented, and the detection results of ORB features are given. Table 8 shows the time and number of ORB features detected by the image enhancement method (Ours) proposed in this application for underwater visual navigation and mapping and other image enhancement methods. The time refers to the total time for image enhancement and ORB feature detection.

[0131] Table 8: ORB feature detection time and number of methods for each image enhancement method

[0132] Original image 0.018 1 Rayleigh Distribution 0.919 79 RGHS 2.957 15 ULAP 2.061 10 WaterGAN 0.032 63 FunieGAN 0.034 7 Shallow-Uwnet 0.795 1 TOPAL 0.041 68 Ours 0.022 124

[0133] The requirements for ORB features in underwater visual SLAM are as follows: front-end feature matching should ideally involve no fewer than 70 feature points, and the processing time must be less than 0.03 seconds (i.e., 30 frames per second). Figure 11 As can be seen from Table 8, most image enhancement methods have a certain effect on increasing the number of feature points. Among them, RayleighDistribution and TOPAL basically meet the requirements for the number of feature points, but the processing time is not up to standard (RayleighDistribution requires stretching the histogram in both RGB and HSV color spaces, while TOPAL's network consists of multiple generators and encoders, which is relatively time-consuming). Only the image enhancement method for underwater visual navigation and mapping proposed in this application can ensure the normal operation of visual SLAM while meeting the requirements for the number of feature points and processing time.

[0134] Another embodiment of this application relates to an electronic device, such as... Figure 12 As shown, it includes: at least one processor 401; and a memory 402 communicatively connected to the at least one processor 401; wherein the memory 402 stores instructions executable by the at least one processor 401, the instructions being executed by the at least one processor 401 to enable the at least one processor 401 to perform the image enhancement method for underwater visual navigation and mapping in the above embodiments.

[0135] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0136] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0137] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0138] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. An image enhancement method for underwater visual navigation and mapping, characterized in that, include: Acquire several sample underwater images and their corresponding reference images to generate a training set and a validation set; The pre-built generator and discriminator are trained based on the training set and the preset loss function to obtain the trained generator; wherein, the pre-built generator is equipped with a hybrid attention module, which is used to focus on and extract important features including color features and texture features, and the loss function includes an adversarial loss term, an L1 loss term, a color bias loss term, and a gradient loss term. The underwater images of each sample in the validation set are input into the trained generator. Based on the output of the trained generator and the reference image corresponding to each sample underwater image, the hyperparameters in the trained generator are adjusted to obtain the enhanced model. The underwater image to be enhanced is input into the enhancement model to obtain the enhanced image for visual SLAM. Both the sample underwater image and the underwater image to be enhanced are RGB images. The pre-built generator consists of several two-dimensional convolutional blocks, several transposed convolutional blocks, and several hybrid attention modules. Each hybrid attention module includes a channel attention module and a spatial attention module. The two-dimensional convolutional block is used to downsample its own input feature map, extract features, and reduce its size; The transposed convolutional block is used to upsample and enlarge its own input feature map; The channel attention module is used to determine the weight values ​​of each channel of its input feature map, so that the generator focuses on the features of the channel with the larger weight value. The spatial attention module is used to determine, based on the average value of each channel of its input feature map, features with pixel values ​​greater than the average value in each channel as important features, and features with pixel values ​​less than or equal to the average value as secondary features, so that the generator focuses on the important features.

2. The image enhancement method for underwater visual navigation and mapping according to claim 1, characterized in that, The channel attention module determines the weight values ​​of each channel of its input feature map through the following steps, so that the generator focuses on the features of the channels with larger weight values: The input feature map is passed through the average pooling layer and the max pooling layer respectively to obtain the average feature tensor and the max feature tensor. Then, it is multiplied by the first learnable parameter and the second learnable parameter respectively to obtain the first intermediate tensor and the second intermediate tensor. The average feature tensor and the maximum feature tensor are each multiplied by a first preset coefficient and then added element by element to obtain a third intermediate tensor. The first intermediate tensor, the second intermediate tensor, and the third intermediate tensor are then added element by element to obtain a channel tensor. The channel tensor is reduced in dimension by a first nonlinear operation, and then increased in dimension by a convolutional layer to obtain the processed channel tensor. The processed channel tensor is normalized to obtain the channel weight tensor, and the channel weight tensor is multiplied element-wise with the input feature map of the channel attention module to obtain the output feature map corresponding to the input feature map of the channel attention module.

3. The image enhancement method for underwater visual navigation and mapping according to claim 2, characterized in that, The channel tensor is reduced in dimension by a first nonlinear operation, and then increased in dimension by a convolutional layer to obtain the processed channel tensor, which is achieved by the following formula: ； in, This indicates the first nonlinear operation, by Convolutional layers, BN layers and The activation functions are arranged in sequence. This indicates that the number of output channels is d and the number of input channels is d. 1×1 two-dimensional convolution, This indicates that the number of output channels is And the input channel is a 1×1 two-dimensional convolution. Indicates the Activation function This represents the channel tensor. This represents the channel tensor after dimensionality reduction. This represents the processed channel tensor; The processed channel tensor is normalized to obtain a channel weight tensor. This channel weight tensor is then multiplied element-wise with the input feature map of the channel attention module to obtain an output feature map corresponding to the input feature map of the channel attention module. This is achieved using the following formula: ； in, This represents the input feature map of the channel attention module itself. This indicates the element-wise multiplication. express Activation function This represents the channel weight tensor. This represents the output feature map corresponding to the input feature map of the channel attention module itself.

4. The image enhancement method for underwater visual navigation and mapping according to claim 1, characterized in that, The spatial attention module, through the following steps, determines features with pixel values ​​greater than the average value of each channel in its input feature map as important features, and features with pixel values ​​less than or equal to the average value as secondary features, thereby enabling the generator to focus on the important features: Adaptive average pooling is performed on each channel of the input feature map to obtain the average value of the pixel value corresponding to each channel. For each channel, the features whose pixel value is greater than the average value are identified as important features, and the features whose pixel value is less than or equal to the average value are identified as minor features. Based on each of the important features, an important tensor with the same size as the input feature map of the spatial attention module is generated, and based on each of the minor features, a minor tensor with the same size as the input feature map of the spatial attention module is generated. The important tensor and the minor tensor are multiplied element-wise with the input feature map of the spatial attention module to obtain the important feature map and the minor feature map, respectively. The important feature maps are subjected to average pooling and max pooling respectively, and then pixel-wise summed to obtain the first feature map. The secondary feature maps are subjected to average pooling and max pooling respectively, and then pixel-wise summed to obtain the second feature map. The first feature map and the second feature map are then subjected to a shared convolution to obtain the fourth intermediate tensor and the fifth intermediate tensor. A second nonlinear operation is performed on the fourth intermediate tensor and the fifth intermediate tensor respectively to obtain a first spatial attention tensor and a second spatial attention tensor. The first spatial attention tensor is multiplied element-wise with the important feature map to obtain the first spatial weighted feature map, and the second spatial attention tensor is multiplied element-wise with the minor feature map to obtain the second spatial weighted feature map. The first spatially weighted feature map and the second spatially weighted feature map are added pixel by pixel to obtain the output feature map corresponding to the input feature map of the spatial attention module itself.

5. The image enhancement method for underwater visual navigation and mapping according to claim 4, characterized in that, The important feature maps are subjected to average pooling and max pooling respectively, and then summed pixel by pixel to obtain a first feature map. The secondary feature maps are subjected to average pooling and max pooling respectively, and then summed pixel by pixel to obtain a second feature map. The first feature map and the second feature map are then subjected to a shared convolution to obtain a fourth intermediate tensor and a fifth intermediate tensor. The fourth intermediate tensor and the fifth intermediate tensor are then subjected to a second nonlinear operation to obtain a first spatial attention tensor and a second spatial attention tensor. This is achieved through the following formula: ； ； in, This represents the important feature map. This represents the secondary feature map. This indicates the average pooling. This indicates the max pooling. This represents the first feature map. This represents the second feature map. This represents a 7×7 two-dimensional convolution with 1 output channel and 1 input channel. This represents the fourth intermediate tensor. This represents the fifth intermediate tensor. This represents the second nonlinear operation, consisting of a BN layer and The activation functions are arranged in sequence. This represents the first spatial attention tensor. This represents the second spatial attention tensor; The first spatial attention tensor is multiplied element-wise with the important feature map to obtain a first spatially weighted feature map, and the second spatial attention tensor is multiplied element-wise with the minor feature map to obtain a second spatially weighted feature map. The first and second spatially weighted feature maps are then summed pixel-wise to obtain an output feature map corresponding to the input feature map of the spatial attention module itself. This is achieved through the following formula: ； in, This represents the first spatially weighted feature map. This represents the second spatially weighted feature map. This indicates that the pixel-by-pixel addition is performed. This indicates the element-wise multiplication. This represents the output feature map corresponding to the input feature map of the spatial attention module itself.

6. The image enhancement method for underwater visual navigation and mapping according to any one of claims 1 to 5, characterized in that, The discriminator includes a 3×3 max pooling branch, a 1×1 convolution branch, a 3×3 convolution branch, a 5×5 convolution branch, and a 3×3 average pooling branch. Each branch is followed by a 3×3 dilated convolution. The dilation coefficients of the 3×3 dilated convolutions at the ends of different branches are different. Each 3×3 dilated convolution is connected to the same 1×1 aggregated convolution. The 1×1 aggregated convolution is then connected to a... Activation function layer; The 3×3 max pooling branch, the 1×1 convolution branch, the 3×3 convolution branch, and the 5×5 convolution branch are used to increase the discriminator's adaptability to different scales. The 1×1 convolutional branch is also used to limit the number of channels in the discriminator; The 3×3 average pooling branch is used to enhance the discriminator's learning of background features; The 3×3 dilated convolution with different dilation coefficients is used to expand the receptive field; The 1×1 aggregation convolution is used to aggregate the information of each branch and restore the number of channels to be the same as the input of the discriminator.

7. The image enhancement method for underwater visual navigation and mapping according to any one of claims 1 to 5, characterized in that, For the generator, the preset loss function is expressed by the following formula: ； ； ； ； ； in, Let be the loss function for the generator. , , The preset weighting coefficients for the loss term. For the aforementioned adversarial loss term, For the L1 loss term, For the color shift loss term, For the gradient loss term, This refers to the generator. This refers to the discriminator. Represents the mathematical expectation. Describing the L1 norm, Represents the average color difference function. Represents the center distance function, This represents the gradient operator in the horizontal direction. This represents the gradient operator in the vertical direction.

8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the image enhancement method for underwater visual navigation and mapping as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the image enhancement method for underwater visual navigation and mapping as described in any one of claims 1 to 7.

Images