Underwater image enhancement method for target detection

Abstract

The invention relates to the field of underwater image enhancement and machine vision detection, in particular to a target detection-oriented underwater image enhancement method, which comprises the following steps of: S1, acquiring an underwater image to be detected; s2, inputting an underwater image to be detected into the trained image enhancement model, and outputting an enhanced underwater image; during training, underwater content features are extracted through an underwater content encoder; feature enhancement of contextual information of the aggregation space is carried out through a grouping space attention module; the enhanced underwater content features and the corresponding gradient texture images are fused and then input into an underwater image enhancement generator for image generation; and S3, applying the enhanced underwater image output by the image enhancement model to underwater target detection. According to the method, the characterization capability of the model on edge and texture details can be enhanced, and the model can be better focused on a specific feature region, so that the underwater image enhancement effect is improved, and the accuracy of machine vision underwater target detection is improved in an auxiliary manner.

Description

Technical Field

[0001] The present invention relates to the field of underwater image enhancement and machine vision detection, and particularly to an underwater image enhancement method for target detection. Background Art

[0002] The acceleration of the global ocean exploration process has promoted the in-depth exploration and understanding of the deep sea by humans. In this process, researchers have been constantly committed to capturing high-quality underwater images to support a wide range of underwater applications, such as underwater robot navigation, underwater target detection, marine biology tracking, etc. However, the light propagation in the underwater environment is different from that on land. The absorption and scattering phenomena in water lead to a serious impact on the image quality. Therefore, underwater images usually have problems such as low contrast, blurriness, and color distortion. These factors not only affect the observation and research of underwater images by humans, but also pose higher requirements for subsequent underwater target detection tasks based on machine vision.

[0003] To address these challenges, researchers have developed various underwater image enhancement techniques aimed at correcting color deviations and restoring details. The existing underwater image enhancement methods can be roughly divided into image enhancement methods based on image processing, physical models, and deep learning. The methods based on image processing mainly improve the visual perception of the image by directly processing the image pixels to improve contrast, sharpness, and color. The image restoration methods based on physical models deduce the degradation model from the physical process, aiming to restore the quality of the image before degradation. In recent years, the methods of deep learning have developed rapidly. Among them, the Convolutional Neural Network (CNN) has achieved remarkable results in the field of underwater image enhancement by learning the mapping model between the degraded underwater image and the high-quality reference image and using its powerful feature representation and non-linear mapping capabilities. In addition, due to the difficulty of obtaining real underwater images, researchers have proposed methods based on the Generative Adversarial Network (GAN), which generate more realistic synthetic images by simulating the underwater degradation process.

[0004] However, most of the existing underwater image enhancement methods are mainly oriented to human visual perception, while the research on underwater image enhancement for machine vision detection tasks is relatively less. Moreover, the underwater image enhancement based on human visual perception may introduce problems such as noise interference, edge blurring, and texture damage, seriously affecting the performance of the detector. In addition, the overprocessing of image attributes (such as contrast, saturation, and color richness) may also lead to a decline in the detector performance. Therefore, how to design an underwater image enhancement method that is more suitable for the requirements of machine vision tasks to avoid the loss of target information caused by overprocessing of the image and affect the detector performance has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In view of the deficiencies of the above-mentioned prior art, the technical problem to be solved by the present invention is: how to provide an underwater image enhancement method for object detection, construct a network framework that combines gradient information to guide the generator and grouped spatial attention, enhance the model's ability to represent edges and texture details, and better focus on specific feature regions, so as to improve the effect of underwater image enhancement and assist in improving the accuracy of underwater object detection in machine vision.

[0006] To solve the above technical problems, the present invention adopts the following technical solutions:

[0007] An underwater image enhancement method for object detection, comprising:

[0008] S1: Obtain the underwater image to be detected;

[0009] S2: Input the underwater image to be detected into the trained image enhancement model, and output the enhanced underwater image;

[0010] The processing steps during the training of the image enhancement model are as follows:

[0011] S201: Obtain the real underwater image and the real aerial image as training data; calculate the gradient texture map of the real underwater image;

[0012] S202: Extract the underwater content features of the real underwater image through the underwater content encoder; input the underwater content features into the grouped spatial attention module for feature enhancement of aggregating spatial context information to obtain enhanced underwater content features; fuse the enhanced underwater content features with their corresponding gradient texture maps and input them into the underwater image enhancement generator to generate a simulated aerial image;

[0013] S203: Extract the underwater distortion features of the real underwater image through the distortion encoder; extract the aerial content features of the real aerial image through the aerial content encoder; fuse the underwater distortion features and the aerial content features and input them into the aerial image synthesis generator to generate a simulated underwater image;

[0014] S204: Extract the underwater distortion features of the simulated underwater image through the distortion encoder; extract the aerial content features of the simulated aerial image through the aerial content encoder; fuse the underwater distortion features and the aerial content features and input them into the aerial image synthesis generator to generate a reconstructed underwater image;

[0015] S205: Extract the underwater content features of the simulated underwater image through the underwater content encoder; input the underwater content features into the underwater image enhancement generator to obtain a reconstructed aerial image;

[0016] S206: Calculate the loss function based on the real underwater images, real aerial images, simulated aerial images, simulated underwater images, reconstructed underwater images, and reconstructed aerial images;

[0017] S207: Based on the loss function, inversely optimize the parameters of the underwater content encoder, grouped spatial attention module, underwater image enhancement generator, distortion encoder, aerial content encoder, and aerial image synthesis generator;

[0018] S208: Repeat steps S201 to S207 for iterative training until the model converges or reaches the preset number of iterations;

[0019] After the image enhancement model is trained, calculate the gradient texture map of the underwater image to be detected; extract the underwater content features of the underwater image to be detected through the underwater content encoder; input the underwater content features into the grouped spatial attention module for feature enhancement to obtain enhanced underwater content features; fuse the enhanced underwater content features with their corresponding gradient texture maps and then input them into the underwater image enhancement generator to generate enhanced underwater images;

[0020] S3: Apply the enhanced underwater images output by the image enhancement model to object detection.

[0021] Preferably, in step S201, calculate the gradient texture map of the underwater image through the following steps:

[0022] S2011: Decompose the underwater image into three channels of R, G, and B;

[0023] S2012: Calculate the horizontal gradient v c and vertical gradient h c ;

[0024] S2013: Calculate the corresponding gradient magnitude according to the horizontal gradient v c and vertical gradient h c of each channel;

[0025] The formula is expressed as:

[0026]

[0027] where: c ∈ {R, G, B}; β is set to 1e-6;

[0028] S2014: Combine the gradient magnitudes of the three channels to generate the gradient texture map.

[0029] Preferably, in step S202, the underwater content encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, a fourth scale dense block, and a latent feature extraction layer connected end to end in sequence;

[0030] The first convolutional module includes a convolutional layer and a Tanh function connected end to end;

[0031] The dense block includes four second convolutional modules connected end to end in sequence; the input of the first second convolutional module is the input of the dense block, and the concatenation of the input and output of the first second convolutional module serves as the input of the second second convolutional module; the concatenation of the input and output of the first second convolutional module and the output of the second second convolutional module serves as the input of the third second convolutional module; the concatenation of the input and output of the first second convolutional module, the output of the second second convolutional module, and the output of the third second convolutional module serves as the input of the fourth second convolutional module; the output of the fourth second convolutional module is element-wise added to the input of the first second convolutional module to serve as the output of the dense block;

[0032] The second convolutional module includes a convolutional layer and a ReLU function connected end to end;

[0033] The outputs of the first-scale dense block, second-scale dense block, third-scale dense block, and fourth-scale dense block of the underwater content encoder are the first-scale underwater content features, second-scale underwater content features, third-scale underwater content features, and fourth-scale underwater content features respectively.

[0034] Preferably, in step S202, the four-scale underwater content features output by the four-scale dense blocks in the underwater content encoder are respectively input into four grouped spatial attention modules to obtain four-scale enhanced underwater content features;

[0035] After the gradient texture map is input into the second convolutional module, the output of the second convolutional module is element-wise added to the four-scale enhanced underwater content features to obtain gradient-enhanced features.

[0036] Preferably, in step S202, the underwater image enhancement generator includes a fourth-scale dense block, a third-scale dense block, a second-scale dense block, a first-scale dense block, a first convolutional module, and a third convolutional module connected end to end in sequence;

[0037] The third convolutional module includes a convolutional layer, an IN layer, and a ReLU function connected end to end;

[0038] The latent features extracted by the latent feature extraction layer of the underwater content encoder are concatenated with the fourth-scale underwater content features and serve as the input of the fourth-scale dense block of the underwater image enhancement generator;

[0039] The output of the fourth-scale dense block of the underwater image enhancement generator is concatenated with the third-scale underwater content features and serves as the input of the third-scale dense block of the underwater image enhancement generator;

[0040] The output of the third-scale dense block of the underwater image enhancement generator is concatenated with the second-scale underwater content features and used as the input of the second-scale dense block of the underwater image enhancement generator;

[0041] The output of the second-scale dense block of the underwater image enhancement generator is concatenated with the first-scale underwater content features and used as the input of the first-scale dense block of the underwater image enhancement generator;

[0042] The output of the first-scale dense block of the underwater image enhancement generator is element-wise added to the gradient enhancement features and used as the input of the first convolutional module;

[0043] The output of the first convolutional module is used as the input of the third convolutional module, and the output of the third convolutional module is the output of the underwater image enhancement generator.

[0044] Preferably, in step S202, the processing steps of the grouped spatial attention module include:

[0045] S2021: Perform max-pooling and average-pooling on the input underwater content features with a size of C×H×W, and then constrain them within the range of [0,1] through the Sigmoid function to obtain the spatial attention map S;

[0046] S2022: Divide the input underwater content features along the channel dimension into N groups of local features to obtain feature groups;

[0047] The formula is expressed as:

[0048]

[0049] In the formula: x i is the i-th dimensional local feature in the feature group, and R represents the set of real numbers;

[0050] S2023: Multiply the spatial attention map S with each group of local features element-wise to obtain the feature map corresponding to each group of local features

[0051] The formula is expressed as:

[0052]

[0053] S2024: Perform a linear transformation on the feature map corresponding to each group of local features using the learnable weight ω and bias α to obtain the new features after enhancing each group of local features;

[0054] The formula is expressed as:

[0055]

[0056] S2025: Feature filtering is performed through the ReLU activation function;

[0057] S2026: Integrate all enhanced new features into a new feature group, that is, enhance the underwater content features;

[0058] The formula is expressed as:

[0059]

[0060] Preferably, in step S203, the distortion encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, and a fourth scale dense block connected end to end in sequence;

[0061] The first convolutional module includes a convolutional layer and a Tanh function connected end to end;

[0062] The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the distortion encoder are the first scale underwater distortion feature, the second scale underwater distortion feature, the third scale underwater distortion feature, and the fourth scale underwater distortion feature respectively.

[0063] Preferably, in step S203, the aerial content encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, a fourth scale dense block, and a latent feature extraction layer connected end to end in sequence;

[0064] The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the aerial content encoder are the first scale aerial content feature, the second scale aerial content feature, the third scale aerial content feature, and the fourth scale aerial content feature respectively.

[0065] Preferably, in step S203, the aerial image synthesis generator includes a fourth scale dense block, a third scale dense block, a second scale dense block, a first scale dense block, a first convolutional module, and a third convolutional module connected end to end in sequence;

[0066] The latent features extracted by the latent feature extraction layer of the aerial content encoder are added element by element to the fourth scale underwater distortion feature and then concatenated with the fourth scale aerial content feature as the input of the fourth scale dense block of the aerial image synthesis generator;

[0067] The output of the fourth scale dense block of the aerial image synthesis generator is added element by element to the third scale underwater distortion feature and then concatenated with the third scale aerial content feature as the input of the third scale dense block of the aerial image synthesis generator;

[0068] The output of the third-scale dense block of the aerial image synthesis generator is element-wise added to the second-scale underwater distortion features and then concatenated with the second-scale aerial content features as the input of the second-scale dense block of the aerial image synthesis generator;

[0069] The output of the second-scale dense block of the aerial image synthesis generator is element-wise added to the first-scale underwater distortion features and then concatenated with the first-scale aerial content features as the input of the first-scale dense block of the aerial image synthesis generator;

[0070] The output of the first-scale dense block of the aerial image synthesis generator serves as the input of the first convolutional module;

[0071] The output of the first convolutional module serves as the input of the third convolutional module, and the output of the third convolutional module is the output of the third convolutional module.

[0072] Preferably, in step S206, the loss function is calculated through the following steps:

[0073] S2061: Calculate the cycle consistency loss function based on the reconstructed underwater image, the reconstructed aerial image, the real underwater image, and the real aerial image;

[0074] The formula is expressed as:

[0075] L cyc = ||I RU - I U ||1 + ||I RA - I A ||1;

[0076] In the formula: L cyc represents the cycle consistency loss function; I RU represents the reconstructed underwater image; I U represents the real underwater image; I RA represents the reconstructed aerial image; I A represents the real aerial image; ||·||1 represents calculating the L1 norm;

[0077] S2062: Calculate the adversarial loss function between the simulated underwater image and the real underwater image through the discriminator;

[0078] The formula is expressed as:

[0079]

[0080] In the formula: L adv represents the adversarial loss function; D(·) represents the discriminator, which is used to distinguish the real underwater image and the simulated underwater image;

[0081] S2063: Input the real aerial image into the aerial content encoder and the distortion encoder to extract the content feature and the distortion feature respectively; splice the content feature and the distortion feature of the real aerial image and input them into the aerial image synthesis generator to obtain the aerial image identity mapping; calculate the difference loss function based on the aerial image identity mapping and the real aerial image;

[0082] The formula is expressed as:

[0083] L idt = || GB (E A′ (I A ), E U (I A )) - I A ||1;

[0084] In the formula: L idt represents the difference loss function; G B (E A′ (I A ), E U (I A )) represents the aerial image identity mapping; E U (·) represents the distortion encoder; E A′ (·) represents the aerial content encoder; G B (·) represents the aerial image synthesis generator;

[0085] S2064: Extract the semantic perception features of the real underwater image, the real aerial image, the simulated aerial image, and the simulated underwater image, and calculate the semantic feature loss function based on the semantic perception features;

[0086] The formula is expressed as:

[0087]

[0088] In the formula: L con represents the semantic feature loss function; I′ FA 、I′ FU 、I′ U 、I′ A are the semantic perception features of the simulated aerial image I FA 、the simulated underwater image I FU 、the real underwater image I U 、and the real aerial image I A respectively;

[0089] S2065: For the cycle consistency loss function L cyc 、the adversarial loss function L adv 、the difference loss function L idt and the semantic feature loss function L conPerform weighted summation to obtain the loss function of the image enhancement model;

[0090] The formula is expressed as:

[0091] L all = a·L cyc + b·L adv + c·L idt + d·L con ;

[0092] In the formula: L all represents the loss function of the image enhancement model; a, b, c, and d represent the set weights, where a = 10, b = 1, c = 5, and d = 5.

[0093] Compared with the prior art, the underwater image enhancement method for object detection in the present invention has the following beneficial effects:

[0094] Aiming at the problem of the decline in the accuracy of underwater object detection caused by the lack of image edge information and the destruction of the target texture structure, the present invention proposes an underwater image enhancement framework for object detection, and constructs a network framework combining a gradient information-guided generator and grouped spatial attention. First, when training the model, the underwater content features and the gradient texture map are fused and then input into the underwater image enhancement generator for image generation. By calculating the gradient texture map, the gradient information of the underwater image is extracted and strengthened, guiding the generator to retain the edges and textures of the underwater image, enhancing the model's representation ability of edge and texture details, improving the problems of blurred object contours and missing details, ensuring the effect of underwater image enhancement, and thus assisting in improving the accuracy of machine vision underwater object detection. Second, during training, the grouped spatial attention module (GSA) adaptively assigns higher weights to the key regions of the underwater image by aggregating spatial context information, can perceive and highlight the significant features of the target region, suppress irrelevant background information, and then achieve the enhancement of underwater content features, enabling the model to better focus on specific feature regions, thereby further improving the effect of underwater image enhancement, and also being able to assist in improving the accuracy of machine vision object detection. Finally, during the training process, a self-supervised image conversion training process based on the generative adversarial network (GAN) is adopted, aiming to achieve style transfer between underwater images and aerial images, and improve the performance of the image enhancement model. Traditional methods rely on pairs of underwater images and their corresponding "clear aerial images", while the training framework of the present invention does not require paired data, so that the model can be efficiently trained in a self-supervised environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] In order to make the purpose, technical solutions, and advantages of the invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings, where:

[0096] Figure 1It is a logic block diagram of an underwater image enhancement method for object detection.

[0097] Figure 2 It is a network structure diagram of an image enhancement model. Specific implementation manners

[0098] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Components of the embodiments of the present invention generally described and illustrated in the drawings herein can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0099] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the inventive product is usually placed during use. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance. In addition, terms such as "horizontal" and "vertical" do not mean that the components are required to be absolutely horizontal or hanging, but can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined. In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "set", "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and can be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0100] The following is a further detailed description through specific implementation manners:

[0101] Example:

[0102] This example discloses an underwater image enhancement method for target detection.

[0103] As Figure 1 shown, an underwater image enhancement method for target detection includes:

[0104] S1: Obtain the underwater image to be detected;

[0105] S2: Input the underwater image to be detected into the trained image enhancement model, and output the enhanced underwater image;

[0106] The processing steps during training the image enhancement model are as follows:

[0107] S201: Obtain the real underwater image I U and the real aerial image I A ; Calculate the gradient texture map I of the real underwater image G ;

[0108] S202: Extract the underwater content features of the real underwater image I A through the underwater content encoder E; Input the underwater content features into the Group Spatial Attention (GSA) module for feature enhancement of aggregated spatial context information to obtain enhanced underwater content features; After fusing the enhanced underwater content features and the gradient texture map I U input them into the underwater image enhancement generator G G to generate the simulated aerial image I A ; FA ;

[0109] S203: Extract the underwater distortion features of the real underwater image I U through the distortion encoder E; Extract the aerial content features of the real aerial image I U through the aerial content encoder E; After fusing the underwater distortion features and the aerial content features, input them into the aerial image synthesis generator G A′ to generate the simulated underwater image I A ; B ; FU ;

[0110] S204: Extract the underwater distortion features of the simulated underwater image I U through the distortion encoder E; Extract the aerial content features of the simulated aerial image I FU through the aerial content encoder E; Extract the aerial content features of the simulated aerial image I A′ through the aerial content encoder E; FAUnderwater distortion features and aerial content features; after fusing the underwater distortion features and aerial content features, input them into the aerial image synthesis generator G B to generate the reconstructed underwater image I RU ;

[0111] S205: Through the underwater content encoder E A Extract the underwater content features of the simulated underwater image I FU ; input the underwater content features into the underwater image enhancement generator G A to obtain the reconstructed aerial image I RA ;

[0112] S206: Based on the real underwater image I U , real aerial image I A , simulated aerial image I FA , simulated underwater image I FU , reconstructed underwater image I RU and reconstructed aerial image I RA calculate the loss function;

[0113] S207: Based on the loss function, reverse-optimize the parameters of the underwater content encoder E A , grouped spatial attention module, underwater image enhancement generator G A , distortion encoder E U , aerial content encoder E A′ , aerial image synthesis generator G B ;

[0114] S208: Repeat steps S201 to S207 for iterative training until the model converges or reaches the preset number of iterations;

[0115] After the image enhancement model training is completed, calculate the gradient texture map of the underwater image to be detected; extract the underwater content features of the underwater image to be detected through the underwater content encoder; input the underwater content features into the grouped spatial attention module for feature enhancement to obtain enhanced underwater content features; fuse the enhanced underwater content features with their corresponding gradient texture maps and input them into the underwater image enhancement generator to generate an enhanced underwater image;

[0116] S3: Apply the enhanced underwater image output by the image enhancement model to underwater target detection.

[0117] In this embodiment, use the trained YOLOv5 model as the target detection model to perform target detection on the enhanced underwater image to obtain the corresponding target detection results.

[0118] Aiming at the problem of the decline in the accuracy of underwater target detection caused by the lack of image edge information and the damage of the target texture structure, the present invention proposes an underwater image enhancement framework for target detection, and constructs a network framework combining a gradient information-guided generator and grouped spatial attention. First, when training the model, the underwater content features and the gradient texture map are fused and then input into the underwater image enhancement generator for image generation. By calculating the gradient texture map, the gradient information of the underwater image is extracted and strengthened, guiding the generator to retain the edges and textures of the underwater image, enhancing the model's representation ability for edge and texture details, improving the problems of blurred target contours and missing details, ensuring the effect of underwater image enhancement, and thus assisting in improving the accuracy of machine vision underwater target detection. Second, during training, the grouped spatial attention module (GSA) adaptively assigns higher weights to the key regions of the underwater image by aggregating spatial context information, can perceive and highlight the significant features of the target region, suppress irrelevant background information, and then achieve the enhancement of underwater content features, enabling the model to better focus on specific feature regions, thereby further improving the effect of underwater image enhancement, and also being able to assist in improving the accuracy of machine vision target detection. Finally, during the training process, a self-supervised image conversion training process based on the generative adversarial network (GAN) is adopted, aiming to achieve the style transfer between underwater images and aerial images, and improve the performance of the image enhancement model. Traditional methods rely on pairs of underwater images and their corresponding "clear aerial images", while the training framework of the present invention does not require paired data, thus enabling efficient training of the model in a self-supervised environment.

[0119] In this embodiment, the method proposed by the present invention is evaluated on multiple underwater image datasets. The experimental results on the ChinaMM dataset show that the method of the present invention reaches 98.3% on mAP@0.5 and 78.7% on mAP@0.5:0.95, significantly improving the accuracy of target detection, and the overall performance is better than other comparison methods.

[0120] To better introduce the technical solution of the present invention, this embodiment is described in the following several parts.

[0121] I. Gradient texture map

[0122] The gradient texture map plays a key role in the process of underwater image enhancement. The gradient information of the gradient texture map can accurately represent the change rate of pixel intensity or color in the underwater image. Especially in the edge region, the gradient amplitude is often large, reflecting significant texture features. By calculating the gradient information of the input underwater image, it can guide the underwater image enhancement generator to pay more attention to the detail regions during the image enhancement process, especially the improvement of edge and texture information.

[0123] Specifically, the gradient texture map of the real underwater image is calculated through the following steps:

[0124] S2011: Decompose the underwater image into three channels: red (R), green (G), and blue (B);

[0125] S2012: Calculate the horizontal gradient v and vertical gradient h of each channel in the horizontal and vertical directions through two 3×3 Sobel operators c and vertical gradient h c ; The Sobel operator can enhance the network's sensitivity to edge details and provide more discriminative feature information for subsequent image processing tasks.

[0126] S2013: Calculate the corresponding gradient magnitude according to the horizontal gradient v c and vertical gradient h c ;

[0127] The formula is expressed as:

[0128]

[0129] where: c ∈ {R, G, B}; To avoid division-by-zero errors during calculation, β is set to 1e-6;

[0130] S2014: Combine the gradient magnitudes of the three channels to generate a gradient texture map.

[0131] By fusing the underwater content features and the gradient texture map, the present invention provides guidance for detail enhancement to the underwater image enhancement generator, enabling the generator to more accurately restore the texture and edges in the underwater image, thereby significantly improving the overall quality of the underwater image.

[0132] II. Underwater Content Encoder

[0133] As shown in combination Figure 2 , the underwater content encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, a fourth scale dense block, and a latent feature extraction layer that are sequentially connected end to end; among them, the shallow dense blocks mainly capture low-level information, such as color and texture, etc.; the deep dense blocks focus on the extraction of high-level information, such as semantic information, etc.

[0134] The first convolutional module includes a convolutional layer and a Tanh function connected end to end;

[0135] The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the underwater content encoder are the first scale (1 / 2) underwater content features, the second scale (1 / 4) underwater content features, the third scale (1 / 8) underwater content features, and the fourth scale (1 / 16) underwater content features, respectively.

[0136] The dense block includes four second convolutional modules connected end to end in sequence; the input of the first second convolutional module is the input of the dense block, and the concatenation of the input and output of the first second convolutional module serves as the input of the second second convolutional module; the concatenation of the input of the first second convolutional module, the output of the first second convolutional module, and the output of the second second convolutional module serves as the input of the third second convolutional module; the concatenation of the input of the first second convolutional module, the output of the first second convolutional module, the output of the second second convolutional module, and the output of the third second convolutional module serves as the input of the fourth second convolutional module; the output of the fourth second convolutional module is element-wise added to the input of the first second convolutional module to obtain the output of the dense block;

[0137] The second convolutional module includes a convolutional layer and a ReLU function connected end to end.

[0138] III. Underwater Image Enhancement Generator

[0139] Combined with Figure 2 As shown, the underwater content features at four scales output by the four dense blocks in the underwater content encoder are respectively input into four grouped spatial attention modules to obtain enhanced underwater content features at four scales;

[0140] After the gradient texture map is input into the second convolutional module, the output of the second convolutional module is element-wise added to the enhanced underwater content features at four scales to obtain gradient-enhanced features.

[0141] Combined with Figure 2 As shown, the underwater image enhancement generator includes a fourth-scale dense block, a third-scale dense block, a second-scale dense block, a first-scale dense block, a first convolutional module, and a third convolutional module connected end to end in sequence;

[0142] The third convolutional module includes a convolutional layer, an IN layer, and a ReLU function connected end to end;

[0143] The latent features extracted by the latent feature extraction layer of the underwater content encoder are concatenated with the fourth-scale underwater content features and used as the input of the fourth-scale dense block of the underwater image enhancement generator;

[0144] The output of the fourth-scale dense block of the underwater image enhancement generator is concatenated with the third-scale underwater content features and used as the input of the third-scale dense block of the underwater image enhancement generator;

[0145] The output of the third-scale dense block of the underwater image enhancement generator is concatenated with the second-scale underwater content features and used as the input of the second-scale dense block of the underwater image enhancement generator;

[0146] The output of the second-scale dense block of the underwater image enhancement generator is concatenated with the first-scale underwater content features and used as the input of the first-scale dense block of the underwater image enhancement generator;

[0147] The output of the first-scale dense block of the underwater image enhancement generator is added element-wise to the gradient-enhanced features and used as the input to the first convolutional module;

[0148] The output of the first convolutional module is used as the input to the third convolutional module, and the output of the third convolutional module is the output of the underwater image enhancement generator.

[0149] IV. Group Spatial Attention Module (hereinafter referred to as GSA)

[0150] To improve the model's focusing ability on regions of interest, GSA is designed to group the feature maps at different downsampling stages in the encoder and adaptively weight and adjust each group of feature maps. In addition to grouping the channel dimension to represent different semantic sub-features, another dimension in the convolutional feature map - space - is further considered. GSA enhances feature representation by aggregating spatial context information, enabling the model to better focus on specific feature regions. At the same time, GSA dynamically adjusts the influence of each group through learnable weights and biases. In this way, the model can automatically optimize these parameters during training, thereby effectively learning more discriminative and useful features.

[0151] Specifically, the processing steps of the group spatial attention module include:

[0152] S2021: After performing max-pooling and average-pooling on the input underwater content features (image) of size C×H×W, and constraining them within the range [0,1] through the Sigmoid function, a spatial attention map S is obtained;

[0153] S2022: The input underwater content features (image) are divided into N groups of local features along the channel dimension to obtain feature groups;

[0154] The formula is expressed as:

[0155]

[0156] where: x i is the i-th dimensional local feature in the feature group, and R represents the set of real numbers;

[0157] S2023: The spatial attention map S is multiplied element-wise with each group of local features to enhance the features at important positions, obtaining the feature maps corresponding to each group of local features

[0158] The formula is expressed as:

[0159]

[0160] S2024: Use the learnable weights ω and biases α to perform a linear transformation on the feature maps corresponding to each group of local features to obtain new features with enhanced local features for each group;

[0161] The formula is expressed as:

[0162]

[0163] S2025: Filter out noise or unimportant new features through the ReLU activation function;

[0164] S2026: Integrate all the enhanced new features into a new feature group, namely the enhanced underwater content feature (figure);

[0165] The formula is expressed as:

[0166]

[0167] V. Distortion Encoder

[0168] Combined Figure 2 as shown, the distortion encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, and a fourth scale dense block connected end to end in sequence;

[0169] The first convolutional module includes a convolutional layer and a Tanh function connected end to end;

[0170] The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the distortion encoder are the first scale underwater distortion feature, the second scale underwater distortion feature, the third scale underwater distortion feature, and the fourth scale underwater distortion feature, respectively.

[0171] VI. Aerial Content Encoder

[0172] Combined Figure 2 as shown, the aerial content encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, a fourth scale dense block, and a latent feature extraction layer connected end to end in sequence;

[0173] The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the aerial content encoder are the first scale aerial content feature, the second scale aerial content feature, the third scale aerial content feature, and the fourth scale aerial content feature, respectively.

[0174] VII. Aerial Image Synthesis Generator

[0175] Combined Figure 2As shown in the figure, the aerial image synthesis generator includes a fourth-scale dense block, a third-scale dense block, a second-scale dense block, a first-scale dense block, a first convolutional module, and a third convolutional module that are connected end to end in sequence;

[0176] The latent features extracted by the latent feature extraction layer of the aerial content encoder are added element-wise to the fourth-scale underwater distortion features and then concatenated with the fourth-scale aerial content features as the input of the fourth-scale dense block of the aerial image synthesis generator;

[0177] The output of the fourth-scale dense block of the aerial image synthesis generator is added element-wise to the third-scale underwater distortion features and then concatenated with the third-scale aerial content features as the input of the third-scale dense block of the aerial image synthesis generator;

[0178] The output of the third-scale dense block of the aerial image synthesis generator is added element-wise to the second-scale underwater distortion features and then concatenated with the second-scale aerial content features as the input of the second-scale dense block of the aerial image synthesis generator;

[0179] The output of the second-scale dense block of the aerial image synthesis generator is added element-wise to the first-scale underwater distortion features and then concatenated with the first-scale aerial content features as the input of the first-scale dense block of the aerial image synthesis generator;

[0180] The output of the first-scale dense block of the aerial image synthesis generator is used as the input of the first convolutional module;

[0181] The output of the first convolutional module is used as the input of the third convolutional module, and the output of the third convolutional module is the output of the third convolutional module.

[0182] VIII. Loss Function

[0183] In this embodiment, the loss function is calculated through the following steps:

[0184] S2061: To reduce the space of possible mapping functions, it is proposed that the learned mapping function should be cycle-consistent. Therefore, the cycle-consistency loss function is calculated based on the reconstructed underwater image, the reconstructed aerial image, the real underwater image, and the real aerial image;

[0185] The formula is expressed as:

[0186] L cyc = ||I RU - I U ||1 + ||I RA - I A ||1;

[0187] In the formula: L cyc represents the cycle-consistency loss function; I RUDenote the reconstructed underwater image as \(I\); U Denote the real underwater image as \(I\); RA Denote the reconstructed aerial image as \(I\); A Denote the real aerial image as \(||·||_1\) represents calculating the \(L_1\) norm;

[0188] S2062: Calculate the adversarial loss function between the simulated underwater image and the real underwater image through the discriminator;

[0189] The formula is expressed as:

[0190]

[0191] In the formula: \(L\) adv represents the adversarial loss function; \(D(·)\) represents the discriminator, which is used to distinguish between the real underwater image and the simulated underwater image;

[0192] S2063: Input the real aerial image into the aerial content encoder and the distortion encoder respectively to extract the content feature and the distortion feature; splice the content feature and the distortion feature of the real aerial image and input them into the aerial image synthesis generator to obtain the aerial image identity mapping; calculate the difference loss function based on the aerial image identity mapping and the real aerial image;

[0193] The formula is expressed as:

[0194] \(L\) idt = || \(G\) B (\(E\) A′ (\(I\) A ), \(E\) U (\(I\) A )) - \(I\) A ||_1;

[0195] In the formula: \(L\) idt represents the difference loss function; \(G\) B (\(E\) A ′(\(I\) A ), \(E\) U (\(I\) A )) represents the aerial image identity mapping; \(E\) U (·) represents the distortion encoder; \(E\) A′ (·) represents the aerial content encoder; \(G\) B (·) represents the aerial image synthesis generator;

[0196] S2064: Extract the semantic perception features of the real underwater image, the real aerial image, the simulated aerial image, and the simulated underwater image through the PFC module, and calculate the semantic feature loss function based on the semantic perception features;

[0197] The formula is expressed as:

[0198]

[0199] Where: L con represents the semantic feature loss function; I′ FA 、I′ FU 、I′ U 、I′ A are respectively the semantic perception features extracted from the simulated aerial image I FA 、the simulated underwater image I FU 、the real underwater image I U 、the real aerial image I A through the PFC module;

[0200] In this embodiment, the PFC module represents a semantic perception feature extractor, and the features it extracts have the same resolution as the input image to retain rich detail information.

[0201] S2065: Perform a weighted sum of the cycle consistency loss function L cyc 、the adversarial loss function L adv 、the difference loss function L idt and the semantic feature loss function L con to obtain the loss function of the image enhancement model;

[0202] The formula is expressed as:

[0203] L all = a·L cyc + b·L adv + c·L idt + d·L con ;

[0204] Where: L all represents the loss function of the image enhancement model; a, b, c, d represent the set weights, where a = 10, b = 1, c = 5, d = 5.

[0205] IX. Experimental Explanation

[0206] To better illustrate the advantages of the technical solution of the present invention, the following experiments are disclosed in this embodiment.

[0207] 1. Experimental Details

[0208] This experiment is implemented based on PyTorch and trained on a Windows workstation equipped with an NVIDIA GTX 4090 GPU. During the training process, the PatchGAN (from Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2017:1125-1134.) structure is used to generate the binary mapping. The Adam optimizer is selected, and all hyperparameters are default values. The entire training process is carried out for 150 rounds, in which the learning rate is set to 2×10 -4 , and gradually linearly decays to 0 in the last 75 rounds.

[0209] The dataset contains 1,634 unpaired samples. For underwater images, 1,634 images from the SUIM (from Islam M J, Edge C, Xiao Y, et al. Semantic segmentation of underwater imagery: Dataset and benchmark[C] / / 2020 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020:1769-1776.) dataset are used, and the size is changed to 256×256; for ground images, 1,634 high-quality images are randomly selected from the Flickr 2R (from Agustsson E, Timofte R. Ntire 2017 challenge on single image super-resolution: Dataset and study[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2017:126-135.) dataset and randomly cropped into 256×256 image patches.

[0210] 2. Dataset and Evaluation Metrics

[0211] For the underwater image detection task, we used ChinaMM (from Liu R, Fan X, Zhu M, et al. Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light[J]. IEEE transactions on circuits and systems for video technology, 2020, 30(12): 4861-4875.). The ChinaMM dataset includes 2071 training images and 676 test images, covering three underwater objects: sea urchins, scallops, and sea cucumbers. We adopted mean Average Precision (mAP) as the evaluation metric for object detection, and used mAP@0.5 and map@0.75 to quantify the average precision of various object categories.

[0212] 3. Underwater Object Detection Performance Evaluation

[0213] To study the impact of underwater image enhancement on object detection performance, the present invention applied the enhanced images to the detector. The present invention conducted experiments on the ChinaMM dataset and compared with various underwater image enhancement methods, including the physics-based methods UDCP, IBLA, the image processing-based method Bayesian, and the deep learning-based methods USUIR, TACL, X-CAUNET, TFUIE. Qualitative and quantitative methods were used for comparison. Table 1 lists the quantitative results on the ChinaMM dataset. In this study, YOLOv5 was used to evaluate the impact of the proposed method and the comparative methods on subsequent object detection performance. The results in Table 1 show that in the ChinaMM dataset, even though the existing methods have achieved high accuracy (mAP@0.5 is 98.2%, mAP@0.5:0.95 is 78.3%), the present invention still improved by 0.1% and 0.6% on mAP@0.5 and mAP@0.5:0.95 compared with the original images. The effectiveness of the method was verified.

[0214] Table 1 Object Detection Accuracy of the ChinaMM Dataset on YOLOv5 (The best results are marked in bold)

[0215]

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Those of ordinary skill in the art should understand that any modifications or equivalent replacements made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions should be covered within the scope of the claims of the present invention.

Claims

1. An underwater image enhancement method for object detection, characterized in that, Including: S1: Obtain the underwater image to be detected; S2: Input the underwater image to be detected into the trained image enhancement model, and output the enhanced underwater image; The processing steps during training the image enhancement model are as follows: S201: Obtain the real underwater image and the real aerial image as training data; Calculate the gradient texture map of the real underwater image; S202: Extract the underwater content features of the real underwater image through the underwater content encoder; Input the underwater content features into the grouped spatial attention module for feature enhancement of aggregated spatial context information to obtain enhanced underwater content features; Fuse the enhanced underwater content features with their corresponding gradient texture maps and input them into the underwater image enhancement generator to generate a simulated aerial image; S203: Extract the underwater distortion features of the real underwater image through the distortion encoder; Extract the aerial content features of the real aerial image through the aerial content encoder; Fuse the underwater distortion features and the aerial content features and input them into the aerial image synthesis generator to generate a simulated underwater image; S204: Extract the underwater distortion features of the simulated underwater image through the distortion encoder; Extract the aerial content features of the simulated aerial image through the aerial content encoder; Fuse the underwater distortion features and the aerial content features and input them into the aerial image synthesis generator to generate a reconstructed underwater image; S205: Extract the underwater content features of the simulated underwater image through the underwater content encoder; Input the underwater content features into the underwater image enhancement generator to obtain a reconstructed aerial image; S206: Calculate the loss function based on the real underwater image, the real aerial image, the simulated aerial image, the simulated underwater image, the reconstructed underwater image, and the reconstructed aerial image; S207: Based on the loss function, inversely optimize the parameters of the underwater content encoder, the grouped spatial attention module, the underwater image enhancement generator, the distortion encoder, the aerial content encoder, and the aerial image synthesis generator; S208: Repeat steps S201 to S207 for iterative training until the model converges or reaches the preset number of iterations; After the image enhancement model is trained, calculate the gradient texture map of the underwater image to be detected; Extract the underwater content features of the underwater image to be detected through the underwater content encoder; Input the underwater content features into the grouped spatial attention module for feature enhancement to obtain enhanced underwater content features; Fuse the enhanced underwater content features with their corresponding gradient texture maps and input them into the underwater image enhancement generator to generate an enhanced underwater image; S3: Apply the enhanced underwater image output by the image enhancement model to object detection.

2. The underwater image enhancement method for object detection according to claim 1, wherein: In step S201, the gradient texture map of the underwater image is calculated through the following steps: S2011: Decompose the underwater image into three channels of R, G, and B; S2012: Calculate the horizontal gradient vc and the vertical gradient hc of each channel in the horizontal and vertical directions through two 3×3 Sobel operators; S2013: Calculate the corresponding gradient magnitude according to the horizontal gradient vc and the vertical gradient hc of each channel; The formula is expressed as: where: c ∈ {R, G, B}; β is set to 1e-6; S2014: Combine the gradient magnitudes of the three channels to generate the gradient texture map.

3. The underwater image enhancement method for target detection according to claim 1, wherein: In step S202, the underwater content encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, a fourth scale dense block, and a latent feature extraction layer that are connected end to end in sequence; The first convolutional module includes a convolutional layer and a Tanh function connected end to end; The dense block includes four second convolutional modules connected end to end in sequence; the input of the first second convolutional module is the input of the dense block, and the input and output of the first second convolutional module are concatenated and used as the input of the second second convolutional module; the input of the first second convolutional module, the output of the first second convolutional module, and the output of the second second convolutional module are concatenated and used as the input of the third second convolutional module; the input of the first second convolutional module, the output of the first second convolutional module, the output of the second second convolutional module, and the output of the third second convolutional module are concatenated and used as the input of the fourth second convolutional module; the output of the fourth second convolutional module is element-wise added to the input of the first second convolutional module and used as the output of the dense block; The second convolutional module includes a convolutional layer and a ReLU function connected end to end; The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the underwater content encoder are the first scale underwater content feature, the second scale underwater content feature, the third scale underwater content feature, and the fourth scale underwater content feature, respectively.

4. The underwater image enhancement method for target detection according to claim 3, wherein: In step S202, the four scale underwater content features output by the four scale dense blocks in the underwater content encoder are respectively input into four grouped spatial attention modules to obtain four scale enhanced underwater content features; After the gradient texture map is input into the second convolutional module, the output of the second convolutional module is element-wise added to the four scale enhanced underwater content features to obtain the gradient enhanced feature.

5. The underwater image enhancement method for target detection according to claim 4, characterized in that: In step S202, the underwater image enhancement generator includes a fourth scale dense block, a third scale dense block, a second scale dense block, a first scale dense block, a first convolutional module, and a third convolutional module that are connected end to end in sequence; The third convolutional module includes a convolutional layer, an IN layer, and a ReLU function connected end to end; The latent feature extracted by the latent feature extraction layer of the underwater content encoder is concatenated with the fourth scale underwater content feature and used as the input of the fourth scale dense block of the underwater image enhancement generator; The output of the fourth scale dense block of the underwater image enhancement generator is concatenated with the third scale underwater content feature and used as the input of the third scale dense block of the underwater image enhancement generator; The output of the third scale dense block of the underwater image enhancement generator is concatenated with the second scale underwater content feature and used as the input of the second scale dense block of the underwater image enhancement generator; The output of the second scale dense block of the underwater image enhancement generator is concatenated with the first scale underwater content feature and used as the input of the first scale dense block of the underwater image enhancement generator; The output of the first scale dense block of the underwater image enhancement generator is element-wise added to the gradient enhanced feature and used as the input of the first convolutional module; The output of the first convolutional module is used as the input of the third convolutional module, and the output of the third convolutional module is the output of the underwater image enhancement generator.

6. The underwater image enhancement method for target detection according to claim 4, wherein: In step S202, the processing steps of the grouped spatial attention module include: S2021: After performing max pooling and average pooling on the input underwater content features with a size of C×H×W, and constraining them within the range of [0, 1] through the Sigmoid function, a spatial attention map S is obtained. S2022: The input underwater content features are divided into N groups of local features along the channel dimension to obtain feature groups. The formula is expressed as: where: x i is the i-th dimensional local feature in the feature group, and R represents the set of real numbers; S2023: Element-wise multiply the spatial attention map S with each group of local features to obtain the feature map corresponding to each group of local features The formula is expressed as: S2024: Perform a linear transformation on the feature maps corresponding to each group of local features using the learnable weights ω and biases a to obtain new features with enhanced local features for each group; ​ The formula is expressed as: S2025: Feature filtering is performed through the ReLU activation function. S2026: All enhanced new features are integrated into a new feature group, namely the enhanced underwater content features. The formula is expressed as:

7. The underwater image enhancement method for object detection according to claim 1, characterized in that: In step S203, the distortion encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, and a fourth scale dense block connected end to end in sequence. The first convolutional module includes a convolutional layer and a Tanh function connected end to end. The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the distortion encoder are the first scale underwater distortion features, the second scale underwater distortion features, the third scale underwater distortion features, and the fourth scale underwater distortion features respectively.

8. The underwater image enhancement method for object detection according to claim 7, wherein: In step S203, the aerial content encoder includes a first convolutional module, a first scale dense block, a second scale dense block, a third scale dense block, a fourth scale dense block, and a latent feature extraction layer connected end to end in sequence. The outputs of the first scale dense block, the second scale dense block, the third scale dense block, and the fourth scale dense block of the aerial content encoder are the first scale aerial content features, the second scale aerial content features, the third scale aerial content features, and the fourth scale aerial content features respectively.

9. The underwater image enhancement method for target detection according to claim 8, wherein: In step S203, the aerial image synthesis generator includes a fourth scale dense block, a third scale dense block, a second scale dense block, a first scale dense block, a first convolutional module, and a third convolutional module connected end to end in sequence. The latent features extracted by the latent feature extraction layer of the aerial content encoder are added element-wise to the fourth scale underwater distortion features and then concatenated with the fourth scale aerial content features as the input of the fourth scale dense block of the aerial image synthesis generator. The output of the fourth scale dense block of the aerial image synthesis generator is added element-wise to the third scale underwater distortion features and then concatenated with the third scale aerial content features as the input of the third scale dense block of the aerial image synthesis generator. The output of the third scale dense block of the aerial image synthesis generator is added element-wise to the second scale underwater distortion features and then concatenated with the second scale aerial content features as the input of the second scale dense block of the aerial image synthesis generator. The output of the second scale dense block of the aerial image synthesis generator is added element-wise to the first scale underwater distortion features and then concatenated with the first scale aerial content features as the input of the first scale dense block of the aerial image synthesis generator. The output of the first scale dense block of the aerial image synthesis generator serves as the input of the first convolutional module. The output of the first convolutional module serves as the input of the third convolutional module, and the output of the third convolutional module is the output of the third convolutional module.

10. The underwater image enhancement method for target detection according to claim 1, characterized in that: In step S206, the loss function is calculated through the following steps: S2061: Calculate the cycle consistency loss function based on the reconstructed underwater image, the reconstructed aerial image, the real underwater image, and the real aerial image; The formula is expressed as: L cyc = ||I RU -I U ||1 + ||I RA -I A ||1; Where: L cyc represents the cyclic consistency loss function; I RU represents the reconstructed underwater image; I U represents the real underwater image; I RA represents the reconstructed aerial image; I A represents the real aerial image; ||·||1 represents calculating the L1 norm; S2062: Calculate the adversarial loss function between the simulated underwater image and the real underwater image through the discriminator; The formula is expressed as: where: l adv represents the adversarial loss function; D(·) represents the discriminator, which is used to distinguish between real underwater images and simulated underwater images; S2063: Input the real aerial image into the aerial content encoder and the distortion encoder to extract the content feature and the distortion feature respectively; Concatenate the content feature and the distortion feature of the real aerial image and input them into the aerial image synthesis generator to obtain the aerial image identity mapping; calculate the difference loss function based on the aerial image identity mapping and the real aerial image; The formula is expressed as: L idt =||G B (E A′ (I A ),E U (I A ))-I A ||1; where: l idt represents the difference loss function; G B (E A′ (I A ), E U (I A )) represents the aerial image identity mapping; E U (·) represents the distortion encoder; E A′ (·) represents the aerial content encoder; G B (·) represents the aerial image synthesis generator; S2064: Extract the semantic perception features of the real underwater image, the real aerial image, the simulated aerial image, and the simulated underwater image, and calculate the semantic feature loss function based on the semantic perception features; The formula is expressed as: where: L con represents the semantic feature loss function; I′ FA 、I′ FU 、I′ U 、I′ A are the semantic perception features of the simulated aerial image I FA 、the simulated underwater image I FU 、the real underwater image I U 、the real aerial image I A respectively; S2065: Perform weighted summation on the cycle consistency loss function \(L\) cyc , the adversarial loss function \(l\) adv , the difference loss function \(l\) idt and the semantic feature loss function \(l\) con to obtain the loss function of the image enhancement model; The formula is expressed as: l all = a·l cyc + b·l adv + c·l idt + d·l con ; Where: l all represents the loss function of the image enhancement model; a, b, c, and d represent the set weights, where a = 10, b = 1, c = 5, and d = 5.

Images